* Re: [net-next PATCH 6/8] mlx4: Add support for inner IPv6 checksum offloads and TSO
From: Tariq Toukan @ 2016-04-28 13:26 UTC (permalink / raw)
To: Alexander Duyck, Saeed Mahameed
Cc: Alex Duyck, Tal Alon, Linux Kernel Network Developers,
David Miller, Gal Pressman, Or Gerlitz, Eran Ben Elisha
In-Reply-To: <5720FF84.9020404@gmail.com>
On 27/04/2016 9:05 PM, Alexander Duyck wrote:
> On 04/27/2016 08:39 AM, Tariq Toukan wrote:
>>
>>
>> On 27/04/2016 12:01 AM, Alexander Duyck wrote:
>>> On Tue, Apr 26, 2016 at 1:23 PM, Saeed Mahameed
>>> <saeedm@dev.mellanox.co.il> wrote:
>>>> On Tue, Apr 26, 2016 at 6:50 PM, Alex Duyck <aduyck@mirantis.com>
>>>> wrote:
>>>>> The setup is pretty straight forward. Basically I left the first
>>>>> port
>>>>> in the default namespace and moved the second int a secondary
>>>>> namespace referred to below as $netns. I then assigned the IPv6
>>>>> addresses fec0::10:1 and fec0::10:2. After that I ran the following:
>>>>>
>>>>> VXLAN=vx$net
>>>>> echo $VXLAN ${test_options[$i]}
>>>>> ip link add $VXLAN type vxlan id $net \
>>>>> local fec0::10:1 remote $addr6 dev $PF0 \
>>>>> ${test_options[$i]} dstport `expr 8800 + $net`
>>>>> ip netns exec $netns ip link add $VXLAN type vxlan id $net \
>>>>> local $addr6 remote fec0::10:1
>>>>> dev $port \
>>>>> ${test_options[$i]} dstport `expr
>>>>> 8800 + $net`
>>>>> ifconfig $VXLAN 192.168.${net}.1/24
>>>>> ip netns exec $netns ifconfig $VXLAN 192.168.${net}.2/24
>>>>>
>>>> Thanks, indeed i see that GSO is not working with vxlan over IPv6 over
>>>> mlx5 device.
>>>> We will test out those patches on both mlx4 and mlx5, and debug mlx4
>>>> IPv6 issue you see.
>>>>
>>>>>> Anyway, I suspect it might be related to a driver bug most likely in
>>>>>> get_real_size function @en_tx.c
>>>>>> specifically in : *lso_header_size =
>>>>>> (skb_inner_transport_header(skb) -
>>>>>> skb->data) + inner_tcp_hdrlen(skb);
>>>>>>
>>>>>> will check this and get back to you.
>>>>> I'm not entirely convinced. What I was seeing is t hat the hardware
>>>>> itself was performing Rx checksum offload only on tunnels with an
>>>>> outer IPv4 header and ignoring tunnels with an outer IPv6 header.
>>>> I don't get it, are you trying to say that the issue is in the RX
>>>> side ?
>>>> what do you mean by ignoring ? Dropping ? or just not validating
>>>> checksum ?
>>>> if so why would you disable GSO and IPv6 checksumming on TX ?
>>> I'm suspecting that whatever parsing logic exists in either the
>>> hardware or firmware may not be configured to parse tunnels with outer
>>> IPv6 headers. The tell-tale sign is what occurs with an IPv6 based
>>> tunnel with no outer checksum. The hardware is not performing a
>>> checksum on the inner headers so it reports it as a UDP frame with no
>>> checksum to the stack which ends up preventing us from doing GRO.
>>> That tells me that the hardware is not parsing IPv6 based tunnels on
>>> Rx. I am assuming that if the Rx side doesn't work then there is a
>>> good chance that the Tx won't.
>>>
>>>>>>> @@ -2431,7 +2435,18 @@ static netdev_features_t
>>>>>>> mlx4_en_features_check(struct sk_buff *skb,
>>>>>>> netdev_features_t
>>>>>>> features)
>>>>>>> {
>>>>>>> features = vlan_features_check(skb, features);
>>>>>>> - return vxlan_features_check(skb, features);
>>>>>>> + features = vxlan_features_check(skb, features);
>>>>>>> +
>>>>>>> + /* The ConnectX-3 doesn't support outer IPv6 checksums but
>>>>>>> it does
>>>>>>> + * support inner IPv6 checksums and segmentation so we
>>>>>>> need to
>>>>>>> + * strip that feature if this is an IPv6 encapsulated
>>>>>>> frame.
>>>>>>> + */
>>>>>>> + if (skb->encapsulation &&
>>>>>>> + (skb->ip_summed == CHECKSUM_PARTIAL) &&
>>>>>>> + (ip_hdr(skb)->version != 4))
>>>>>>> + features &= ~(NETIF_F_CSUM_MASK |
>>>>>>> NETIF_F_GSO_MASK);
>>>>>> Dejavu, didn't you fix this already in harmonize_features, in
>>>>>> i.e, it is enough to do here:
>>>>>>
>>>>>> if (skb->encapsulation && (skb->ip_summed == CHECKSUM_PARTIAL))
>>>>>> features &= ~NETIF_F_IPV6_CSUM;
>>>>>>
>>>>> So what this patch is doing is enabling an inner IPv6 header
>>>>> offloads.
>>>>> Up above we set the NETIF_F_IPV6_CSUM bit and we want it to stay set
>>>>> unless we have an outer IPv6 header because the inner headers may
>>>>> still need that bit set. If I did what you suggest it strips IPv6
>>>>> checksum support for inner headers and if we have to use GSO
>>>>> partial I
>>>>> ended up encountering some of the other bugs that I have fixed for
>>>>> GSO
>>>>> partial where either sg or csum are not defined.
>>>>>
>>>> I see, you mean that you want to disable checksumming and GSO only for
>>>> packets with Outer(IPv6):Inner(X) and keep it in case for
>>>> Outer(IPv4):Inner(IPv6)
>>>> but i think it is weird that the driver decides to disable features it
>>>> didn't declare in first place (NETIF_F_CSUM_MASK | NETIF_F_GSO_MASK)
>>>>
>>>> Retry:
>>>>
>>>> if (skb->encapsulation && (skb->ip_summed == CHECKSUM_PARTIAL) &&
>>>> (ip_hdr(skb)->version != 4))
>>>> features &= ~NETIF_F_IPV6_CSUM;
>>>>
>>>> will this work ?
>>> Sort of. All that would happen is that you would fall through to
>>> harmonize_features where NETIF_F_CSUM_MASK | NETIF_F_GSO_MASK gets
>>> cleared. I just figured I would short-cut things since we cannot
>>> support inner checksum or any GSO offloads if the tunnel has an outer
>>> IPv6 header. In addition this happens to effectively be the same code
>>> I am using in vxlan_features_check to disable things if we cannot
>>> checksum a protocol so it should help to keep the code size smaller
>>> for the function if the compiler is smart enough to coalesce similar
>>> code.
>>>
>>>> Anyway i prefer to debug the mlx4 issue first before we discuss the
>>>> best approach to disable checksumming & GSO for outer IPv6 in mlx4.
>>> The current code as-is already has it disabled. All I am doing is
>>> enabling IPv6 checksums for inner headers as it seems like it doesn't
>>> work for outer headers.
>>
>> Hi Alex,
>> I will be working on the mlx4 issue next week after the holidays.
>> I will check this offline in-house, without blocking the series.
>>
>> Regards,
>> Tariq
>>
>
> If it helps below are the kind of results I am seeing with the patch
> series applied versus what is currently there. The big problem areas
> are IPv6 over IPv4 tunnels, and IPv4 over IPv6 tunnels without checksums.
>
> The breakdown below is bare-ip-version without tunnel, outer-inner
> with tunnel, or outer-csum-inner if a checksum is present on the outer
> UDP header.
>
> After the series is applied you can see the v6 over v4 issues are
> addressed, and GSO partial has improved the performance for traffic
> over v4 tunnels with outer UDP checksums.
>
> Throughput Throughput Local Local Result
> Units CPU Service Tag
> Util Demand
> %
> mlx4 - Before
> -------------------------------------------------
> 26616.45 10^6bits/s 3.62 0.357 "v4"
> 26101.18 10^6bits/s 6.72 0.675 "v6"
> 22289.41 10^6bits/s 6.49 0.764 "v4-v4"
> N/A - could not connect "v4-v6"
> 12743.91 10^6bits/s 4.25 0.874 "v4-csum-v4"
> N/A - could not connect "v4-csum-v6"
> 0.69 10^6bits/s 0.66 2519.1 "v6-v4"
> 5924.47 10^6bits/s 4.23 1.871 "v6-v6"
> 10369.95 10^6bits/s 4.33 1.096 "v6-csum-v4"
> 10648.51 10^6bits/s 4.10 1.010 "v6-csum-v6"
>
> mlx4 - After
> -------------------------------------------------
> 26585.36 10^6bits/s 3.60 0.355 "v4"
> 26342.86 10^6bits/s 6.67 0.664 "v6"
> 22295.93 10^6bits/s 6.34 0.746 "v4-v4"
> 19977.24 10^6bits/s 6.04 0.793 "v4-v6"
> 22164.71 10^6bits/s 6.46 0.763 "v4-csum-v4"
> 19685.22 10^6bits/s 6.12 0.815 "v4-csum-v6"
> 6126.80 10^6bits/s 4.29 1.835 "v6-v4"
> 5793.53 10^6bits/s 4.24 1.917 "v6-v6"
> 10278.52 10^6bits/s 4.07 1.039 "v6-csum-v4"
> 10526.68 10^6bits/s 4.11 1.024 "v6-csum-v6"
Yeah that can help in comparing our results and getting aligned to
address the issues.
Thanks for sharing.
Regards,
Tariq
^ permalink raw reply
* Re: [net-next PATCH V4 1/5] samples/bpf: add back functionality to redefine LLC command
From: Naveen N. Rao @ 2016-04-28 13:21 UTC (permalink / raw)
To: Jesper Dangaard Brouer
Cc: netdev, linux-kbuild, bblanco, borkmann, alexei.starovoitov
In-Reply-To: <20160428122053.16807.93143.stgit@firesoul>
On 2016/04/28 02:20PM, Jesper Dangaard Brouer wrote:
> It is practical to be-able-to redefine the location of the LLVM
> command 'llc', because not all distros have a LLVM version with bpf
> target support. Thus, it is sometimes required to compile LLVM from
> source, and sometimes it is not desired to overwrite the distros
> default LLVM version.
>
> This feature was removed with 128d1514be35 ("samples/bpf: Use llc in
> PATH, rather than a hardcoded value").
>
> Add this features back. Note that it is possible to redefine the LLC
> on the make command like:
>
> make samples/bpf/ LLC=~/git/llvm/build/bin/llc
I don't have an objection to this patch, but you didn't explain why/how
this approach is better than just doing:
PATH=~/git/llvm/build/bin make samples/bpf/
- Naveen
^ permalink raw reply
* RE: [PATCH v2] can: rcar_canfd: Add Renesas R-Car CAN FD driver
From: Ramesh Shanmugasundaram @ 2016-04-28 12:31 UTC (permalink / raw)
To: Oliver Hartkopp, wg@grandegger.com, robh+dt@kernel.org,
pawel.moll@arm.com, mark.rutland@arm.com,
ijc+devicetree@hellion.org.uk, galak@codeaurora.org,
corbet@lwn.net, mkl@pengutronix.de
Cc: linux-renesas-soc@vger.kernel.org, devicetree@vger.kernel.org,
linux-can@vger.kernel.org, netdev@vger.kernel.org,
linux-doc@vger.kernel.org, geert+renesas@glider.be,
Chris Paterson
In-Reply-To: <5721AD57.8060105@hartkopp.net>
Hi Oliver,
Thanks :-)
Actually it will be v5 patch set & I have sent it now. Marc comments were on old v2 patch.
Thanks,
Ramesh
> -----Original Message-----
> From: Oliver Hartkopp [mailto:socketcan@hartkopp.net]
> Sent: 28 April 2016 07:28
> To: Ramesh Shanmugasundaram <ramesh.shanmugasundaram@bp.renesas.com>;
> wg@grandegger.com; robh+dt@kernel.org; pawel.moll@arm.com;
> mark.rutland@arm.com; ijc+devicetree@hellion.org.uk; galak@codeaurora.org;
> corbet@lwn.net; mkl@pengutronix.de
> Cc: linux-renesas-soc@vger.kernel.org; devicetree@vger.kernel.org; linux-
> can@vger.kernel.org; netdev@vger.kernel.org; linux-doc@vger.kernel.org;
> geert+renesas@glider.be; Chris Paterson <Chris.Paterson2@renesas.com>
> Subject: Re: [PATCH v2] can: rcar_canfd: Add Renesas R-Car CAN FD driver
>
> Hello Ramesh,
>
> please send out a new v3 patchset to trigger the process again :-)
>
> Best regards,
> Oliver
>
> On 04/13/2016 08:25 AM, Ramesh Shanmugasundaram wrote:
> > HI Marc,
> >
> > Gentle reminder!
> > Are you happy with the open comment's disposition? I can send a next
> version of patch if we have a closure on current set of comments.
> >
> > Thanks,
> > Ramesh
> >
> >> -----Original Message-----
> >> From: Ramesh Shanmugasundaram
> >> Sent: 01 April 2016 13:49
> >> To: Ramesh Shanmugasundaram <ramesh.shanmugasundaram@bp.renesas.com>;
> >> wg@grandegger.com; robh+dt@kernel.org; pawel.moll@arm.com;
> >> mark.rutland@arm.com; ijc+devicetree@hellion.org.uk;
> >> galak@codeaurora.org; corbet@lwn.net
> >> Cc: linux-renesas-soc@vger.kernel.org; devicetree@vger.kernel.org;
> >> linux- can@vger.kernel.org; netdev@vger.kernel.org;
> >> linux-doc@vger.kernel.org;
> >> geert+renesas@glider.be; Chris Paterson <Chris.Paterson2@renesas.com>
> >> Subject: RE: [PATCH v2] can: rcar_canfd: Add Renesas R-Car CAN FD
> >> driver
> >>
> >> Hi Marc,
> >>
> >> Thanks for your time & review comments.
> >>
> >>> -----Original Message-----
> >> (...)
> >>>> +#define RCANFD_NAPI_WEIGHT 8 /* Rx poll quota */
> >>>> +
> >>>> +#define RCANFD_NUM_CHANNELS 2
> >>>> +#define RCANFD_CHANNELS_MASK 0x3 /* Two channels max */
> >>>
> >>> (BIT(RCANFD_NUM_CHANNELS) - 1
> >>
> >> OK
> >>
> >>>
> >>>> +
> >>>> +/* Rx FIFO is a global resource of the controller. There are 8
> >>>> +such
> >>> FIFOs
> >>>> + * available. Each channel gets a dedicated Rx FIFO (i.e.) the
> >>>> + channel
> >> (...)
> >>>> +#define RCANFD_CMFIFO_CFDLC(x) (((x) & 0xf) << 28)
> >>>> +#define RCANFD_CMFIFO_CFPTR(x) (((x) & 0xfff) << 16)
> >>>> +#define RCANFD_CMFIFO_CFTS(x) (((x) & 0xff) << 0)
> >>>> +
> >>>> +/* Global Test Config register */
> >>>> +#define RCANFD_GTSTCFG_C0CBCE BIT(0)
> >>>> +#define RCANFD_GTSTCFG_C1CBCE BIT(1)
> >>>> +
> >>>> +#define RCANFD_GTSTCTR_ICBCTME BIT(0)
> >>>> +
> >>>> +/* AFL Rx rules registers */
> >>>> +#define RCANFD_AFLCFG_SETRNC0(x) (((x) & 0xff) << 24)
> >>>> +#define RCANFD_AFLCFG_SETRNC1(x) (((x) & 0xff) << 16)
> >>>
> >>> What about something like:
> >>>
> >>> #define RCANFD_AFLCFG_SETRNC(n, x) (((x) & 0xff) << (24 - n * 8))
> >>>
> >>> This will save some if()s in the code
> >>
> >> Nice :-). Will update.
> >>
> >>>
> >>>> +#define RCANFD_AFLCFG_GETRNC0(x) (((x) >> 24) & 0xff)
> >>>> +#define RCANFD_AFLCFG_GETRNC1(x) (((x) >> 16) & 0xff)
> >>>> +
> >>>> +#define RCANFD_AFL_PAGENUM(entry) ((entry) / 16)
> >> (...)
> >>>> +#define rcar_canfd_read(priv, offset) \
> >>>> + readl(priv->base + (offset))
> >>>> +#define rcar_canfd_write(priv, offset, val) \
> >>>> + writel(val, priv->base + (offset))
> >>>> +#define rcar_canfd_set_bit(priv, reg, val) \
> >>>> + rcar_canfd_update(val, val, priv->base + (reg))
> >>>> +#define rcar_canfd_clear_bit(priv, reg, val) \
> >>>> + rcar_canfd_update(val, 0, priv->base + (reg))
> >>>> +#define rcar_canfd_update_bit(priv, reg, mask, val) \
> >>>> + rcar_canfd_update(mask, val, priv->base + (reg))
> >>>
> >>> please use static inline functions instead of defines.
> >>
> >> OK.
> >>
> >>>
> >>>> +
> >>>> +static void rcar_canfd_get_data(struct canfd_frame *cf,
> >>>> + struct rcar_canfd_channel *priv, u32 off)
> >>>
> >>> Please use "struct rcar_canfd_channel *priv" as first argument,
> >>> struct canfd_frame *cf as second. Remove off, as the offset is
> >>> already defined by the channel.
> >>
> >> I'll re-order priv, cf as you mentioned. I'll leave "off" arg as it
> >> is because it is based on FIFO number of channel + mode (CAN only or
> >> CANFD only). Otherwise, I will have to add another check inside this
> >> function for mode.
> >>
> >>>> +{
> >>>> + u32 i, lwords;
> >>>> +
> >>>> + lwords = cf->len / sizeof(u32);
> >>>> + if (cf->len % sizeof(u32))
> >>>> + lwords++;
> >>>
> >>> Use DIV_ROUND_UP() instead of open coding it.
> >>
> >> Agreed. Thanks.
> >>
> >>>
> >>>> + for (i = 0; i < lwords; i++)
> >>>> + *((u32 *)cf->data + i) =
> >>>> + rcar_canfd_read(priv, off + (i * sizeof(u32))); }
> >>>> +
> >>>> +static void rcar_canfd_put_data(struct canfd_frame *cf,
> >>>> + struct rcar_canfd_channel *priv, u32 off)
> >>>
> >>> same here
> >>
> >> Yes (same as _get_data)
> >>
> >>>
> >>>> +{
> >>>> + u32 i, j, lwords, leftover;
> >>>> + u32 data = 0;
> >>>> +
> >>>> + lwords = cf->len / sizeof(u32);
> >>>> + leftover = cf->len % sizeof(u32);
> >>>> + for (i = 0; i < lwords; i++)
> >>>> + rcar_canfd_write(priv, off + (i * sizeof(u32)),
> >>>> + *((u32 *)cf->data + i));
> >>>
> >>> Here you don't convert the endianess...
> >>
> >> Yes
> >>
> >>>> +
> >>>> + if (leftover) {
> >>>> + u8 *p = (u8 *)((u32 *)cf->data + i);
> >>>> +
> >>>> + for (j = 0; j < leftover; j++)
> >>>> + data |= p[j] << (j * 8);
> >>>
> >>> ...here you do an implicit endianess conversion. "data" is little
> >>> endian, while p[j] is big endian.
> >>
> >> Not sure I got the question correctly.
> >> Controller expectation of data bytes in 32bit register is bits[7:0] =
> >> byte0, bits[15:8] = byte1 and so on - little endian.
> >> For e.g. if cf->data points to byte stream H'112233445566
> >> (cf->data[0] = 0x11), first rcar_canfd_write will write 0x44332211
> >> value to register. Yes the host cpu is assumed little endian. In
> >> leftover case, data will be
> >> 0x00006655 - again little endian.
> >> I think I should remove this leftover logic and just mask the unused
> >> bits to zero as cf->data is pre-allocated for max length anyway.
> >>
> >> p[j] is a byte?? Am I missing something?
> >>
> >>>
> >>>> + rcar_canfd_write(priv, off + (i * sizeof(u32)), data);
> >>>> + }
> >>>
> >>> Have you tested to send CAN frames with len != n*4 against a
> >>> different controller?
> >>
> >> Yes, with Vector VN1630A.
> >>
> >>>> +}
> >>>> +
> >>>> +static void rcar_canfd_tx_failure_cleanup(struct net_device *ndev)
> >>>> +{
> >>>> + u32 i;
> >>>> +
> >>>> + for (i = 0; i < RCANFD_FIFO_DEPTH; i++)
> >>>> + can_free_echo_skb(ndev, i);
> >>>> +}
> >>>> +
> >> (...)
> >>>> +static void rcar_canfd_tx_done(struct net_device *ndev) {
> >>>> + struct rcar_canfd_channel *priv = netdev_priv(ndev);
> >>>> + struct net_device_stats *stats = &ndev->stats;
> >>>> + u32 sts;
> >>>> + unsigned long flags;
> >>>> + u32 ch = priv->channel;
> >>>> +
> >>>> + do {
> >>>
> >>> You should iterare over all pending CAN frames:
> >>>
> >>>> for (/* nix */; (priv->tx_head - priv->tx_tail) > 0; priv-
> >>>> tx_tail++) {
> >>>
> >>
> >> Yes, current code does iterate over pending tx'ed frames. If we use
> >> this for loop semantics, we may have to protect the whole loop with
> >> no real benefit.
> >>
> >>>
> >>>> + u8 unsent, sent;
> >>>> +
> >>>> + sent = priv->tx_tail % RCANFD_FIFO_DEPTH;
> >>>
> >>> and check here, if that packet has really been tramsitted. Exit the
> >>> loop otherweise.
> >>
> >> We are here because of tx_done and hence no need to check tx done
> >> again for the first iteration. Hence the do-while loop. Checks
> >> further down takes care of the need for more iterations/pending tx'ed
> frames.
> >>
> >>>
> >>>> + stats->tx_packets++;
> >>>> + stats->tx_bytes += priv->tx_len[sent];
> >>>> + priv->tx_len[sent] = 0;
> >>>> + can_get_echo_skb(ndev, sent);
> >>>> +
> >>>> + spin_lock_irqsave(&priv->tx_lock, flags);
> >>>
> >>> What does the tx_lock protect? The tx path is per channel, isn't it?
> >>
> >> You are right - tx path is per channel. In tx path, head & tail are
> >> also used to determine when to stop/wakeup netif queue. They are
> >> incremented & compared in different contexts to make this decision.
> >> Per channel tx_lock protects this critical section.
> >>
> >>>
> >>>> + priv->tx_tail++;
> >>>> + sts = rcar_canfd_read(priv,
> >>>> + RCANFD_CFSTS(ch, RCANFD_CFFIFO_IDX));
> >>>> + unsent = RCANFD_CMFIFO_CFMC(sts);
> >>>> +
> >>>> + /* Wake producer only when there is room */
> >>>> + if (unsent != RCANFD_FIFO_DEPTH)
> >>>> + netif_wake_queue(ndev);
> >>>
> >>> Move the netif_wake_queue() out of the loop.
> >>
> >> With the tx logic mentioned above, I think keeping it in the loop
> >> seems better. For e.g. say cpu1 pumps 8 frames from an app loop and
> >> cpu0 handles tx_done interrupt handling: cpu1 would have stopped the
> >> queue because FIFO is full -> cpu0 gets tx_done interrupt and by the
> time it checks "unsent"
> >> there could be one or more frames transmitted by device (i.e.) there
> >> would be more space in fifo. It is better to wakeup the netif queue
> >> then and there so that app running on cpu1 can pump more. If we move
> >> it out of the loop we wake up the queue only in the end. Have I missed
> anything?
> >>
> >>>
> >>>> +
> >>>> + if (priv->tx_head - priv->tx_tail <= unsent) {
> >>>> + spin_unlock_irqrestore(&priv->tx_lock, flags);
> >>>> + break;
> >>>> + }
> >>>> + spin_unlock_irqrestore(&priv->tx_lock, flags);
> >>>> +
> >>>> + } while (1);
> >>>> +
> >>>> + /* Clear interrupt */
> >>>> + rcar_canfd_write(priv, RCANFD_CFSTS(ch, RCANFD_CFFIFO_IDX),
> >>>> + sts & ~RCANFD_CMFIFO_CFTXIF);
> >>>> + can_led_event(ndev, CAN_LED_EVENT_TX); }
> >>>> +
> >>>> + if (cf->can_id & CAN_RTR_FLAG)
> >>>> + id |= RCANFD_CMFIFO_CFRTR;
> >>>> +
> >>>> + rcar_canfd_write(priv, RCANFD_F_CFID(ch, RCANFD_CFFIFO_IDX),
> >>>> + id);
> >>>> + ptr = RCANFD_CMFIFO_CFDLC(can_len2dlc(cf->len));
> >>>
> >>> ptr usually means pointer, better call it dlc.
> >>
> >> I used the register name "ptr". OK, will change it do dlc.
> >>
> >>>
> >>>> + rcar_canfd_write(priv, RCANFD_F_CFPTR(ch, RCANFD_CFFIFO_IDX),
> >>>> + ptr);
> >>>> +
> >> (...)
> >>>> + can_put_echo_skb(skb, ndev, priv->tx_head %
> RCANFD_FIFO_DEPTH);
> >>>> +
> >>>> + spin_lock_irqsave(&priv->tx_lock, flags);
> >>>> + priv->tx_head++;
> >>>> +
> >>>> + /* Start Tx: Write 0xff to CFPC to increment the CPU-side
> >>>> + * pointer for the Common FIFO
> >>>> + */
> >>>> + rcar_canfd_write(priv, RCANFD_CFPCTR(ch, RCANFD_CFFIFO_IDX),
> >>>> +0xff);
> >>>> +
> >>>> + /* Stop the queue if we've filled all FIFO entries */
> >>>> + if (priv->tx_head - priv->tx_tail >= RCANFD_FIFO_DEPTH)
> >>>> + netif_stop_queue(ndev);
> >>>
> >>> Please move the check of stop_queue, before starting the send.
> >>
> >> OK.
> >>
> >>>
> >>>> +
> >>>> + spin_unlock_irqrestore(&priv->tx_lock, flags);
> >>>> + return NETDEV_TX_OK;
> >>>> +}
> >>>> +
> >> (...)
> >>>> +{
> >>>> + struct rcar_canfd_channel *priv =
> >>>> + container_of(napi, struct rcar_canfd_channel, napi);
> >>>> + int num_pkts;
> >>>> + u32 sts;
> >>>> + u32 ch = priv->channel;
> >>>> + u32 ridx = ch + RCANFD_RFFIFO_IDX;
> >>>> +
> >>>> + for (num_pkts = 0; num_pkts < quota; num_pkts++) {
> >>>> + sts = rcar_canfd_read(priv, RCANFD_RFSTS(ridx));
> >>>> + /* Clear interrupt bit */
> >>>> + if (sts & RCANFD_RFFIFO_RFIF)
> >>>> + rcar_canfd_write(priv, RCANFD_RFSTS(ridx),
> >>>> + sts & ~RCANFD_RFFIFO_RFIF);
> >>>> +
> >>>> + /* Check FIFO empty condition */
> >>>> + if (sts & RCANFD_RFFIFO_RFEMP)
> >>>> + break;
> >>>> +
> >>>> + rcar_canfd_rx_pkt(priv);
> >>>
> >>> This sequence looks strange. You first conditionally ack the
> >>> interrupt then you check for empty fifo, then read the CAN frame. I
> >>> would assume that you first check if there's a CAN frame, read it
> >>> and then clear the interrupt.
> >>
> >> Yes, I shall re-arrange the sequence as you mentioned.
> >>
> >>>
> >>>> + }
> >>>> +
> >>>> + /* All packets processed */
> >>>> + if (num_pkts < quota) {
> >>>> + napi_complete(napi);
> >>>> + /* Enable Rx FIFO interrupts */
> >>>> + rcar_canfd_set_bit(priv, RCANFD_RFCC(ridx),
> >>> RCANFD_RFFIFO_RFIE);
> >> (...)
> >>>> + for_each_set_bit(ch, &gpriv->channels_mask,
> RCANFD_NUM_CHANNELS) {
> >>>> + err = rcar_canfd_channel_probe(gpriv, ch);
> >>>> + if (err)
> >>>> + goto fail_channel;
> >>>> + }
> >>>
> >>> Should the CAN IP core be clocked the whole time? What about shuting
> >>> down the clock and enabling it on ifup?
> >>
> >> The fCAN clock is enabled only on ifup of one of the channels.
> >> However, the peripheral clock is enabled during probe to bring the
> >> controller to Global Operational mode. This clock cannot be turned
> >> off with the register values & mode retained.
> >>
> >>>> +
> >>>> + platform_set_drvdata(pdev, gpriv);
> >>>> + dev_info(&pdev->dev, "global operational state (clk %d)\n",
> >>>> + gpriv->clock_select);
> >> (...)
> >>>> + rcar_canfd_reset_controller(gpriv);
> >>>> + rcar_canfd_disable_global_interrupts(gpriv);
> >>>> +
> >>>> + for_each_set_bit(ch, &gpriv->channels_mask,
> RCANFD_NUM_CHANNELS) {
> >>>> + priv = gpriv->ch[ch];
> >>>> + if (priv) {
> >>>
> >>> This should always be true.
> >>
> >> I agree. I thought I cleaned this up but it's in my local commits :-(
> >>
> >>>
> >>>> + rcar_canfd_disable_channel_interrupts(priv);
> >>>> + unregister_candev(priv->ndev);
> >>>> + netif_napi_del(&priv->napi);
> >>>> + free_candev(priv->ndev);
> >>>
> >>> Please make use of rcar_canfd_channel_remove(), as you already have
> >>> the function.
> >>
> >> Yes.
> >>
> >> Thanks,
> >> Ramesh
^ permalink raw reply
* [PATCH v5 2/2] can: rcar_can: Move Renesas CAN driver to rcar dir
From: Ramesh Shanmugasundaram @ 2016-04-28 12:23 UTC (permalink / raw)
To: mkl, wg, socketcan, robh+dt, pawel.moll, mark.rutland,
ijc+devicetree, galak
Cc: linux-renesas-soc, devicetree, linux-can, netdev, chris.paterson2,
Ramesh Shanmugasundaram
In-Reply-To: <1461846188-31249-1-git-send-email-ramesh.shanmugasundaram@bp.renesas.com>
This patch clubs the Renesas controller drivers in one rcar dir.
Signed-off-by: Ramesh Shanmugasundaram <ramesh.shanmugasundaram@bp.renesas.com>
---
drivers/net/can/Kconfig | 10 ----------
drivers/net/can/Makefile | 1 -
drivers/net/can/rcar/Kconfig | 10 ++++++++++
drivers/net/can/rcar/Makefile | 3 ++-
drivers/net/can/{ => rcar}/rcar_can.c | 0
5 files changed, 12 insertions(+), 12 deletions(-)
rename drivers/net/can/{ => rcar}/rcar_can.c (100%)
diff --git a/drivers/net/can/Kconfig b/drivers/net/can/Kconfig
index 13003a9..22570ea 100644
--- a/drivers/net/can/Kconfig
+++ b/drivers/net/can/Kconfig
@@ -104,16 +104,6 @@ config CAN_JANZ_ICAN3
This driver can also be built as a module. If so, the module will be
called janz-ican3.ko.
-config CAN_RCAR
- tristate "Renesas R-Car CAN controller"
- depends on ARCH_RENESAS || ARM
- ---help---
- Say Y here if you want to use CAN controller found on Renesas R-Car
- SoCs.
-
- To compile this driver as a module, choose M here: the module will
- be called rcar_can.
-
config CAN_SUN4I
tristate "Allwinner A10 CAN controller"
depends on MACH_SUN4I || MACH_SUN7I || COMPILE_TEST
diff --git a/drivers/net/can/Makefile b/drivers/net/can/Makefile
index 226d5b5..26ba4b7 100644
--- a/drivers/net/can/Makefile
+++ b/drivers/net/can/Makefile
@@ -25,7 +25,6 @@ obj-$(CONFIG_CAN_IFI_CANFD) += ifi_canfd/
obj-$(CONFIG_CAN_JANZ_ICAN3) += janz-ican3.o
obj-$(CONFIG_CAN_MSCAN) += mscan/
obj-$(CONFIG_CAN_M_CAN) += m_can/
-obj-$(CONFIG_CAN_RCAR) += rcar_can.o
obj-$(CONFIG_CAN_SJA1000) += sja1000/
obj-$(CONFIG_CAN_SUN4I) += sun4i_can.o
obj-$(CONFIG_CAN_TI_HECC) += ti_hecc.o
diff --git a/drivers/net/can/rcar/Kconfig b/drivers/net/can/rcar/Kconfig
index 6ea64d1..7b03a3a 100644
--- a/drivers/net/can/rcar/Kconfig
+++ b/drivers/net/can/rcar/Kconfig
@@ -1,3 +1,13 @@
+config CAN_RCAR
+ tristate "Renesas R-Car CAN controller"
+ depends on ARCH_RENESAS || ARM
+ ---help---
+ Say Y here if you want to use CAN controller found on Renesas R-Car
+ SoCs.
+
+ To compile this driver as a module, choose M here: the module will
+ be called rcar_can.
+
config CAN_RCAR_CANFD
tristate "Renesas R-Car CAN FD controller"
depends on ARCH_RENESAS || ARM
diff --git a/drivers/net/can/rcar/Makefile b/drivers/net/can/rcar/Makefile
index cbaf498e..08de36a 100644
--- a/drivers/net/can/rcar/Makefile
+++ b/drivers/net/can/rcar/Makefile
@@ -1,5 +1,6 @@
#
-# Makefile for the Renesas R-Car CAN FD controller driver
+# Makefile for the Renesas R-Car CAN & CAN FD controller drivers
#
+obj-$(CONFIG_CAN_RCAR) += rcar_can.o
obj-$(CONFIG_CAN_RCAR_CANFD) += rcar_canfd.o
diff --git a/drivers/net/can/rcar_can.c b/drivers/net/can/rcar/rcar_can.c
similarity index 100%
rename from drivers/net/can/rcar_can.c
rename to drivers/net/can/rcar/rcar_can.c
--
1.9.1
^ permalink raw reply related
* [PATCH v5 1/2] can: rcar_canfd: Add Renesas R-Car CAN FD driver
From: Ramesh Shanmugasundaram @ 2016-04-28 12:23 UTC (permalink / raw)
To: mkl, wg, socketcan, robh+dt, pawel.moll, mark.rutland,
ijc+devicetree, galak
Cc: linux-renesas-soc, devicetree, linux-can, netdev, chris.paterson2,
Ramesh Shanmugasundaram
In-Reply-To: <1461846188-31249-1-git-send-email-ramesh.shanmugasundaram@bp.renesas.com>
This patch adds support for the CAN FD controller found in Renesas R-Car
SoCs. The controller operates in CAN FD only mode by default.
CAN FD mode supports both Classical CAN & CAN FD frame formats. The
controller supports ISO 11898-1:2015 CAN FD format only.
This controller supports two channels and the driver can enable either
or both of the channels.
Driver uses Rx FIFOs (one per channel) for reception & Common FIFOs (one
per channel) for transmission. Rx filter rules are configured to the
minimum (one per channel) and it accepts Standard, Extended, Data &
Remote Frame combinations.
Note: There are few documentation errors in R-Car Gen3 Hardware User
Manual v0.5E with respect to CAN FD controller. They are listed below:
1. CAN FD interrupt numbers 29 & 30 are listed as per channel
interrupts. However, they are common to both channels (i.e.) they are
global and channel interrupts respectively.
2. CANFD clock is derived from PLL1. This is not documented.
3. CANFD clock is further divided by (1/2) within the CAN FD controller.
This is not documented.
4. The minimum value of NTSEG1 in RSCFDnCFDCmNCFG register is 2 Tq. It
is specified 4 Tq in the manual.
5. The maximum number of message RAM area the controller can use is 3584
bytes. It is specified 10752 bytes in the manual.
Signed-off-by: Ramesh Shanmugasundaram <ramesh.shanmugasundaram@bp.renesas.com>
---
.../devicetree/bindings/net/can/rcar_canfd.txt | 89 ++
drivers/net/can/Kconfig | 1 +
drivers/net/can/Makefile | 1 +
drivers/net/can/rcar/Kconfig | 11 +
drivers/net/can/rcar/Makefile | 5 +
drivers/net/can/rcar/rcar_canfd.c | 1624 ++++++++++++++++++++
6 files changed, 1731 insertions(+)
create mode 100644 Documentation/devicetree/bindings/net/can/rcar_canfd.txt
create mode 100644 drivers/net/can/rcar/Kconfig
create mode 100644 drivers/net/can/rcar/Makefile
create mode 100644 drivers/net/can/rcar/rcar_canfd.c
diff --git a/Documentation/devicetree/bindings/net/can/rcar_canfd.txt b/Documentation/devicetree/bindings/net/can/rcar_canfd.txt
new file mode 100644
index 0000000..d45182b
--- /dev/null
+++ b/Documentation/devicetree/bindings/net/can/rcar_canfd.txt
@@ -0,0 +1,89 @@
+Renesas R-Car CAN FD controller Device Tree Bindings
+----------------------------------------------------
+
+Required properties:
+- compatible: Must contain one or more of the following:
+ - "renesas,rcar-gen3-canfd" for R-Car Gen3 compatible controller.
+ - "renesas,r8a7795-canfd" for R8A7795 (R-Car H3) compatible controller.
+
+ When compatible with the generic version, nodes must list the
+ SoC-specific version corresponding to the platform first, followed by the
+ family-specific and/or generic versions.
+
+- reg: physical base address and size of the R-Car CAN FD register map.
+- interrupts: interrupt specifier for the Global & Channel interrupts
+- clocks: phandles and clock specifiers for 3 clock inputs.
+- clock-names: 3 clock input name strings: "fck", "canfd", "can_clk".
+- pinctrl-0: pin control group to be used for this controller.
+- pinctrl-names: must be "default".
+
+Required child nodes:
+The controller supports two channels and each is represented as a child node.
+The name of the child nodes are "channel0" and "channel1" respectively. Each
+child node supports the "status" property only, which is used to
+enable/disable the respective channel.
+
+Required properties for "renesas,r8a7795-canfd" compatible:
+In R8A7795 SoC, canfd clock is a div6 clock and can be used by both CAN
+and CAN FD controller at the same time. It needs to be scaled to maximum
+frequency if any of these controllers use it. This is done using the
+below properties.
+
+- assigned-clocks: phandle of canfd clock.
+- assigned-clock-rates: maximum frequency of this clock.
+
+Example
+-------
+
+SoC common .dtsi file:
+
+ canfd: can@e66c0000 {
+ compatible = "renesas,r8a7795-canfd",
+ "renesas,rcar-gen3-canfd";
+ reg = <0 0xe66c0000 0 0x8000>;
+ interrupts = <GIC_SPI 29 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 30 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&cpg CPG_MOD 914>,
+ <&cpg CPG_CORE R8A7795_CLK_CANFD>,
+ <&can_clk>;
+ clock-names = "fck", "canfd", "can_clk";
+ assigned-clocks = <&cpg CPG_CORE R8A7795_CLK_CANFD>;
+ assigned-clock-rates = <40000000>;
+ power-domains = <&cpg>;
+ status = "disabled";
+
+ channel0 {
+ status = "disabled";
+ };
+
+ channel1 {
+ status = "disabled";
+ };
+ };
+
+Board specific .dts file:
+
+E.g. below enables Channel 1 alone in the board.
+
+&canfd {
+ pinctrl-0 = <&canfd1_pins>;
+ pinctrl-names = "default";
+ status = "okay";
+
+ channel1 {
+ status = "okay";
+ };
+};
+
+E.g. below enables Channel 0 alone in the board using External clock
+as fCAN clock.
+
+&canfd {
+ pinctrl-0 = <&canfd0_pins &can_clk_pins>;
+ pinctrl-names = "default";
+ status = "okay";
+
+ channel0 {
+ status = "okay";
+ };
+};
diff --git a/drivers/net/can/Kconfig b/drivers/net/can/Kconfig
index 0d40aef..13003a9 100644
--- a/drivers/net/can/Kconfig
+++ b/drivers/net/can/Kconfig
@@ -152,6 +152,7 @@ source "drivers/net/can/cc770/Kconfig"
source "drivers/net/can/ifi_canfd/Kconfig"
source "drivers/net/can/m_can/Kconfig"
source "drivers/net/can/mscan/Kconfig"
+source "drivers/net/can/rcar/Kconfig"
source "drivers/net/can/sja1000/Kconfig"
source "drivers/net/can/softing/Kconfig"
source "drivers/net/can/spi/Kconfig"
diff --git a/drivers/net/can/Makefile b/drivers/net/can/Makefile
index e3db0c8..226d5b5 100644
--- a/drivers/net/can/Makefile
+++ b/drivers/net/can/Makefile
@@ -10,6 +10,7 @@ can-dev-y := dev.o
can-dev-$(CONFIG_CAN_LEDS) += led.o
+obj-y += rcar/
obj-y += spi/
obj-y += usb/
obj-y += softing/
diff --git a/drivers/net/can/rcar/Kconfig b/drivers/net/can/rcar/Kconfig
new file mode 100644
index 0000000..6ea64d1
--- /dev/null
+++ b/drivers/net/can/rcar/Kconfig
@@ -0,0 +1,11 @@
+config CAN_RCAR_CANFD
+ tristate "Renesas R-Car CAN FD controller"
+ depends on ARCH_RENESAS || ARM
+ ---help---
+ Say Y here if you want to use CAN FD controller found on
+ Renesas R-Car SoCs. The driver puts the controller in CAN FD only
+ mode, which can interoperate with CAN2.0 nodes but does not support
+ dedicated CAN 2.0 mode.
+
+ To compile this driver as a module, choose M here: the module will
+ be called rcar_canfd.
diff --git a/drivers/net/can/rcar/Makefile b/drivers/net/can/rcar/Makefile
new file mode 100644
index 0000000..cbaf498e
--- /dev/null
+++ b/drivers/net/can/rcar/Makefile
@@ -0,0 +1,5 @@
+#
+# Makefile for the Renesas R-Car CAN FD controller driver
+#
+
+obj-$(CONFIG_CAN_RCAR_CANFD) += rcar_canfd.o
diff --git a/drivers/net/can/rcar/rcar_canfd.c b/drivers/net/can/rcar/rcar_canfd.c
new file mode 100644
index 0000000..e198732
--- /dev/null
+++ b/drivers/net/can/rcar/rcar_canfd.c
@@ -0,0 +1,1624 @@
+/* Renesas R-Car CAN FD device driver
+ *
+ * Copyright (C) 2015 Renesas Electronics Corp.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+/* The R-Car CAN FD controller can operate in either one of the below two modes
+ * - CAN FD only mode
+ * - Classical CAN (CAN 2.0) only mode
+ *
+ * This driver puts the controller in CAN FD only mode by default. In this
+ * mode, the controller acts as a CAN FD node that can also interoperate with
+ * CAN 2.0 nodes.
+ *
+ * As of now, this driver does not support the Classical CAN (CAN 2.0) mode,
+ * which is handled by a different register map compared to CAN FD only mode.
+ */
+
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/kernel.h>
+#include <linux/types.h>
+#include <linux/interrupt.h>
+#include <linux/errno.h>
+#include <linux/netdevice.h>
+#include <linux/platform_device.h>
+#include <linux/can/led.h>
+#include <linux/can/dev.h>
+#include <linux/clk.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/bitmap.h>
+#include <linux/bitops.h>
+#include <linux/iopoll.h>
+
+#define RCANFD_DRV_NAME "rcar_canfd"
+
+#define RCANFD_FIFO_DEPTH 8 /* Tx FIFO depth */
+#define RCANFD_NAPI_WEIGHT 8 /* Rx poll quota */
+
+#define RCANFD_NUM_CHANNELS 2 /* Two channels max */
+#define RCANFD_CHANNELS_MASK BIT((RCANFD_NUM_CHANNELS) - 1)
+
+/* Rx FIFO is a global resource of the controller. There are 8 such FIFOs
+ * available. Each channel gets a dedicated Rx FIFO (i.e.) the channel
+ * number is added to RFFIFO index.
+ */
+#define RCANFD_RFFIFO_IDX 0
+
+/* Tx/Rx or Common FIFO is a per channel resource. Each channel has 3 Common
+ * FIFOs dedicated to them. Use the first (index 0) FIFO out of the 3 for Tx.
+ */
+#define RCANFD_CFFIFO_IDX 0
+
+/* Global register bits */
+#define RCANFD_GINTF_CANFD BIT(0)
+
+#define RCANFD_GCFG_TPRI BIT(0)
+#define RCANFD_GCFG_DCE BIT(1)
+#define RCANFD_GCFG_DCS BIT(4)
+#define RCANFD_GCFG_CMPOC BIT(5)
+#define RCANFD_GCFG_EEFE BIT(6)
+
+#define RCANFD_GCTR_SLPR BIT(2)
+#define RCANFD_GCTR_MODEMASK (0x3)
+#define RCANFD_GCTR_GOPM (0x0)
+#define RCANFD_GCTR_GRESET (0x1)
+#define RCANFD_GCTR_GHLT (0x2)
+
+#define RCANFD_GCTR_DEIE BIT(8)
+#define RCANFD_GCTR_MEIE BIT(9)
+#define RCANFD_GCTR_THLEIE BIT(10)
+#define RCANFD_GCTR_CFMPOFIE BIT(11)
+#define RCANFD_GCTR_TSRST BIT(16)
+
+#define RCANFD_GSTS_RAMINIT BIT(3)
+#define RCANFD_GSTS_SLP BIT(2)
+#define RCANFD_GSTS_HLT BIT(1)
+#define RCANFD_GSTS_RESET BIT(0)
+
+#define RCANFD_GSTS_GNOPM (BIT(0) | BIT(1) | BIT(2) | BIT(3))
+
+/* Channel register bits */
+#define RCANFD_CCTR_CSLPR BIT(2)
+#define RCANFD_CCTR_MODEMASK (0x3)
+#define RCANFD_CCTR_COPM (0x0)
+#define RCANFD_CCTR_CRESET (0x1)
+#define RCANFD_CCTR_CHLT (0x2)
+#define RCANFD_CCTR_CTMASK (0x3 << 25)
+#define RCANFD_CCTR_CTMS_ILB (0x3 << 25)
+#define RCANFD_CCTR_CTME BIT(24)
+#define RCANFD_CCTR_ERRD BIT(23)
+#define RCANFD_CCTR_BOMMASK (0x3 << 21)
+#define RCANFD_CCTR_BOM_ENTRY (0x1 << 21)
+#define RCANFD_CCTR_TDCVFIE BIT(19)
+#define RCANFD_CCTR_SOCOIE BIT(18)
+#define RCANFD_CCTR_EOCOIE BIT(17)
+#define RCANFD_CCTR_TAIE BIT(16)
+#define RCANFD_CCTR_ALIE BIT(15)
+#define RCANFD_CCTR_BLIE BIT(14)
+#define RCANFD_CCTR_OLIE BIT(13)
+#define RCANFD_CCTR_BORIE BIT(12)
+#define RCANFD_CCTR_BOEIE BIT(11)
+#define RCANFD_CCTR_EPIE BIT(10)
+#define RCANFD_CCTR_EWIE BIT(9)
+#define RCANFD_CCTR_BEIE BIT(8)
+
+#define RCANFD_CSTS_COM BIT(7)
+#define RCANFD_CSTS_REC BIT(6)
+#define RCANFD_CSTS_TRM BIT(5)
+#define RCANFD_CSTS_BO BIT(4)
+#define RCANFD_CSTS_EP BIT(3)
+#define RCANFD_CSTS_SLP BIT(2)
+#define RCANFD_CSTS_HALT BIT(1)
+#define RCANFD_CSTS_RESET BIT(0)
+
+#define RCANFD_CSTS_TECCNT(x) (((x) >> 24) & 0xff)
+#define RCANFD_CSTS_RECCNT(x) (((x) >> 16) & 0xff)
+
+/* Bit Configuration register */
+#define RCANFD_BRP(x) (((x) & 0x3ff) << 0)
+#define RCANFD_SJW(x) (((x) & 0x3) << 24)
+#define RCANFD_TSEG1(x) (((x) & 0xf) << 16)
+#define RCANFD_TSEG2(x) (((x) & 0x7) << 20)
+
+#define RCANFD_NR_BRP(x) (((x) & 0x3ff) << 0)
+#define RCANFD_NR_SJW(x) (((x) & 0x1f) << 11)
+#define RCANFD_NR_TSEG1(x) (((x) & 0x7f) << 16)
+#define RCANFD_NR_TSEG2(x) (((x) & 0x1f) << 24)
+
+#define RCANFD_DR_BRP(x) (((x) & 0xff) << 0)
+#define RCANFD_DR_SJW(x) (((x) & 0x7) << 24)
+#define RCANFD_DR_TSEG1(x) (((x) & 0xf) << 16)
+#define RCANFD_DR_TSEG2(x) (((x) & 0x7) << 20)
+
+/* Global Error flag bits */
+#define RCANFD_GERFL_EEF1 BIT(17)
+#define RCANFD_GERFL_EEF0 BIT(16)
+#define RCANFD_GERFL_CMPOF BIT(3)
+#define RCANFD_GERFL_THLES BIT(2)
+#define RCANFD_GERFL_MES BIT(1)
+#define RCANFD_GERFL_DEF BIT(0)
+
+#define RCANFD_GERFL_ERR(x) ((x) & (RCANFD_GERFL_EEF1 |\
+ RCANFD_GERFL_EEF0 |\
+ RCANFD_GERFL_MES |\
+ RCANFD_GERFL_CMPOF))
+
+/* Channel Error flag bits */
+#define RCANFD_CERFL_ADEF BIT(14)
+#define RCANFD_CERFL_B0EF BIT(13)
+#define RCANFD_CERFL_B1EF BIT(12)
+#define RCANFD_CERFL_CEF BIT(11)
+#define RCANFD_CERFL_AEF BIT(10)
+#define RCANFD_CERFL_FEF BIT(9)
+#define RCANFD_CERFL_SEF BIT(8)
+#define RCANFD_CERFL_ALEF BIT(7)
+#define RCANFD_CERFL_BLEF BIT(6)
+#define RCANFD_CERFL_OLEF BIT(5)
+#define RCANFD_CERFL_BOREF BIT(4)
+#define RCANFD_CERFL_BOEEF BIT(3)
+#define RCANFD_CERFL_EPEF BIT(2)
+#define RCANFD_CERFL_EWEF BIT(1)
+#define RCANFD_CERFL_BEF BIT(0)
+
+#define RCANFD_CERFL_ERR(x) ((x) & (0x7fff)) /* above bits 14:0 */
+
+/* CAN FD specific registers */
+#define RCANFD_DCFG_TDCE BIT(9)
+#define RCANFD_DCFG_TDCOC BIT(8)
+#define RCANFD_DCFG_TDCO(x) (((x) & 0x7f) >> 16)
+
+#define RCANFD_DCSTS_TDCR(x) (((x) >> 0) & 0x7f)
+#define RCANFD_DCSTS_SOCCNT(x) (((x) >> 24) & 0xff)
+#define RCANFD_DCSTS_EOCCNT(x) (((x) >> 16) & 0xff)
+
+#define RCANFD_DCSTS_TDCVF BIT(7)
+#define RCANFD_DCSTS_EOCO BIT(8)
+#define RCANFD_DCSTS_SOCO BIT(9)
+
+/* Rx FIFO bits */
+#define RCANFD_RFFIFO_RFIF BIT(3)
+#define RCANFD_RFFIFO_RFMLT BIT(2)
+#define RCANFD_RFFIFO_RFFLL BIT(1)
+#define RCANFD_RFFIFO_RFEMP BIT(0)
+
+#define RCANFD_RFFIFO_RFESI BIT(0)
+#define RCANFD_RFFIFO_RFBRS BIT(1)
+#define RCANFD_RFFIFO_RFFDF BIT(2)
+
+#define RCANFD_RFFIFO_RFIDE BIT(31)
+#define RCANFD_RFFIFO_RFRTR BIT(30)
+
+#define RCANFD_RFFIFO_RFDLC(x) (((x) >> 28) & 0xf)
+#define RCANFD_RFFIFO_RFPTR(x) (((x) >> 16) & 0xfff)
+#define RCANFD_RFFIFO_RFTS(x) (((x) >> 0) & 0xff)
+
+#define RCANFD_RFFIFO_RFIM BIT(12)
+#define RCANFD_RFFIFO_RFDC(x) (((x) & 0x7) << 8)
+#define RCANFD_RFFIFO_RFPLS(x) (((x) & 0x7) << 4)
+#define RCANFD_RFFIFO_RFIE BIT(1)
+#define RCANFD_RFFIFO_RFE BIT(0)
+
+/* Common FIFO bits */
+#define RCANFD_CMFIFO_TML(x) (((x) & 0xf) << 20)
+#define RCANFD_CMFIFO_M(x) (((x) & 0x3) << 16)
+#define RCANFD_CMFIFO_CFIM BIT(12)
+#define RCANFD_CMFIFO_DC(x) (((x) & 0x7) << 8)
+#define RCANFD_CMFIFO_PLS(x) (((x) & 0x7) << 4)
+#define RCANFD_CMFIFO_CFTXIE BIT(2)
+#define RCANFD_CMFIFO_CFE BIT(0)
+
+#define RCANFD_CMFIFO_CFTXIF BIT(4)
+#define RCANFD_CMFIFO_CFMLT BIT(2)
+#define RCANFD_CMFIFO_CFFLL BIT(1)
+#define RCANFD_CMFIFO_CFEMP BIT(0)
+#define RCANFD_CMFIFO_CFMC(x) (((x) >> 8) & 0xff)
+
+#define RCANFD_CMFIFO_CFESI BIT(0)
+#define RCANFD_CMFIFO_CFBRS BIT(1)
+#define RCANFD_CMFIFO_CFFDF BIT(2)
+
+#define RCANFD_CMFIFO_CFIDE BIT(31)
+#define RCANFD_CMFIFO_CFRTR BIT(30)
+#define RCANFD_CMFIFO_CFID(x) ((x) & 0x1fffffff)
+
+#define RCANFD_CMFIFO_CFDLC(x) (((x) & 0xf) << 28)
+#define RCANFD_CMFIFO_CFPTR(x) (((x) & 0xfff) << 16)
+#define RCANFD_CMFIFO_CFTS(x) (((x) & 0xff) << 0)
+
+/* Global Test Config register */
+#define RCANFD_GTSTCFG_C0CBCE BIT(0)
+#define RCANFD_GTSTCFG_C1CBCE BIT(1)
+
+#define RCANFD_GTSTCTR_ICBCTME BIT(0)
+
+/* AFL Rx rules registers */
+#define RCANFD_AFLCFG_SETRNC(n, x) (((x) & 0xff) << (24 - n * 8))
+#define RCANFD_AFLCFG_GETRNC(n, x) (((x) >> (24 - n * 8)) & 0xff)
+
+#define RCANFD_AFL_PAGENUM(entry) ((entry) / 16)
+
+#define RCANFD_AFLCTR_AFLDAE BIT(8)
+#define RCANFD_AFLCTR_AFLPN(x) ((x) & 0x1f)
+#define RCANFD_CHANNEL_NUMRULES 1 /* only one rule per channel */
+#define RCANFD_AFLID_GAFLLB BIT(29)
+#define RCANFD_AFLPTR1_RFFIFO(x) (1 << (x))
+
+/* Common register map offsets */
+
+/* Nominal rate registers */
+#define RCANFD_CCFG(m) (0x0000 + (0x10 * (m)))
+#define RCANFD_CCTR(m) (0x0004 + (0x10 * (m)))
+#define RCANFD_CSTS(m) (0x0008 + (0x10 * (m)))
+#define RCANFD_CERFL(m) (0x000C + (0x10 * (m)))
+
+/* Global registers */
+#define RCANFD_GCFG (0x0084)
+#define RCANFD_GCTR (0x0088)
+#define RCANFD_GSTS (0x008c)
+#define RCANFD_GERFL (0x0090)
+#define RCANFD_GTSC (0x0094)
+#define RCANFD_GAFLECTR (0x0098)
+#define RCANFD_GAFLCFG0 (0x009c)
+#define RCANFD_GAFLCFG1 (0x00a0)
+#define RCANFD_RMNB (0x00a4)
+
+#define RCANFD_RMND(y) (0x00a8 + (0x04 * (y)))
+
+/* Rx FIFO Control registers */
+#define RCANFD_RFCC(x) (0x00b8 + (0x04 * (x)))
+#define RCANFD_RFSTS(x) (0x00d8 + (0x04 * (x)))
+#define RCANFD_RFPCTR(x) (0x00f8 + (0x04 * (x)))
+
+/* Common FIFO Control registers */
+#define RCANFD_CFCC(ch, idx) (0x0118 + (0x0c * (ch)) + \
+ (0x04 * (idx)))
+#define RCANFD_CFSTS(ch, idx) (0x0178 + (0x0c * (ch)) + \
+ (0x04 * (idx)))
+#define RCANFD_CFPCTR(ch, idx) (0x01d8 + (0x0c * (ch)) + \
+ (0x04 * (idx)))
+
+#define RCANFD_FESTS (0x0238)
+#define RCANFD_FFSTS (0x023c)
+#define RCANFD_FMSTS (0x0240)
+#define RCANFD_RFISTS (0x0244)
+#define RCANFD_CFRISTS (0x0248)
+#define RCANFD_CFTISTS (0x024c)
+
+#define RCANFD_TMC(p) (0x0250 + (0x01 * (p)))
+#define RCANFD_TMSTS(p) (0x02d0 + (0x01 * (p)))
+
+#define RCANFD_TMTRSTS(y) (0x0350 + (0x04 * (y)))
+#define RCANFD_TMTARSTS(y) (0x0360 + (0x04 * (y)))
+#define RCANFD_TMTCSTS(y) (0x0370 + (0x04 * (y)))
+#define RCANFD_TMTASTS(y) (0x0380 + (0x04 * (y)))
+#define RCANFD_TMIEC(y) (0x0390 + (0x04 * (y)))
+
+#define RCANFD_TXQCC(m) (0x03a0 + (0x04 * (m)))
+#define RCANFD_TXQSTS(m) (0x03c0 + (0x04 * (m)))
+#define RCANFD_TXQPCTR(m) (0x03e0 + (0x04 * (m)))
+
+#define RCANFD_THLCC(m) (0x0400 + (0x04 * (m)))
+#define RCANFD_THLSTS(m) (0x0420 + (0x04 * (m)))
+#define RCANFD_THLPCTR(m) (0x0440 + (0x04 * (m)))
+
+#define RCANFD_GTINTSTS0 (0x0460)
+#define RCANFD_GTINTSTS1 (0x0464)
+#define RCANFD_GTSTCFG (0x0468)
+#define RCANFD_GTSTCTR (0x046c)
+#define RCANFD_GLOCKK (0x047c)
+#define RCANFD_GRMCFG (0x04fc)
+
+/* Receive rule registers */
+#define RCANFD_GAFLID(offset, j) ((offset) + (0x10 * (j)))
+#define RCANFD_GAFLM(offset, j) ((offset) + 0x04 + (0x10 * (j)))
+#define RCANFD_GAFLP0(offset, j) ((offset) + 0x08 + (0x10 * (j)))
+#define RCANFD_GAFLP1(offset, j) ((offset) + 0x0c + (0x10 * (j)))
+
+/* CAN FD mode specific regsiter map */
+
+/* Data bitrate registers */
+#define RCANFD_F_CDFG(m) (0x0500 + (0x20 * (m)))
+#define RCANFD_F_CFDCFG(m) (0x0504 + (0x20 * (m)))
+#define RCANFD_F_CFDCTR(m) (0x0508 + (0x20 * (m)))
+#define RCANFD_F_CFDSTS(m) (0x050c + (0x20 * (m)))
+#define RCANFD_F_CFDCRC(m) (0x0510 + (0x20 * (m)))
+
+#define RCANFD_F_GAFL_OFFSET (0x1000)
+
+#define RCANFD_F_RMID(q) (0x2000 + (0x10 * (q)))
+#define RCANFD_F_RMPTR(q) (0x2004 + (0x10 * (q)))
+#define RCANFD_F_RMFDSTS(q) (0x2008 + (0x10 * (q)))
+#define RCANFD_F_RMDF(q, b) (0x200c + (0x04 * (b)) + (0x20 * (q)))
+
+/* Rx FIFO data registers */
+#define RCANFD_F_RFOFFSET (0x3000)
+#define RCANFD_F_RFID(x) (RCANFD_F_RFOFFSET + (0x80 * (x)))
+#define RCANFD_F_RFPTR(x) (RCANFD_F_RFOFFSET + 0x04 + \
+ (0x80 * (x)))
+#define RCANFD_F_RFFDSTS(x) (RCANFD_F_RFOFFSET + 0x08 + \
+ (0x80 * (x)))
+#define RCANFD_F_RFDF(x, df) (RCANFD_F_RFOFFSET + 0x0c + \
+ (0x80 * (x)) + (0x04 * (df)))
+
+/* Common FIFO data registers */
+#define RCANFD_F_CFOFFSET (0x3400)
+#define RCANFD_F_CFID(ch, idx) (RCANFD_F_CFOFFSET + (0x180 * (ch)) + \
+ (0x80 * (idx)))
+#define RCANFD_F_CFPTR(ch, idx) (RCANFD_F_CFOFFSET + 0x04 + \
+ (0x180 * (ch)) + (0x80 * (idx)))
+#define RCANFD_F_CFFDCSTS(ch, idx) (RCANFD_F_CFOFFSET + 0x08 + \
+ (0x180 * (ch)) + (0x80 * (idx)))
+#define RCANFD_F_CFDF(ch, idx, df) (RCANFD_F_CFOFFSET + 0x0c + \
+ (0x180 * (ch)) + (0x80 * (idx)) + \
+ (0x04 * (df)))
+
+#define RCANFD_F_TMID(p) (0x4000 + (0x20 * (p)))
+#define RCANFD_F_TMPTR(p) (0x4004 + (0x20 * (p)))
+#define RCANFD_F_TMFDCTR(p) (0x4008 + (0x20 * (p)))
+#define RCANFD_F_TMDF(p, b) (0x400c + (0x20 * (p)) + (0x04 * (b)))
+
+#define RCANFD_F_THLACC(m) (0x6000 + (0x04 * (m)))
+#define RCANFD_F_RPGACC(r) (0x6400 + (0x04 * (r)))
+
+/* fCAN clock select register settings */
+enum rcar_canfd_fcanclk {
+ RCANFD_CANFDCLK = 0, /* CANFD clock */
+ RCANFD_EXTCLK, /* Externally input clock */
+};
+
+struct rcar_canfd_global;
+
+/* Channel priv data */
+struct rcar_canfd_channel {
+ struct can_priv can; /* Must be the first member */
+ struct net_device *ndev;
+ struct rcar_canfd_global *gpriv; /* Controller reference */
+ void __iomem *base; /* Register base address */
+ struct napi_struct napi;
+ u8 tx_len[RCANFD_FIFO_DEPTH]; /* For net stats */
+ u32 tx_head; /* Incremented on xmit */
+ u32 tx_tail; /* Incremented on xmit done */
+ u32 channel; /* Channel number */
+ spinlock_t tx_lock; /* To protect tx path */
+};
+
+/* Global priv data */
+struct rcar_canfd_global {
+ struct rcar_canfd_channel *ch[RCANFD_NUM_CHANNELS];
+ void __iomem *base; /* Register base address */
+ struct platform_device *pdev; /* Respective platform device */
+ struct clk *clkp; /* Peripheral clock */
+ struct clk *can_clk; /* fCAN clock */
+ enum rcar_canfd_fcanclk fcan; /* CANFD or Ext clock */
+ unsigned long channels_mask; /* Enabled channels mask */
+};
+
+/* CAN FD mode nominal rate constants */
+static const struct can_bittiming_const rcar_canfd_nom_bittiming_const = {
+ .name = RCANFD_DRV_NAME,
+ .tseg1_min = 2,
+ .tseg1_max = 128,
+ .tseg2_min = 2,
+ .tseg2_max = 32,
+ .sjw_max = 32,
+ .brp_min = 1,
+ .brp_max = 1024,
+ .brp_inc = 1,
+};
+
+/* CAN FD mode data rate constants */
+static const struct can_bittiming_const rcar_canfd_data_bittiming_const = {
+ .name = RCANFD_DRV_NAME,
+ .tseg1_min = 2,
+ .tseg1_max = 16,
+ .tseg2_min = 2,
+ .tseg2_max = 8,
+ .sjw_max = 8,
+ .brp_min = 1,
+ .brp_max = 256,
+ .brp_inc = 1,
+};
+
+/* Helper functions */
+static inline void rcar_canfd_update(u32 mask, u32 val, u32 __iomem *reg)
+{
+ u32 data = readl(reg);
+
+ data &= ~mask;
+ data |= (val & mask);
+ writel(data, reg);
+}
+
+static inline u32 rcar_canfd_read(void __iomem *base, u32 offset)
+{
+ return readl(base + (offset));
+}
+
+static inline void rcar_canfd_write(void __iomem *base, u32 offset, u32 val)
+{
+ writel(val, base + (offset));
+}
+
+static void rcar_canfd_set_bit(void __iomem *base, u32 reg, u32 val)
+{
+ rcar_canfd_update(val, val, base + (reg));
+}
+
+static void rcar_canfd_clear_bit(void __iomem *base, u32 reg, u32 val)
+{
+ rcar_canfd_update(val, 0, base + (reg));
+}
+
+static void rcar_canfd_update_bit(void __iomem *base, u32 reg,
+ u32 mask, u32 val)
+{
+ rcar_canfd_update(mask, val, base + (reg));
+}
+
+static void rcar_canfd_get_data(struct rcar_canfd_channel *priv,
+ struct canfd_frame *cf, u32 off)
+{
+ u32 i, lwords;
+
+ lwords = DIV_ROUND_UP(cf->len, sizeof(u32));
+ for (i = 0; i < lwords; i++)
+ *((u32 *)cf->data + i) =
+ rcar_canfd_read(priv->base, off + (i * sizeof(u32)));
+}
+
+static void rcar_canfd_put_data(struct rcar_canfd_channel *priv,
+ struct canfd_frame *cf, u32 off)
+{
+ u32 i, lwords;
+
+ lwords = DIV_ROUND_UP(cf->len, sizeof(u32));
+ for (i = 0; i < lwords; i++)
+ rcar_canfd_write(priv->base, off + (i * sizeof(u32)),
+ *((u32 *)cf->data + i));
+}
+
+static void rcar_canfd_tx_failure_cleanup(struct net_device *ndev)
+{
+ u32 i;
+
+ for (i = 0; i < RCANFD_FIFO_DEPTH; i++)
+ can_free_echo_skb(ndev, i);
+}
+
+static int rcar_canfd_reset_controller(struct rcar_canfd_global *gpriv)
+{
+ u32 sts, ch;
+ int err;
+
+ /* Check RAMINIT flag as CAN RAM initialization takes place
+ * after the MCU reset
+ */
+ err = readl_poll_timeout((gpriv->base + RCANFD_GSTS), sts,
+ !(sts & RCANFD_GSTS_RAMINIT), 2, 500000);
+ if (err) {
+ dev_dbg(&gpriv->pdev->dev, "global raminit failed\n");
+ return err;
+ }
+
+ /* Transition to Global Reset mode */
+ rcar_canfd_clear_bit(gpriv->base, RCANFD_GCTR, RCANFD_GCTR_SLPR);
+ rcar_canfd_update_bit(gpriv->base, RCANFD_GCTR,
+ RCANFD_GCTR_MODEMASK, RCANFD_GCTR_GRESET);
+
+ /* Ensure Global reset mode */
+ err = readl_poll_timeout((gpriv->base + RCANFD_GSTS), sts,
+ (sts & RCANFD_GSTS_RESET), 2, 500000);
+ if (err) {
+ dev_dbg(&gpriv->pdev->dev, "global reset failed\n");
+ return err;
+ }
+
+ /* Reset Global error flags */
+ rcar_canfd_write(gpriv->base, RCANFD_GERFL, 0x0);
+
+ /* Set the controller into FD mode */
+ rcar_canfd_set_bit(gpriv->base, RCANFD_GRMCFG, RCANFD_GINTF_CANFD);
+
+ /* Transition all Channels to reset mode */
+ for_each_set_bit(ch, &gpriv->channels_mask, RCANFD_NUM_CHANNELS) {
+ rcar_canfd_clear_bit(gpriv->base,
+ RCANFD_CCTR(ch), RCANFD_CCTR_CSLPR);
+
+ rcar_canfd_update_bit(gpriv->base, RCANFD_CCTR(ch),
+ RCANFD_CCTR_MODEMASK,
+ RCANFD_CCTR_CRESET);
+
+ /* Ensure Channel reset mode */
+ err = readl_poll_timeout((gpriv->base + RCANFD_CSTS(ch)), sts,
+ (sts & RCANFD_CSTS_RESET), 2, 500000);
+ if (err) {
+ dev_dbg(&gpriv->pdev->dev,
+ "channel %u reset failed\n", ch);
+ return err;
+ }
+ }
+ return 0;
+}
+
+static void rcar_canfd_configure_controller(struct rcar_canfd_global *gpriv)
+{
+ u32 cfg, ch;
+
+ /* Global Configuration settings */
+ cfg = RCANFD_GCFG_EEFE; /* ECC error flag enabled */
+
+ /* Set External Clock if selected */
+ if (gpriv->fcan != RCANFD_CANFDCLK)
+ cfg |= RCANFD_GCFG_DCS;
+
+ /* Truncate payload to configured message size RFPLS */
+ cfg |= RCANFD_GCFG_CMPOC;
+
+ rcar_canfd_set_bit(gpriv->base, RCANFD_GCFG, cfg);
+
+ /* Channel configuration settings */
+ for_each_set_bit(ch, &gpriv->channels_mask, RCANFD_NUM_CHANNELS) {
+ rcar_canfd_set_bit(gpriv->base, RCANFD_CCTR(ch),
+ RCANFD_CCTR_ERRD);
+ rcar_canfd_update_bit(gpriv->base, RCANFD_CCTR(ch),
+ RCANFD_CCTR_BOMMASK,
+ RCANFD_CCTR_BOM_ENTRY);
+ }
+}
+
+static void rcar_canfd_configure_afl_rules(struct rcar_canfd_global *gpriv,
+ u32 ch)
+{
+ u32 cfg;
+ int start, page, num_rules = RCANFD_CHANNEL_NUMRULES;
+ u32 ridx = ch + RCANFD_RFFIFO_IDX;
+
+ if (ch == 0) {
+ start = 0; /* Channel 0 always starts from 0th rule */
+ } else {
+ /* Get number of Channel 0 rules and adjust */
+ cfg = rcar_canfd_read(gpriv->base, RCANFD_GAFLCFG0);
+ start = RCANFD_AFLCFG_GETRNC(0, cfg);
+ }
+
+ /* Enable write access to entry */
+ page = RCANFD_AFL_PAGENUM(start);
+ rcar_canfd_set_bit(gpriv->base, RCANFD_GAFLECTR,
+ (RCANFD_AFLCTR_AFLPN(page) | RCANFD_AFLCTR_AFLDAE));
+
+ /* Write number of rules for channel */
+ rcar_canfd_set_bit(gpriv->base, RCANFD_GAFLCFG0,
+ RCANFD_AFLCFG_SETRNC(ch, num_rules));
+
+ /* Accept all IDs */
+ rcar_canfd_write(gpriv->base,
+ RCANFD_GAFLID(RCANFD_F_GAFL_OFFSET, start), 0);
+ /* IDE or RTR is not considered for matching */
+ rcar_canfd_write(gpriv->base,
+ RCANFD_GAFLM(RCANFD_F_GAFL_OFFSET, start), 0);
+ /* Any data length accepted */
+ rcar_canfd_write(gpriv->base,
+ RCANFD_GAFLP0(RCANFD_F_GAFL_OFFSET, start), 0);
+ /* Place the msg in corresponding Rx FIFO entry */
+ rcar_canfd_write(gpriv->base,
+ RCANFD_GAFLP1(RCANFD_F_GAFL_OFFSET, start),
+ RCANFD_AFLPTR1_RFFIFO(ridx));
+
+ /* Disable write access to page */
+ rcar_canfd_clear_bit(gpriv->base,
+ RCANFD_GAFLECTR, RCANFD_AFLCTR_AFLDAE);
+}
+
+static void rcar_canfd_configure_rx(struct rcar_canfd_global *gpriv, u32 ch)
+{
+ /* Rx FIFO is used for reception */
+ u32 cfg;
+ u16 rfdc, rfpls;
+
+ /* Select Rx FIFO based on channel */
+ u32 ridx = ch + RCANFD_RFFIFO_IDX;
+
+ rfdc = 2; /* b010 - 8 messages Rx FIFO depth */
+ rfpls = 7; /* b111 - Max 64 bytes payload */
+
+ cfg = (RCANFD_RFFIFO_RFIM | RCANFD_RFFIFO_RFDC(rfdc) |
+ RCANFD_RFFIFO_RFPLS(rfpls) | RCANFD_RFFIFO_RFIE);
+ rcar_canfd_write(gpriv->base, RCANFD_RFCC(ridx), cfg);
+}
+
+static void rcar_canfd_configure_tx(struct rcar_canfd_global *gpriv, u32 ch)
+{
+ /* Tx/Rx(Common) FIFO configured in Tx mode is
+ * used for transmission
+ *
+ * Each channel has 3 Common FIFO dedicated to them.
+ * Use the 1st (index 0) out of 3
+ */
+ u32 cfg;
+ u16 cftml, cfm, cfdc, cfpls;
+
+ cftml = 0; /* 0th buffer */
+ cfm = 1; /* b01 - Transmit mode */
+ cfdc = 2; /* b010 - 8 messages Tx FIFO depth */
+ cfpls = 7; /* b111 - Max 64 bytes payload */
+
+ cfg = (RCANFD_CMFIFO_TML(cftml) | RCANFD_CMFIFO_M(cfm) |
+ RCANFD_CMFIFO_CFIM | RCANFD_CMFIFO_DC(cfdc) |
+ RCANFD_CMFIFO_PLS(cfpls) | RCANFD_CMFIFO_CFTXIE);
+ rcar_canfd_write(gpriv->base, RCANFD_CFCC(ch, RCANFD_CFFIFO_IDX), cfg);
+
+ /* Clear FD mode specific control/status register */
+ rcar_canfd_write(gpriv->base,
+ RCANFD_F_CFFDCSTS(ch, RCANFD_CFFIFO_IDX), 0);
+}
+
+static void rcar_canfd_enable_global_interrupts(struct rcar_canfd_global *gpriv)
+{
+ u32 ctr;
+
+ /* Clear any stray error interrupt flags */
+ rcar_canfd_write(gpriv->base, RCANFD_GERFL, 0);
+
+ /* Global interrupts setup */
+ ctr = RCANFD_GCTR_MEIE;
+ ctr |= RCANFD_GCTR_CFMPOFIE;
+
+ rcar_canfd_set_bit(gpriv->base, RCANFD_GCTR, ctr);
+}
+
+static void rcar_canfd_disable_global_interrupts(struct rcar_canfd_global
+ *gpriv)
+{
+ /* Disable all interrupts */
+ rcar_canfd_write(gpriv->base, RCANFD_GCTR, 0);
+
+ /* Clear any stray error interrupt flags */
+ rcar_canfd_write(gpriv->base, RCANFD_GERFL, 0);
+}
+
+static void rcar_canfd_enable_channel_interrupts(struct rcar_canfd_channel
+ *priv)
+{
+ u32 ctr, ch = priv->channel;
+
+ /* Clear any stray error flags */
+ rcar_canfd_write(priv->base, RCANFD_CERFL(ch), 0);
+
+ /* Channel interrupts setup */
+ ctr = (RCANFD_CCTR_TAIE |
+ RCANFD_CCTR_ALIE | RCANFD_CCTR_BLIE |
+ RCANFD_CCTR_OLIE | RCANFD_CCTR_BORIE |
+ RCANFD_CCTR_BOEIE | RCANFD_CCTR_EPIE |
+ RCANFD_CCTR_EWIE | RCANFD_CCTR_BEIE);
+ rcar_canfd_set_bit(priv->base, RCANFD_CCTR(ch), ctr);
+}
+
+static void rcar_canfd_disable_channel_interrupts(struct rcar_canfd_channel
+ *priv)
+{
+ u32 ctr, ch = priv->channel;
+
+ ctr = (RCANFD_CCTR_TAIE |
+ RCANFD_CCTR_ALIE | RCANFD_CCTR_BLIE |
+ RCANFD_CCTR_OLIE | RCANFD_CCTR_BORIE |
+ RCANFD_CCTR_BOEIE | RCANFD_CCTR_EPIE |
+ RCANFD_CCTR_EWIE | RCANFD_CCTR_BEIE);
+ rcar_canfd_clear_bit(priv->base, RCANFD_CCTR(ch), ctr);
+
+ /* Clear any stray error flags */
+ rcar_canfd_write(priv->base, RCANFD_CERFL(ch), 0);
+}
+
+static void rcar_canfd_global_error(struct net_device *ndev)
+{
+ struct rcar_canfd_channel *priv = netdev_priv(ndev);
+ struct net_device_stats *stats = &ndev->stats;
+ u32 ch = priv->channel;
+ u32 gerfl, sts;
+ u32 ridx = ch + RCANFD_RFFIFO_IDX;
+
+ gerfl = rcar_canfd_read(priv->base, RCANFD_GERFL);
+ if ((gerfl & RCANFD_GERFL_EEF0) && (ch == 0)) {
+ netdev_dbg(ndev, "Ch0: ECC Error flag\n");
+ stats->tx_dropped++;
+ }
+ if ((gerfl & RCANFD_GERFL_EEF1) && (ch == 1)) {
+ netdev_dbg(ndev, "Ch1: ECC Error flag\n");
+ stats->tx_dropped++;
+ }
+ if (gerfl & RCANFD_GERFL_MES) {
+ sts = rcar_canfd_read(priv->base,
+ RCANFD_CFSTS(ch, RCANFD_CFFIFO_IDX));
+ if (sts & RCANFD_CMFIFO_CFMLT) {
+ netdev_dbg(ndev, "Tx Message Lost flag\n");
+ stats->tx_dropped++;
+ rcar_canfd_write(priv->base,
+ RCANFD_CFSTS(ch, RCANFD_CFFIFO_IDX),
+ sts & ~RCANFD_CMFIFO_CFMLT);
+ }
+
+ sts = rcar_canfd_read(priv->base, RCANFD_RFSTS(ridx));
+ if (sts & RCANFD_RFFIFO_RFMLT) {
+ netdev_dbg(ndev, "Rx Message Lost flag\n");
+ stats->rx_dropped++;
+ rcar_canfd_write(priv->base, RCANFD_RFSTS(ridx),
+ sts & ~RCANFD_RFFIFO_RFMLT);
+ }
+ }
+ if (gerfl & RCANFD_GERFL_CMPOF) {
+ /* Message Lost flag will be set for respective channel
+ * when this condition happens with counters and flags
+ * already updated.
+ */
+ netdev_dbg(ndev, "global payload overflow interrupt\n");
+ }
+
+ /* Clear all global error interrupts. Only affected channels bits
+ * get cleared
+ */
+ rcar_canfd_write(priv->base, RCANFD_GERFL, 0);
+}
+
+static void rcar_canfd_error(struct net_device *ndev)
+{
+ struct rcar_canfd_channel *priv = netdev_priv(ndev);
+ struct net_device_stats *stats = &ndev->stats;
+ struct can_frame *cf;
+ struct sk_buff *skb;
+ u32 cerfl, csts;
+ u32 txerr = 0, rxerr = 0;
+ u32 ch = priv->channel;
+
+ /* Propagate the error condition to the CAN stack */
+ skb = alloc_can_err_skb(ndev, &cf);
+ if (!skb) {
+ stats->rx_dropped++;
+ return;
+ }
+
+ /* Channel error interrupt */
+ cerfl = rcar_canfd_read(priv->base, RCANFD_CERFL(ch));
+ csts = rcar_canfd_read(priv->base, RCANFD_CSTS(ch));
+ txerr = RCANFD_CSTS_TECCNT(csts);
+ rxerr = RCANFD_CSTS_RECCNT(csts);
+
+ netdev_dbg(ndev, "ch erfl %x sts %x txerr %u rxerr %u\n",
+ cerfl, csts, txerr, rxerr);
+
+ if (cerfl & RCANFD_CERFL_BEF) {
+ netdev_dbg(ndev, "Bus error\n");
+ cf->can_id |= CAN_ERR_BUSERROR | CAN_ERR_PROT;
+ cf->data[2] = CAN_ERR_PROT_UNSPEC;
+ priv->can.can_stats.bus_error++;
+ }
+ if (cerfl & RCANFD_CERFL_ADEF) {
+ netdev_dbg(ndev, "ACK Delimiter Error\n");
+ stats->tx_errors++;
+ cf->data[3] |= CAN_ERR_PROT_LOC_ACK_DEL;
+ }
+ if (cerfl & RCANFD_CERFL_B0EF) {
+ netdev_dbg(ndev, "Bit Error (dominant)\n");
+ stats->tx_errors++;
+ cf->data[2] |= CAN_ERR_PROT_BIT0;
+ }
+ if (cerfl & RCANFD_CERFL_B1EF) {
+ netdev_dbg(ndev, "Bit Error (recessive)\n");
+ stats->tx_errors++;
+ cf->data[2] |= CAN_ERR_PROT_BIT1;
+ }
+ if (cerfl & RCANFD_CERFL_CEF) {
+ netdev_dbg(ndev, "CRC Error\n");
+ stats->rx_errors++;
+ cf->data[3] |= CAN_ERR_PROT_LOC_CRC_SEQ;
+ }
+ if (cerfl & RCANFD_CERFL_AEF) {
+ netdev_dbg(ndev, "ACK Error\n");
+ stats->tx_errors++;
+ cf->can_id |= CAN_ERR_ACK;
+ cf->data[3] |= CAN_ERR_PROT_LOC_ACK;
+ }
+ if (cerfl & RCANFD_CERFL_FEF) {
+ netdev_dbg(ndev, "Form Error\n");
+ stats->rx_errors++;
+ cf->data[2] |= CAN_ERR_PROT_FORM;
+ }
+ if (cerfl & RCANFD_CERFL_SEF) {
+ netdev_dbg(ndev, "Stuff Error\n");
+ stats->rx_errors++;
+ cf->data[2] |= CAN_ERR_PROT_STUFF;
+ }
+ if (cerfl & RCANFD_CERFL_ALEF) {
+ netdev_dbg(ndev, "Arbitration lost Error\n");
+ priv->can.can_stats.arbitration_lost++;
+ cf->can_id |= CAN_ERR_LOSTARB;
+ cf->data[0] |= CAN_ERR_LOSTARB_UNSPEC;
+ }
+ if (cerfl & RCANFD_CERFL_BLEF) {
+ netdev_dbg(ndev, "Bus Lock Error\n");
+ stats->rx_errors++;
+ cf->can_id |= CAN_ERR_BUSERROR;
+ }
+ if (cerfl & RCANFD_CERFL_EWEF) {
+ netdev_dbg(ndev, "Error warning interrupt\n");
+ priv->can.state = CAN_STATE_ERROR_WARNING;
+ priv->can.can_stats.error_warning++;
+ cf->can_id |= CAN_ERR_CRTL;
+ cf->data[1] = txerr > rxerr ? CAN_ERR_CRTL_TX_WARNING :
+ CAN_ERR_CRTL_RX_WARNING;
+ cf->data[6] = txerr;
+ cf->data[7] = rxerr;
+ }
+ if (cerfl & RCANFD_CERFL_EPEF) {
+ netdev_dbg(ndev, "Error passive interrupt\n");
+ priv->can.state = CAN_STATE_ERROR_PASSIVE;
+ priv->can.can_stats.error_passive++;
+ cf->can_id |= CAN_ERR_CRTL;
+ cf->data[1] = txerr > rxerr ? CAN_ERR_CRTL_TX_PASSIVE :
+ CAN_ERR_CRTL_RX_PASSIVE;
+ cf->data[6] = txerr;
+ cf->data[7] = rxerr;
+ }
+ if (cerfl & RCANFD_CERFL_BOEEF) {
+ netdev_dbg(ndev, "Bus-off entry interrupt\n");
+ rcar_canfd_tx_failure_cleanup(ndev);
+ priv->can.state = CAN_STATE_BUS_OFF;
+ priv->can.can_stats.bus_off++;
+ can_bus_off(ndev);
+ cf->can_id |= CAN_ERR_BUSOFF;
+ }
+ if (cerfl & RCANFD_CERFL_OLEF) {
+ netdev_dbg(ndev,
+ "Overload Frame Transmission error interrupt\n");
+ stats->tx_errors++;
+ cf->can_id |= CAN_ERR_PROT;
+ cf->data[2] |= CAN_ERR_PROT_OVERLOAD;
+ }
+
+ /* Clear all channel error interrupts */
+ rcar_canfd_write(priv->base, RCANFD_CERFL(ch), 0);
+ stats->rx_packets++;
+ stats->rx_bytes += cf->can_dlc;
+ netif_rx(skb);
+}
+
+static void rcar_canfd_tx_done(struct net_device *ndev)
+{
+ struct rcar_canfd_channel *priv = netdev_priv(ndev);
+ struct net_device_stats *stats = &ndev->stats;
+ u32 sts;
+ unsigned long flags;
+ u32 ch = priv->channel;
+
+ do {
+ u8 unsent, sent;
+
+ sent = priv->tx_tail % RCANFD_FIFO_DEPTH;
+ stats->tx_packets++;
+ stats->tx_bytes += priv->tx_len[sent];
+ priv->tx_len[sent] = 0;
+ can_get_echo_skb(ndev, sent);
+
+ spin_lock_irqsave(&priv->tx_lock, flags);
+ priv->tx_tail++;
+ sts = rcar_canfd_read(priv->base,
+ RCANFD_CFSTS(ch, RCANFD_CFFIFO_IDX));
+ unsent = RCANFD_CMFIFO_CFMC(sts);
+
+ /* Wake producer only when there is room */
+ if (unsent != RCANFD_FIFO_DEPTH)
+ netif_wake_queue(ndev);
+
+ if (priv->tx_head - priv->tx_tail <= unsent) {
+ spin_unlock_irqrestore(&priv->tx_lock, flags);
+ break;
+ }
+ spin_unlock_irqrestore(&priv->tx_lock, flags);
+
+ } while (1);
+
+ /* Clear interrupt */
+ rcar_canfd_write(priv->base, RCANFD_CFSTS(ch, RCANFD_CFFIFO_IDX),
+ sts & ~RCANFD_CMFIFO_CFTXIF);
+ can_led_event(ndev, CAN_LED_EVENT_TX);
+}
+
+static irqreturn_t rcar_canfd_global_interrupt(int irq, void *dev_id)
+{
+ struct rcar_canfd_global *gpriv = dev_id;
+ struct net_device *ndev;
+ struct rcar_canfd_channel *priv;
+ u32 sts, gerfl;
+ u32 ch, ridx;
+
+ /* Global error interrupts still indicate a condition specific
+ * to a channel. RxFIFO interrupt is a global interrupt.
+ */
+ for_each_set_bit(ch, &gpriv->channels_mask, RCANFD_NUM_CHANNELS) {
+ priv = gpriv->ch[ch];
+ ndev = priv->ndev;
+ ridx = ch + RCANFD_RFFIFO_IDX;
+
+ /* Global error interrupts */
+ gerfl = rcar_canfd_read(priv->base, RCANFD_GERFL);
+ if (RCANFD_GERFL_ERR(gerfl))
+ rcar_canfd_global_error(ndev);
+
+ /* Handle Rx interrupts */
+ sts = rcar_canfd_read(priv->base, RCANFD_RFSTS(ridx));
+ if (sts & RCANFD_RFFIFO_RFIF) {
+ if (napi_schedule_prep(&priv->napi)) {
+ /* Disable Rx FIFO interrupts */
+ rcar_canfd_clear_bit(priv->base,
+ RCANFD_RFCC(ridx),
+ RCANFD_RFFIFO_RFIE);
+ __napi_schedule(&priv->napi);
+ }
+ }
+ }
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t rcar_canfd_channel_interrupt(int irq, void *dev_id)
+{
+ struct rcar_canfd_global *gpriv = dev_id;
+ struct net_device *ndev;
+ struct rcar_canfd_channel *priv;
+ u32 sts, cerfl, ch;
+
+ /* Common FIFO is a per channel resource */
+ for_each_set_bit(ch, &gpriv->channels_mask, RCANFD_NUM_CHANNELS) {
+ priv = gpriv->ch[ch];
+ ndev = priv->ndev;
+
+ /* Channel error interrupts */
+ cerfl = rcar_canfd_read(priv->base, RCANFD_CERFL(ch));
+ if (RCANFD_CERFL_ERR(cerfl))
+ rcar_canfd_error(ndev);
+
+ /* Handle Tx interrupts */
+ sts = rcar_canfd_read(priv->base,
+ RCANFD_CFSTS(ch, RCANFD_CFFIFO_IDX));
+ if (sts & RCANFD_CMFIFO_CFTXIF)
+ rcar_canfd_tx_done(ndev);
+ }
+ return IRQ_HANDLED;
+}
+
+static void rcar_canfd_set_bittiming(struct net_device *dev)
+{
+ struct rcar_canfd_channel *priv = netdev_priv(dev);
+ const struct can_bittiming *bt = &priv->can.bittiming;
+ const struct can_bittiming *dbt = &priv->can.data_bittiming;
+ u16 brp, sjw, tseg1, tseg2;
+ u32 cfg;
+ u32 ch = priv->channel;
+
+ /* Nominal bit timing settings */
+ brp = bt->brp - 1;
+ sjw = bt->sjw - 1;
+ tseg1 = bt->prop_seg + bt->phase_seg1 - 1;
+ tseg2 = bt->phase_seg2 - 1;
+
+ cfg = (RCANFD_NR_TSEG1(tseg1) | RCANFD_NR_BRP(brp) |
+ RCANFD_NR_SJW(sjw) | RCANFD_NR_TSEG2(tseg2));
+
+ rcar_canfd_write(priv->base, RCANFD_CCFG(ch), cfg);
+ netdev_dbg(priv->ndev, "nrate: brp %u, sjw %u, tseg1 %u, tseg2 %u\n",
+ brp, sjw, tseg1, tseg2);
+
+ /* Data bit timing settings */
+ brp = dbt->brp - 1;
+ sjw = dbt->sjw - 1;
+ tseg1 = dbt->prop_seg + dbt->phase_seg1 - 1;
+ tseg2 = dbt->phase_seg2 - 1;
+
+ cfg = (RCANFD_DR_TSEG1(tseg1) | RCANFD_DR_BRP(brp) |
+ RCANFD_DR_SJW(sjw) | RCANFD_DR_TSEG2(tseg2));
+
+ rcar_canfd_write(priv->base, RCANFD_F_CDFG(ch), cfg);
+ netdev_dbg(priv->ndev, "drate: brp %u, sjw %u, tseg1 %u, tseg2 %u\n",
+ brp, sjw, tseg1, tseg2);
+}
+
+static int rcar_canfd_start(struct net_device *ndev)
+{
+ struct rcar_canfd_channel *priv = netdev_priv(ndev);
+ int err = -EOPNOTSUPP;
+ u32 sts, ch = priv->channel;
+ u32 ridx = ch + RCANFD_RFFIFO_IDX;
+
+ rcar_canfd_set_bittiming(ndev);
+
+ rcar_canfd_enable_channel_interrupts(priv);
+
+ /* Set channel to Operational mode */
+ rcar_canfd_update_bit(priv->base, RCANFD_CCTR(ch),
+ RCANFD_CCTR_MODEMASK, RCANFD_CCTR_COPM);
+
+ /* Verify channel mode change */
+ err = readl_poll_timeout((priv->base + RCANFD_CSTS(ch)), sts,
+ (sts & RCANFD_CSTS_COM), 2, 500000);
+ if (err) {
+ netdev_err(ndev, "channel %u communication state failed\n", ch);
+ goto fail_mode_change;
+ }
+
+ /* Enable Common & Rx FIFO */
+ rcar_canfd_set_bit(priv->base, RCANFD_CFCC(ch, RCANFD_CFFIFO_IDX),
+ RCANFD_CMFIFO_CFE);
+ rcar_canfd_set_bit(priv->base, RCANFD_RFCC(ridx), RCANFD_RFFIFO_RFE);
+
+ priv->can.state = CAN_STATE_ERROR_ACTIVE;
+ return 0;
+
+fail_mode_change:
+ rcar_canfd_disable_channel_interrupts(priv);
+ return err;
+}
+
+static int rcar_canfd_open(struct net_device *ndev)
+{
+ struct rcar_canfd_channel *priv = netdev_priv(ndev);
+ struct rcar_canfd_global *gpriv = priv->gpriv;
+ int err;
+
+ /* Peripheral clock is already enabled in probe */
+ err = clk_prepare_enable(gpriv->can_clk);
+ if (err) {
+ netdev_err(ndev, "failed to enable CAN clock, error %d\n", err);
+ goto out_clock;
+ }
+
+ err = open_candev(ndev);
+ if (err) {
+ netdev_err(ndev, "open_candev() failed, error %d\n", err);
+ goto out_can_clock;
+ }
+
+ napi_enable(&priv->napi);
+ err = rcar_canfd_start(ndev);
+ if (err)
+ goto out_close;
+ netif_start_queue(ndev);
+ can_led_event(ndev, CAN_LED_EVENT_OPEN);
+ return 0;
+out_close:
+ napi_disable(&priv->napi);
+ close_candev(ndev);
+out_can_clock:
+ clk_disable_unprepare(gpriv->can_clk);
+out_clock:
+ return err;
+}
+
+static void rcar_canfd_stop(struct net_device *ndev)
+{
+ struct rcar_canfd_channel *priv = netdev_priv(ndev);
+ int err;
+ u32 sts, ch = priv->channel;
+ u32 ridx = ch + RCANFD_RFFIFO_IDX;
+
+ /* Transition to channel reset mode */
+ rcar_canfd_update_bit(priv->base, RCANFD_CCTR(ch),
+ RCANFD_CCTR_MODEMASK, RCANFD_CCTR_CRESET);
+
+ /* Check Channel reset mode */
+ err = readl_poll_timeout((priv->base + RCANFD_CSTS(ch)), sts,
+ (sts & RCANFD_CSTS_RESET), 2, 500000);
+ if (err)
+ netdev_err(ndev, "channel %u reset failed\n", ch);
+
+ rcar_canfd_disable_channel_interrupts(priv);
+
+ /* Disable Common & Rx FIFO */
+ rcar_canfd_clear_bit(priv->base, RCANFD_CFCC(ch, RCANFD_CFFIFO_IDX),
+ RCANFD_CMFIFO_CFE);
+ rcar_canfd_clear_bit(priv->base, RCANFD_RFCC(ridx), RCANFD_RFFIFO_RFE);
+
+ /* Set the state as STOPPED */
+ priv->can.state = CAN_STATE_STOPPED;
+}
+
+static int rcar_canfd_close(struct net_device *ndev)
+{
+ struct rcar_canfd_channel *priv = netdev_priv(ndev);
+ struct rcar_canfd_global *gpriv = priv->gpriv;
+
+ netif_stop_queue(ndev);
+ rcar_canfd_stop(ndev);
+ napi_disable(&priv->napi);
+ clk_disable_unprepare(gpriv->can_clk);
+ close_candev(ndev);
+ can_led_event(ndev, CAN_LED_EVENT_STOP);
+ return 0;
+}
+
+static netdev_tx_t rcar_canfd_start_xmit(struct sk_buff *skb,
+ struct net_device *ndev)
+{
+ struct rcar_canfd_channel *priv = netdev_priv(ndev);
+ struct canfd_frame *cf = (struct canfd_frame *)skb->data;
+ u32 sts = 0, id, dlc;
+ unsigned long flags;
+ u32 ch = priv->channel;
+
+ if (can_dropped_invalid_skb(ndev, skb))
+ return NETDEV_TX_OK;
+
+ if (cf->can_id & CAN_EFF_FLAG) {
+ id = cf->can_id & CAN_EFF_MASK;
+ id |= RCANFD_CMFIFO_CFIDE;
+ } else {
+ id = cf->can_id & CAN_SFF_MASK;
+ }
+
+ if (cf->can_id & CAN_RTR_FLAG)
+ id |= RCANFD_CMFIFO_CFRTR;
+
+ rcar_canfd_write(priv->base,
+ RCANFD_F_CFID(ch, RCANFD_CFFIFO_IDX), id);
+ dlc = RCANFD_CMFIFO_CFDLC(can_len2dlc(cf->len));
+ rcar_canfd_write(priv->base,
+ RCANFD_F_CFPTR(ch, RCANFD_CFFIFO_IDX), dlc);
+
+ if (can_is_canfd_skb(skb)) {
+ /* CAN FD frame format */
+ sts |= RCANFD_CMFIFO_CFFDF;
+ if (cf->flags & CANFD_BRS)
+ sts |= RCANFD_CMFIFO_CFBRS;
+
+ if (priv->can.state == CAN_STATE_ERROR_PASSIVE)
+ sts |= RCANFD_CMFIFO_CFESI;
+ }
+
+ rcar_canfd_write(priv->base, RCANFD_F_CFFDCSTS(ch, RCANFD_CFFIFO_IDX),
+ sts);
+
+ rcar_canfd_put_data(priv, cf,
+ RCANFD_F_CFDF(ch, RCANFD_CFFIFO_IDX, 0));
+
+ priv->tx_len[priv->tx_head % RCANFD_FIFO_DEPTH] = cf->len;
+ can_put_echo_skb(skb, ndev, priv->tx_head % RCANFD_FIFO_DEPTH);
+
+ spin_lock_irqsave(&priv->tx_lock, flags);
+ priv->tx_head++;
+
+ /* Stop the queue if we've filled all FIFO entries */
+ if (priv->tx_head - priv->tx_tail >= RCANFD_FIFO_DEPTH)
+ netif_stop_queue(ndev);
+
+ /* Start Tx: Write 0xff to CFPC to increment the CPU-side
+ * pointer for the Common FIFO
+ */
+ rcar_canfd_write(priv->base,
+ RCANFD_CFPCTR(ch, RCANFD_CFFIFO_IDX), 0xff);
+
+ spin_unlock_irqrestore(&priv->tx_lock, flags);
+ return NETDEV_TX_OK;
+}
+
+static void rcar_canfd_rx_pkt(struct rcar_canfd_channel *priv)
+{
+ struct net_device_stats *stats = &priv->ndev->stats;
+ struct canfd_frame *cf;
+ struct sk_buff *skb;
+ u32 sts = 0, id, ptr;
+ u32 ch = priv->channel;
+ u32 ridx = ch + RCANFD_RFFIFO_IDX;
+
+ id = rcar_canfd_read(priv->base, RCANFD_F_RFID(ridx));
+ ptr = rcar_canfd_read(priv->base, RCANFD_F_RFPTR(ridx));
+
+ sts = rcar_canfd_read(priv->base, RCANFD_F_RFFDSTS(ridx));
+ if (sts & RCANFD_RFFIFO_RFFDF)
+ skb = alloc_canfd_skb(priv->ndev, &cf);
+ else
+ skb = alloc_can_skb(priv->ndev,
+ (struct can_frame **)&cf);
+
+ if (!skb) {
+ stats->rx_dropped++;
+ return;
+ }
+
+ if (sts & RCANFD_RFFIFO_RFFDF)
+ cf->len = can_dlc2len(RCANFD_RFFIFO_RFDLC(ptr));
+ else
+ cf->len = get_can_dlc(RCANFD_RFFIFO_RFDLC(ptr));
+
+ if (sts & RCANFD_RFFIFO_RFESI) {
+ cf->flags |= CANFD_ESI;
+ netdev_dbg(priv->ndev, "ESI Error\n");
+ }
+
+ if (id & RCANFD_RFFIFO_RFIDE)
+ cf->can_id = (id & CAN_EFF_MASK) | CAN_EFF_FLAG;
+ else
+ cf->can_id = id & CAN_SFF_MASK;
+
+ if (!(sts & RCANFD_RFFIFO_RFFDF) && (id & RCANFD_RFFIFO_RFRTR)) {
+ cf->can_id |= CAN_RTR_FLAG;
+ } else {
+ if (sts & RCANFD_RFFIFO_RFBRS)
+ cf->flags |= CANFD_BRS;
+
+ rcar_canfd_get_data(priv, cf, RCANFD_F_RFDF(ridx, 0));
+ }
+
+ /* Write 0xff to RFPC to increment the CPU-side
+ * pointer of the Rx FIFO
+ */
+ rcar_canfd_write(priv->base, RCANFD_RFPCTR(ridx), 0xff);
+
+ can_led_event(priv->ndev, CAN_LED_EVENT_RX);
+
+ stats->rx_bytes += cf->len;
+ stats->rx_packets++;
+ netif_receive_skb(skb);
+}
+
+static int rcar_canfd_rx_poll(struct napi_struct *napi, int quota)
+{
+ struct rcar_canfd_channel *priv =
+ container_of(napi, struct rcar_canfd_channel, napi);
+ int num_pkts;
+ u32 sts;
+ u32 ch = priv->channel;
+ u32 ridx = ch + RCANFD_RFFIFO_IDX;
+
+ for (num_pkts = 0; num_pkts < quota; num_pkts++) {
+ sts = rcar_canfd_read(priv->base, RCANFD_RFSTS(ridx));
+ /* Check FIFO empty condition */
+ if (sts & RCANFD_RFFIFO_RFEMP)
+ break;
+
+ rcar_canfd_rx_pkt(priv);
+
+ /* Clear interrupt bit */
+ if (sts & RCANFD_RFFIFO_RFIF)
+ rcar_canfd_write(priv->base, RCANFD_RFSTS(ridx),
+ sts & ~RCANFD_RFFIFO_RFIF);
+ }
+
+ /* All packets processed */
+ if (num_pkts < quota) {
+ napi_complete(napi);
+ /* Enable Rx FIFO interrupts */
+ rcar_canfd_set_bit(priv->base, RCANFD_RFCC(ridx),
+ RCANFD_RFFIFO_RFIE);
+ }
+ return num_pkts;
+}
+
+static int rcar_canfd_do_set_mode(struct net_device *ndev, enum can_mode mode)
+{
+ int err;
+
+ switch (mode) {
+ case CAN_MODE_START:
+ err = rcar_canfd_start(ndev);
+ if (err)
+ return err;
+ netif_wake_queue(ndev);
+ return 0;
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static int rcar_canfd_get_berr_counter(const struct net_device *dev,
+ struct can_berr_counter *bec)
+{
+ struct rcar_canfd_channel *priv = netdev_priv(dev);
+ u32 val, ch = priv->channel;
+
+ /* Peripheral clock is already enabled in probe */
+ val = rcar_canfd_read(priv->base, RCANFD_CSTS(ch));
+ bec->txerr = RCANFD_CSTS_TECCNT(val);
+ bec->rxerr = RCANFD_CSTS_RECCNT(val);
+ return 0;
+}
+
+static const struct net_device_ops rcar_canfd_netdev_ops = {
+ .ndo_open = rcar_canfd_open,
+ .ndo_stop = rcar_canfd_close,
+ .ndo_start_xmit = rcar_canfd_start_xmit,
+ .ndo_change_mtu = can_change_mtu,
+};
+
+static int rcar_canfd_channel_probe(struct rcar_canfd_global *gpriv, u32 ch,
+ u32 fcan_freq)
+{
+ struct platform_device *pdev = gpriv->pdev;
+ struct rcar_canfd_channel *priv;
+ struct net_device *ndev;
+ int err = -ENODEV;
+
+ ndev = alloc_candev(sizeof(*priv), RCANFD_FIFO_DEPTH);
+ if (!ndev) {
+ dev_err(&pdev->dev, "alloc_candev() failed\n");
+ err = -ENOMEM;
+ goto fail;
+ }
+ priv = netdev_priv(ndev);
+
+ ndev->netdev_ops = &rcar_canfd_netdev_ops;
+ ndev->flags |= IFF_ECHO;
+ priv->ndev = ndev;
+ priv->base = gpriv->base;
+ priv->channel = ch;
+ priv->can.clock.freq = fcan_freq;
+ dev_info(&pdev->dev, "can_clk rate is %u\n", priv->can.clock.freq);
+
+ priv->can.bittiming_const = &rcar_canfd_nom_bittiming_const;
+ priv->can.data_bittiming_const =
+ &rcar_canfd_data_bittiming_const;
+
+ /* Controller starts in CAN FD only mode */
+ can_set_static_ctrlmode(ndev, CAN_CTRLMODE_FD);
+ priv->can.ctrlmode_supported = CAN_CTRLMODE_BERR_REPORTING;
+
+ priv->can.do_set_mode = rcar_canfd_do_set_mode;
+ priv->can.do_get_berr_counter = rcar_canfd_get_berr_counter;
+ priv->gpriv = gpriv;
+ SET_NETDEV_DEV(ndev, &pdev->dev);
+
+ netif_napi_add(ndev, &priv->napi, rcar_canfd_rx_poll,
+ RCANFD_NAPI_WEIGHT);
+ err = register_candev(ndev);
+ if (err) {
+ dev_err(&pdev->dev,
+ "register_candev() failed, error %d\n", err);
+ goto fail_candev;
+ }
+ spin_lock_init(&priv->tx_lock);
+ devm_can_led_init(ndev);
+ gpriv->ch[priv->channel] = priv;
+ dev_info(&pdev->dev, "device registered (channel %u)\n", priv->channel);
+ return 0;
+
+fail_candev:
+ netif_napi_del(&priv->napi);
+ free_candev(ndev);
+fail:
+ return err;
+}
+
+static void rcar_canfd_channel_remove(struct rcar_canfd_global *gpriv, u32 ch)
+{
+ struct rcar_canfd_channel *priv = gpriv->ch[ch];
+
+ if (priv) {
+ unregister_candev(priv->ndev);
+ netif_napi_del(&priv->napi);
+ free_candev(priv->ndev);
+ }
+}
+
+static int rcar_canfd_probe(struct platform_device *pdev)
+{
+ struct resource *mem;
+ void __iomem *addr;
+ u32 sts, ch, fcan_freq;
+ struct rcar_canfd_global *gpriv;
+ struct device_node *of_child;
+ unsigned long channels_mask = 0;
+ int err, ch_irq, g_irq;
+
+ of_child = of_get_child_by_name(pdev->dev.of_node, "channel0");
+ if (of_child && of_device_is_available(of_child))
+ channels_mask |= BIT(0); /* Channel 0 */
+
+ of_child = of_get_child_by_name(pdev->dev.of_node, "channel1");
+ if (of_child && of_device_is_available(of_child))
+ channels_mask |= BIT(1); /* Channel 1 */
+
+ ch_irq = platform_get_irq(pdev, 0);
+ if (ch_irq < 0) {
+ dev_err(&pdev->dev, "no Channel IRQ resource\n");
+ err = ch_irq;
+ goto fail_dev;
+ }
+
+ g_irq = platform_get_irq(pdev, 1);
+ if (g_irq < 0) {
+ dev_err(&pdev->dev, "no Global IRQ resource\n");
+ err = g_irq;
+ goto fail_dev;
+ }
+
+ /* Global controller context */
+ gpriv = devm_kzalloc(&pdev->dev, sizeof(*gpriv), GFP_KERNEL);
+ if (!gpriv) {
+ err = -ENOMEM;
+ goto fail_dev;
+ }
+ gpriv->pdev = pdev;
+ gpriv->channels_mask = channels_mask;
+
+ /* Peripheral clock */
+ gpriv->clkp = devm_clk_get(&pdev->dev, "fck");
+ if (IS_ERR(gpriv->clkp)) {
+ err = PTR_ERR(gpriv->clkp);
+ dev_err(&pdev->dev, "cannot get peripheral clock, error %d\n",
+ err);
+ goto fail_dev;
+ }
+
+ /* fCAN clock: Pick External clock. If not available fallback to
+ * CANFD clock
+ */
+ gpriv->can_clk = devm_clk_get(&pdev->dev, "can_clk");
+ if (IS_ERR(gpriv->can_clk) || (clk_get_rate(gpriv->can_clk) == 0)) {
+ gpriv->can_clk = devm_clk_get(&pdev->dev, "canfd");
+ if (IS_ERR(gpriv->can_clk)) {
+ err = PTR_ERR(gpriv->can_clk);
+ dev_err(&pdev->dev,
+ "cannot get canfd clock, error %d\n", err);
+ goto fail_dev;
+ }
+ gpriv->fcan = RCANFD_CANFDCLK;
+
+ } else {
+ gpriv->fcan = RCANFD_EXTCLK;
+ }
+ fcan_freq = clk_get_rate(gpriv->can_clk);
+
+ if (gpriv->fcan == RCANFD_CANFDCLK)
+ /* CANFD clock is further divided by (1/2) within the IP */
+ fcan_freq /= 2;
+
+ mem = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ addr = devm_ioremap_resource(&pdev->dev, mem);
+ if (IS_ERR(addr)) {
+ err = PTR_ERR(addr);
+ goto fail_dev;
+ }
+ gpriv->base = addr;
+
+ /* Request IRQ that's common for both channels */
+ err = devm_request_irq(&pdev->dev, ch_irq,
+ rcar_canfd_channel_interrupt, 0,
+ "canfd.chn", gpriv);
+ if (err) {
+ dev_err(&pdev->dev, "devm_request_irq(%d) failed, error %d\n",
+ ch_irq, err);
+ goto fail_dev;
+ }
+ err = devm_request_irq(&pdev->dev, g_irq,
+ rcar_canfd_global_interrupt, 0,
+ "canfd.gbl", gpriv);
+ if (err) {
+ dev_err(&pdev->dev, "devm_request_irq(%d) failed, error %d\n",
+ g_irq, err);
+ goto fail_dev;
+ }
+
+ /* Enable peripheral clock for register access */
+ err = clk_prepare_enable(gpriv->clkp);
+ if (err) {
+ dev_err(&pdev->dev,
+ "failed to enable peripheral clock, error %d\n", err);
+ goto fail_dev;
+ }
+
+ err = rcar_canfd_reset_controller(gpriv);
+ if (err) {
+ dev_err(&pdev->dev, "reset controller failed\n");
+ goto fail_clk;
+ }
+
+ /* Controller in Global reset & Channel reset mode */
+ rcar_canfd_configure_controller(gpriv);
+
+ /* Configure per channel attributes */
+ for_each_set_bit(ch, &gpriv->channels_mask, RCANFD_NUM_CHANNELS) {
+ /* Configure Channel's Rx fifo */
+ rcar_canfd_configure_rx(gpriv, ch);
+
+ /* Configure Channel's Tx (Common) fifo */
+ rcar_canfd_configure_tx(gpriv, ch);
+
+ /* Configure receive rules */
+ rcar_canfd_configure_afl_rules(gpriv, ch);
+ }
+
+ /* Configure common interrupts */
+ rcar_canfd_enable_global_interrupts(gpriv);
+
+ /* Start Global operation mode */
+ rcar_canfd_update_bit(gpriv->base, RCANFD_GCTR, RCANFD_GCTR_MODEMASK,
+ RCANFD_GCTR_GOPM);
+
+ /* Verify mode change */
+ err = readl_poll_timeout((gpriv->base + RCANFD_GSTS), sts,
+ !(sts & RCANFD_GSTS_GNOPM), 2, 500000);
+ if (err) {
+ dev_err(&pdev->dev, "global operational mode failed\n");
+ goto fail_mode;
+ }
+
+ for_each_set_bit(ch, &gpriv->channels_mask, RCANFD_NUM_CHANNELS) {
+ err = rcar_canfd_channel_probe(gpriv, ch, fcan_freq);
+ if (err)
+ goto fail_channel;
+ }
+
+ platform_set_drvdata(pdev, gpriv);
+ dev_info(&pdev->dev, "global operational state (clk %d)\n",
+ gpriv->fcan);
+ return 0;
+
+fail_channel:
+ for_each_set_bit(ch, &gpriv->channels_mask, RCANFD_NUM_CHANNELS)
+ rcar_canfd_channel_remove(gpriv, ch);
+fail_mode:
+ rcar_canfd_disable_global_interrupts(gpriv);
+fail_clk:
+ clk_disable_unprepare(gpriv->clkp);
+fail_dev:
+ return err;
+}
+
+static int rcar_canfd_remove(struct platform_device *pdev)
+{
+ struct rcar_canfd_global *gpriv = platform_get_drvdata(pdev);
+ u32 ch;
+
+ rcar_canfd_reset_controller(gpriv);
+ rcar_canfd_disable_global_interrupts(gpriv);
+
+ for_each_set_bit(ch, &gpriv->channels_mask, RCANFD_NUM_CHANNELS) {
+ rcar_canfd_disable_channel_interrupts(gpriv->ch[ch]);
+ rcar_canfd_channel_remove(gpriv, ch);
+ }
+
+ /* Enter global sleep mode */
+ rcar_canfd_set_bit(gpriv->base, RCANFD_GCTR, RCANFD_GCTR_SLPR);
+ clk_disable_unprepare(gpriv->clkp);
+ return 0;
+}
+
+static int __maybe_unused rcar_canfd_suspend(struct device *dev)
+{
+ return 0;
+}
+
+static int __maybe_unused rcar_canfd_resume(struct device *dev)
+{
+ return 0;
+}
+
+static SIMPLE_DEV_PM_OPS(rcar_canfd_pm_ops, rcar_canfd_suspend,
+ rcar_canfd_resume);
+
+static const struct of_device_id rcar_canfd_of_table[] = {
+ { .compatible = "renesas,rcar-gen3-canfd" },
+ { }
+};
+
+MODULE_DEVICE_TABLE(of, rcar_canfd_of_table);
+
+static struct platform_driver rcar_canfd_driver = {
+ .driver = {
+ .name = RCANFD_DRV_NAME,
+ .of_match_table = of_match_ptr(rcar_canfd_of_table),
+ .pm = &rcar_canfd_pm_ops,
+ },
+ .probe = rcar_canfd_probe,
+ .remove = rcar_canfd_remove,
+};
+
+module_platform_driver(rcar_canfd_driver);
+
+MODULE_AUTHOR("Ramesh Shanmugasundaram <ramesh.shanmugasundaram@bp.renesas.com>");
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("CAN FD driver for Renesas R-Car SoC");
+MODULE_ALIAS("platform:" RCANFD_DRV_NAME);
--
1.9.1
^ permalink raw reply related
* [PATCH v5 0/2] Add CAN FD driver support to r8a7795 SoC
From: Ramesh Shanmugasundaram @ 2016-04-28 12:23 UTC (permalink / raw)
To: mkl, wg, socketcan, robh+dt, pawel.moll, mark.rutland,
ijc+devicetree, galak
Cc: linux-renesas-soc, devicetree, linux-can, netdev, chris.paterson2,
Ramesh Shanmugasundaram
In-Reply-To: <1456824849-7987-1-git-send-email-ramesh.shanmugasundaram@bp.renesas.com>
Hi All,
This patch is based on commit "885cc17abad6" of can-next/testing branch
Changes since v4:
* Rebased to latest can-next/testing branch
* Addressed Marc K review comments on v2 patch
* Use clk_get_rate() to find ext clock availability. Refer thread
(https://lkml.org/lkml/2016/4/8/203)
Changes since v3:
* Split v3 patch into two (Marc K)
(1) Introduce rcar_canfd.c driver in drivers/net/can/rcar dir
(2) Move rcar_can.c driver to new drivers/net/can/rcar dir
* Added Acked-by Rob H for the bindings
* Added comments on CAN FD only mode in driver & Kconfig (Oliver H)
Changes since v2:
* Rebased to latest tag (linux-can-next-for-4.6-20160310)
* Cleaned up leftover debugfs code (Thanks Oliver H)
* Revised devicetree documentation text (as suggested by Rob H)
(https://www.mail-archive.com/linux-renesas-soc@vger.kernel.org/msg01597.html)
* Used new can subsystem api to set static configuration & removed check in rcar_canfd_start (as suggested by Oliver H)
(Refer: http://comments.gmane.org/gmane.linux.can/9126 &
https://www.mail-archive.com/linux-renesas-soc@vger.kernel.org/msg01867.html)
* Clubbed Renesas controller drivers to driver/net/can/rcar dir (as suggested by Oliver H)
* Updated commit message
Changes since v1:
* Removed testmodes & debugfs code (suggested by Oliver H)
* Fixed tx path race issue by introducing lock (suggested by Marc K)
* Removed __maybe_unused attribute of rcar_canfd_of_table
Ramesh Shanmugasundaram (2):
can: rcar_canfd: Add Renesas R-Car CAN FD driver
can: rcar_can: Move Renesas CAN driver to rcar dir
.../devicetree/bindings/net/can/rcar_canfd.txt | 89 ++
drivers/net/can/Kconfig | 11 +-
drivers/net/can/Makefile | 2 +-
drivers/net/can/rcar/Kconfig | 21 +
drivers/net/can/rcar/Makefile | 6 +
drivers/net/can/{ => rcar}/rcar_can.c | 0
drivers/net/can/rcar/rcar_canfd.c | 1624 ++++++++++++++++++++
7 files changed, 1742 insertions(+), 11 deletions(-)
create mode 100644 Documentation/devicetree/bindings/net/can/rcar_canfd.txt
create mode 100644 drivers/net/can/rcar/Kconfig
create mode 100644 drivers/net/can/rcar/Makefile
rename drivers/net/can/{ => rcar}/rcar_can.c (100%)
create mode 100644 drivers/net/can/rcar/rcar_canfd.c
--
1.9.1
^ permalink raw reply
* [net-next PATCH V4 5/5] samples/bpf: like LLC also verify and allow redefining CLANG command
From: Jesper Dangaard Brouer @ 2016-04-28 12:21 UTC (permalink / raw)
To: netdev
Cc: linux-kbuild, bblanco, Jesper Dangaard Brouer, naveen.n.rao,
borkmann, alexei.starovoitov
In-Reply-To: <20160428121949.16807.84828.stgit@firesoul>
Users are likely to manually compile both LLVM 'llc' and 'clang'
tools. Thus, also allow redefining CLANG and verify command exist.
Makefile implementation wise, the target that verify the command have
been generalized.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Acked-by: Alexei Starovoitov <ast@kernel.org>
---
samples/bpf/Makefile | 25 ++++++++++++++-----------
samples/bpf/README.rst | 6 +++---
2 files changed, 17 insertions(+), 14 deletions(-)
diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index dd63521832d8..66897e61232c 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile
@@ -81,9 +81,10 @@ HOSTLOADLIBES_spintest += -lelf
HOSTLOADLIBES_map_perf_test += -lelf -lrt
HOSTLOADLIBES_test_overhead += -lelf -lrt
-# Allows pointing LLC to a LLVM backend with bpf support, redefine on cmdline:
-# make samples/bpf/ LLC=~/git/llvm/build/bin/llc
+# Allows pointing LLC/CLANG to a LLVM backend with bpf support, redefine on cmdline:
+# make samples/bpf/ LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
LLC ?= llc
+CLANG ?= clang
# Trick to allow make to be run from this directory
all:
@@ -93,16 +94,18 @@ clean:
$(MAKE) -C ../../ M=$$PWD clean
@rm -f *~
-# Verify LLVM compiler is available and bpf target is supported
-.PHONY: verify_cmd_llc verify_target_bpf
+# Verify LLVM compiler tools are available and bpf target is supported by llc
+.PHONY: verify_cmds verify_target_bpf $(CLANG) $(LLC)
-verify_cmd_llc:
- @if ! (which "${LLC}" > /dev/null 2>&1); then \
- echo "*** ERROR: Cannot find LLVM tool 'llc' (${LLC})" ;\
- exit 1; \
- else true; fi
+verify_cmds: $(CLANG) $(LLC)
+ @for TOOL in $^ ; do \
+ if ! (which -- "$${TOOL}" > /dev/null 2>&1); then \
+ echo "*** ERROR: Cannot find LLVM tool $${TOOL}" ;\
+ exit 1; \
+ else true; fi; \
+ done
-verify_target_bpf: verify_cmd_llc
+verify_target_bpf: verify_cmds
@if ! (${LLC} -march=bpf -mattr=help > /dev/null 2>&1); then \
echo "*** ERROR: LLVM (${LLC}) does not support 'bpf' target" ;\
echo " NOTICE: LLVM version >= 3.7.1 required" ;\
@@ -115,6 +118,6 @@ $(src)/*.c: verify_target_bpf
# But, there is no easy way to fix it, so just exclude it since it is
# useless for BPF samples.
$(obj)/%.o: $(src)/%.c
- clang $(NOSTDINC_FLAGS) $(LINUXINCLUDE) $(EXTRA_CFLAGS) \
+ $(CLANG) $(NOSTDINC_FLAGS) $(LINUXINCLUDE) $(EXTRA_CFLAGS) \
-D__KERNEL__ -D__ASM_SYSREG_H -Wno-unused-value -Wno-pointer-sign \
-O2 -emit-llvm -c $< -o -| $(LLC) -march=bpf -filetype=obj -o $@
diff --git a/samples/bpf/README.rst b/samples/bpf/README.rst
index 0da069ffa434..d60d50d94640 100644
--- a/samples/bpf/README.rst
+++ b/samples/bpf/README.rst
@@ -60,8 +60,8 @@ Quick sniplet for manually compiling LLVM and clang
$ cmake .. -DLLVM_TARGETS_TO_BUILD="BPF;X86"
$ make -j $(getconf _NPROCESSORS_ONLN)
-It is also possible to point make to the newly compiled 'llc' command
-via redefining LLC on the make command line::
+It is also possible to point make to the newly compiled 'llc' or
+'clang' command via redefining LLC or CLANG on the make command line::
- make samples/bpf/ LLC=~/git/llvm/build/bin/llc
+ make samples/bpf/ LLC=~/git/llvm/build/bin/llc CLANG=~/git/llvm/build/bin/clang
^ permalink raw reply related
* [net-next PATCH V4 4/5] samples/bpf: allow make to be run from samples/bpf/ directory
From: Jesper Dangaard Brouer @ 2016-04-28 12:21 UTC (permalink / raw)
To: netdev
Cc: linux-kbuild, bblanco, Jesper Dangaard Brouer, naveen.n.rao,
borkmann, alexei.starovoitov
In-Reply-To: <20160428121949.16807.84828.stgit@firesoul>
It is not intuitive that 'make' must be run from the top level
directory with argument "samples/bpf/" to compile these eBPF samples.
Introduce a kbuild make file trick that allow make to be run from the
"samples/bpf/" directory itself. It basically change to the top level
directory and call "make samples/bpf/" with the "/" slash after the
directory name.
Also add a clean target that only cleans this directory, by taking
advantage of the kbuild external module setting M=$PWD.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Acked-by: Alexei Starovoitov <ast@kernel.org>
---
samples/bpf/Makefile | 8 ++++++++
samples/bpf/README.rst | 3 +++
2 files changed, 11 insertions(+)
diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 45859c99f573..dd63521832d8 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile
@@ -85,6 +85,14 @@ HOSTLOADLIBES_test_overhead += -lelf -lrt
# make samples/bpf/ LLC=~/git/llvm/build/bin/llc
LLC ?= llc
+# Trick to allow make to be run from this directory
+all:
+ $(MAKE) -C ../../ $$PWD/
+
+clean:
+ $(MAKE) -C ../../ M=$$PWD clean
+ @rm -f *~
+
# Verify LLVM compiler is available and bpf target is supported
.PHONY: verify_cmd_llc verify_target_bpf
diff --git a/samples/bpf/README.rst b/samples/bpf/README.rst
index 993d280184fa..0da069ffa434 100644
--- a/samples/bpf/README.rst
+++ b/samples/bpf/README.rst
@@ -36,6 +36,9 @@ top level directory::
Do notice the "/" slash after the directory name.
+It is also possible to call make from this directory. This will just
+hide the the invocation of make as above with the appended "/".
+
Manually compiling LLVM with 'bpf' support
------------------------------------------
^ permalink raw reply related
* [net-next PATCH V4 3/5] samples/bpf: add a README file to get users started
From: Jesper Dangaard Brouer @ 2016-04-28 12:21 UTC (permalink / raw)
To: netdev
Cc: linux-kbuild, bblanco, Jesper Dangaard Brouer, naveen.n.rao,
borkmann, alexei.starovoitov
In-Reply-To: <20160428121949.16807.84828.stgit@firesoul>
Getting started with using examples in samples/bpf/ is not
straightforward. There are several dependencies, and specific
versions of these dependencies.
Just compiling the example tool is also slightly obscure, e.g. one
need to call make like:
make samples/bpf/
Do notice the "/" slash after the directory name.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
---
samples/bpf/README.rst | 64 ++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 64 insertions(+)
create mode 100644 samples/bpf/README.rst
diff --git a/samples/bpf/README.rst b/samples/bpf/README.rst
new file mode 100644
index 000000000000..993d280184fa
--- /dev/null
+++ b/samples/bpf/README.rst
@@ -0,0 +1,64 @@
+eBPF sample programs
+====================
+
+This directory contains a mini eBPF library, test stubs, verifier
+test-suite and examples for using eBPF.
+
+Build dependencies
+==================
+
+Compiling requires having installed:
+ * clang >= version 3.4.0
+ * llvm >= version 3.7.1
+
+Note that LLVM's tool 'llc' must support target 'bpf', list version
+and supported targets with command: ``llc --version``
+
+Kernel headers
+--------------
+
+There are usually dependencies to header files of the current kernel.
+To avoid installing devel kernel headers system wide, as a normal
+user, simply call::
+
+ make headers_install
+
+This will creates a local "usr/include" directory in the git/build top
+level directory, that the make system automatically pickup first.
+
+Compiling
+=========
+
+For building the BPF samples, issue the below command from the kernel
+top level directory::
+
+ make samples/bpf/
+
+Do notice the "/" slash after the directory name.
+
+Manually compiling LLVM with 'bpf' support
+------------------------------------------
+
+Since version 3.7.0, LLVM adds a proper LLVM backend target for the
+BPF bytecode architecture.
+
+By default llvm will build all non-experimental backends including bpf.
+To generate a smaller llc binary one can use::
+
+ -DLLVM_TARGETS_TO_BUILD="BPF"
+
+Quick sniplet for manually compiling LLVM and clang
+(build dependencies are cmake and gcc-c++)::
+
+ $ git clone http://llvm.org/git/llvm.git
+ $ cd llvm/tools
+ $ git clone --depth 1 http://llvm.org/git/clang.git
+ $ cd ..; mkdir build; cd build
+ $ cmake .. -DLLVM_TARGETS_TO_BUILD="BPF;X86"
+ $ make -j $(getconf _NPROCESSORS_ONLN)
+
+It is also possible to point make to the newly compiled 'llc' command
+via redefining LLC on the make command line::
+
+ make samples/bpf/ LLC=~/git/llvm/build/bin/llc
+
^ permalink raw reply related
* [net-next PATCH V4 2/5] samples/bpf: Makefile verify LLVM compiler avail and bpf target is supported
From: Jesper Dangaard Brouer @ 2016-04-28 12:20 UTC (permalink / raw)
To: netdev
Cc: linux-kbuild, bblanco, Jesper Dangaard Brouer, naveen.n.rao,
borkmann, alexei.starovoitov
In-Reply-To: <20160428121949.16807.84828.stgit@firesoul>
Make compiling samples/bpf more user friendly, by detecting if LLVM
compiler tool 'llc' is available, and also detect if the 'bpf' target
is available in this version of LLVM.
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Acked-by: Alexei Starovoitov <ast@kernel.org>
---
samples/bpf/Makefile | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 5bae9536f100..45859c99f573 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile
@@ -85,6 +85,24 @@ HOSTLOADLIBES_test_overhead += -lelf -lrt
# make samples/bpf/ LLC=~/git/llvm/build/bin/llc
LLC ?= llc
+# Verify LLVM compiler is available and bpf target is supported
+.PHONY: verify_cmd_llc verify_target_bpf
+
+verify_cmd_llc:
+ @if ! (which "${LLC}" > /dev/null 2>&1); then \
+ echo "*** ERROR: Cannot find LLVM tool 'llc' (${LLC})" ;\
+ exit 1; \
+ else true; fi
+
+verify_target_bpf: verify_cmd_llc
+ @if ! (${LLC} -march=bpf -mattr=help > /dev/null 2>&1); then \
+ echo "*** ERROR: LLVM (${LLC}) does not support 'bpf' target" ;\
+ echo " NOTICE: LLVM version >= 3.7.1 required" ;\
+ exit 2; \
+ else true; fi
+
+$(src)/*.c: verify_target_bpf
+
# asm/sysreg.h - inline assembly used by it is incompatible with llvm.
# But, there is no easy way to fix it, so just exclude it since it is
# useless for BPF samples.
^ permalink raw reply related
* [net-next PATCH V4 1/5] samples/bpf: add back functionality to redefine LLC command
From: Jesper Dangaard Brouer @ 2016-04-28 12:20 UTC (permalink / raw)
To: netdev
Cc: linux-kbuild, bblanco, Jesper Dangaard Brouer, naveen.n.rao,
borkmann, alexei.starovoitov
In-Reply-To: <20160428121949.16807.84828.stgit@firesoul>
It is practical to be-able-to redefine the location of the LLVM
command 'llc', because not all distros have a LLVM version with bpf
target support. Thus, it is sometimes required to compile LLVM from
source, and sometimes it is not desired to overwrite the distros
default LLVM version.
This feature was removed with 128d1514be35 ("samples/bpf: Use llc in
PATH, rather than a hardcoded value").
Add this features back. Note that it is possible to redefine the LLC
on the make command like:
make samples/bpf/ LLC=~/git/llvm/build/bin/llc
Fixes: 128d1514be35 ("samples/bpf: Use llc in PATH, rather than a hardcoded value")
Signed-off-by: Jesper Dangaard Brouer <brouer@redhat.com>
Acked-by: Alexei Starovoitov <ast@kernel.org>
---
samples/bpf/Makefile | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/samples/bpf/Makefile b/samples/bpf/Makefile
index 744dd7a16144..5bae9536f100 100644
--- a/samples/bpf/Makefile
+++ b/samples/bpf/Makefile
@@ -81,10 +81,14 @@ HOSTLOADLIBES_spintest += -lelf
HOSTLOADLIBES_map_perf_test += -lelf -lrt
HOSTLOADLIBES_test_overhead += -lelf -lrt
+# Allows pointing LLC to a LLVM backend with bpf support, redefine on cmdline:
+# make samples/bpf/ LLC=~/git/llvm/build/bin/llc
+LLC ?= llc
+
# asm/sysreg.h - inline assembly used by it is incompatible with llvm.
# But, there is no easy way to fix it, so just exclude it since it is
# useless for BPF samples.
$(obj)/%.o: $(src)/%.c
clang $(NOSTDINC_FLAGS) $(LINUXINCLUDE) $(EXTRA_CFLAGS) \
-D__KERNEL__ -D__ASM_SYSREG_H -Wno-unused-value -Wno-pointer-sign \
- -O2 -emit-llvm -c $< -o -| llc -march=bpf -filetype=obj -o $@
+ -O2 -emit-llvm -c $< -o -| $(LLC) -march=bpf -filetype=obj -o $@
^ permalink raw reply related
* [net-next PATCH V4 0/5] samples/bpf: Improve user experience
From: Jesper Dangaard Brouer @ 2016-04-28 12:20 UTC (permalink / raw)
To: netdev
Cc: linux-kbuild, bblanco, Jesper Dangaard Brouer, naveen.n.rao,
borkmann, alexei.starovoitov
It is a steep learning curve getting started with using the eBPF
examples in samples/bpf/. There are several dependencies, and
specific versions of these dependencies. Invoking make in the correct
manor is also slightly obscure.
This patchset cleanup, document and hopefully improves the first time
user experience with the eBPF samples directory by auto-detecting
certain scenarios.
V4:
- Address Naveen's nitpicks
- Handle/fail if extra args are passed in LLC or CLANG (David Laight)
V3:
- Add Alexei's ACKs
- Remove README paragraph about LLVM experimental BPF target
as it only existed between LLVM version 3.6 to 3.7.
V2:
- Adjusted recommend minimum versions to 3.7.1
- Included clang build instructions
- New patch adding CLANG variable and validation of command
---
Jesper Dangaard Brouer (5):
samples/bpf: add back functionality to redefine LLC command
samples/bpf: Makefile verify LLVM compiler avail and bpf target is supported
samples/bpf: add a README file to get users started
samples/bpf: allow make to be run from samples/bpf/ directory
samples/bpf: like LLC also verify and allow redefining CLANG command
samples/bpf/Makefile | 37 +++++++++++++++++++++++++--
samples/bpf/README.rst | 67 ++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 102 insertions(+), 2 deletions(-)
create mode 100644 samples/bpf/README.rst
--
^ permalink raw reply
* Re: [PATCH net-next 00/17] net: snmp: update SNMP methods
From: Nicolas Dichtel @ 2016-04-28 12:20 UTC (permalink / raw)
To: Eric Dumazet; +Cc: Eric Dumazet, David S . Miller, netdev
In-Reply-To: <1461844854.5535.93.camel@edumazet-glaptop3.roam.corp.google.com>
Le 28/04/2016 14:00, Eric Dumazet a écrit :
[snip]
> Hi Nicolas, thanks for testing.
>
> Oh right, I shouldn't have changed the BH disabling of 64bit stats on
> 32bit arches, of course.
>
> Can you double check this will fix the problem ? Thanks !
Thank you for the quick fix!
Tested-by: Nicolas Dichtel <nicolas.dichtel@6wind.com>
^ permalink raw reply
* Re: iproute2: bash completion function for tc
From: Quentin Monnet @ 2016-04-28 12:15 UTC (permalink / raw)
To: Stephen Hemminger, Alexei Starovoitov
Cc: Jamal Hadi Salim, Vincent Jardin, netdev
In-Reply-To: <20160427221346.32ec7bcb@samsung9>
Hi Alexei, Stephen,
2016-04-27 (22:13 UTC-0700) ~ Stephen Hemminger:
> On Wed, 27 Apr 2016 20:19:26 -0700
> Alexei Starovoitov <alexei.starovoitov@gmail.com> wrote:
>
>> On Tue, Apr 26, 2016 at 09:28:17AM +0200, Quentin Monnet wrote:
>>> Hi Jamal, Stephen,
>>>
>>> I searched for a function providing auto-completion for `tc` utility in
>>> bash, but I found none. So I have created one, and I would like share it
>>> with the community. It is available here:
>>> https://github.com/6WIND/tc_bash-completion/blob/master/tc
>>> I would like to make it easily available to tc users, so here is a
>>> twofold request:
>>>
>>> * I do not know where to submit the code. Should I submit here on netdev
>>> for inclusion in iproute2 package, or rather to the bash-completion
>>> repository on GitHub? I feel like it would receive better feedback and
>>> updates if pushed to iproute2. Could you please provide some advice here?
>>> * The completion for `tc` seems to work well; I have tested it with many
>>> commands, but I am no tc expert, and there are probably some cases where
>>> the completion fails to propose the correct choices. I would be really
>>> interested in any feedback/bug reports that you, or anyone on this list
>>> who uses tc, could provide.
>>
>> that looks very interesting.
>> I think making it a part of iproute2 is a good thing.
>> How about installing it into /etc/iproute2/ ?
>> Stephen, any comments?
>>
>
> I am ok with keeping it in the repository.
> But it would need to be installed in the standard bash directory,
> is that distro dependent?
>
As far as I know the bash-completion directory is not distro dependent,
but it moved at some point (2011) from /etc/bash_completion.d/ to
/usr/share/bash-completion/completions/. While backward compatibility is
provided with the former location, it is now recommended to use the latter.
So one idea could be to check (in iproute2 Makefile) for existence of
/usr/share/bash-completion/completions/ directory, and if not found to
fall back to /etc/bash_completion.d/. If none is found, use
/usr/share/bash-completion/completions/. Does this seem correct?
^ permalink raw reply
* Re: [PATCH net] ip_tunnel: fix preempt warning in ip tunnel creation/updating
From: Eric Dumazet @ 2016-04-28 12:09 UTC (permalink / raw)
To: Paolo Abeni
Cc: netdev, David S. Miller, Alexey Kuznetsov, tgraf, pshelar, jbenc,
hannes, sergei.shtylyov, Tom Herbert
In-Reply-To: <b21056e2495c485a2084d058d03e1f95480b3510.1461834203.git.pabeni@redhat.com>
On Thu, 2016-04-28 at 11:04 +0200, Paolo Abeni wrote:
> After the commit e09acddf873b ("ip_tunnel: replace dst_cache with generic
> implementation"), a preemption debug warning is triggered on ip4
> tunnels updating; the dst cache helper needs to be invoked in unpreemptible
> context.
>
> We don't need to load the cache on tunnel update, so this commit fixes
> the warning replacing the load with a dst cache reset, which is
> preempt safe.
>
> Fixes: e09acddf873b ("ip_tunnel: replace dst_cache with generic
> implementation")
>
> Reported-by: Eric Dumazet <eric.dumazet@gmail.com>
> Signed-off-by: Paolo Abeni <pabeni@redhat.com>
> ---
Seems good to me, thanks.
Note that the Fixes: tag should not be followed by a blank line.
(And presumably no line wrap is necessary)
It really is part of the standard set of tags.
They should be grouped together.
Acked-by: Eric Dumazet <edumazet@google.com>
^ permalink raw reply
* [PATCH v6 20/21] IB/hns: Kconfig and Makefile for RoCE module
From: Lijun Ou @ 2016-04-28 12:09 UTC (permalink / raw)
To: dledford, sean.hefty, hal.rosenstock, davem, jeffrey.t.kirsher,
jiri, ogerlitz
Cc: linux-rdma, linux-kernel, netdev, gongyangming, xiaokun,
tangchaofei, oulijun, haifeng.wei, yisen.zhuang, yankejian,
charles.chenxin, linuxarm
In-Reply-To: <1461845396-61306-1-git-send-email-oulijun@huawei.com>
This patch added Kconfig and Makefile for building RoCE module.
Signed-off-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Nenglong Zhao <zhaonenglong@hisilicon.com>
Signed-off-by: Lijun Ou <oulijun@huawei.com>
---
drivers/infiniband/Kconfig | 1 +
drivers/infiniband/hw/Makefile | 1 +
drivers/infiniband/hw/hns/Kconfig | 10 ++++++++++
drivers/infiniband/hw/hns/Makefile | 9 +++++++++
4 files changed, 21 insertions(+)
create mode 100644 drivers/infiniband/hw/hns/Kconfig
create mode 100644 drivers/infiniband/hw/hns/Makefile
diff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig
index 6425c0e..726a4ca 100644
--- a/drivers/infiniband/Kconfig
+++ b/drivers/infiniband/Kconfig
@@ -74,6 +74,7 @@ source "drivers/infiniband/hw/mlx5/Kconfig"
source "drivers/infiniband/hw/nes/Kconfig"
source "drivers/infiniband/hw/ocrdma/Kconfig"
source "drivers/infiniband/hw/usnic/Kconfig"
+source "drivers/infiniband/hw/hns/Kconfig"
source "drivers/infiniband/ulp/ipoib/Kconfig"
diff --git a/drivers/infiniband/hw/Makefile b/drivers/infiniband/hw/Makefile
index c7ad0a4..223eb78 100644
--- a/drivers/infiniband/hw/Makefile
+++ b/drivers/infiniband/hw/Makefile
@@ -8,3 +8,4 @@ obj-$(CONFIG_MLX5_INFINIBAND) += mlx5/
obj-$(CONFIG_INFINIBAND_NES) += nes/
obj-$(CONFIG_INFINIBAND_OCRDMA) += ocrdma/
obj-$(CONFIG_INFINIBAND_USNIC) += usnic/
+obj-$(CONFIG_INFINIBAND_HISILICON_HNS) += hns/
diff --git a/drivers/infiniband/hw/hns/Kconfig b/drivers/infiniband/hw/hns/Kconfig
new file mode 100644
index 0000000..c47c168
--- /dev/null
+++ b/drivers/infiniband/hw/hns/Kconfig
@@ -0,0 +1,10 @@
+config INFINIBAND_HISILICON_HNS
+ tristate "Hisilicon Hns ROCE Driver"
+ depends on NET_VENDOR_HISILICON
+ depends on ARM64 && HNS && HNS_DSAF && HNS_ENET
+ ---help---
+ This is a ROCE/RDMA driver for the Hisilicon RoCE engine. The engine
+ is used in Hisilicon Hi1610 and more further ICT SoC.
+
+ To compile this driver as a module, choose M here: the module
+ will be called hns-roce.
diff --git a/drivers/infiniband/hw/hns/Makefile b/drivers/infiniband/hw/hns/Makefile
new file mode 100644
index 0000000..404a700
--- /dev/null
+++ b/drivers/infiniband/hw/hns/Makefile
@@ -0,0 +1,9 @@
+#
+# Makefile for the HISILICON RoCE drivers.
+#
+
+obj-$(CONFIG_INFINIBAND_HISILICON_HNS) += hns-roce.o
+hns-roce-objs := hns_roce_main.o hns_roce_cmd.o hns_roce_eq.o hns_roce_pd.o \
+ hns_roce_ah.o hns_roce_icm.o hns_roce_mr.o hns_roce_qp.o \
+ hns_roce_cq.o hns_roce_alloc.o hns_roce_hw_v1.o
+
--
1.9.1
^ permalink raw reply related
* [PATCH v6 18/21] IB/hns: Add CQ operations support
From: Lijun Ou @ 2016-04-28 12:09 UTC (permalink / raw)
To: dledford, sean.hefty, hal.rosenstock, davem, jeffrey.t.kirsher,
jiri, ogerlitz
Cc: linux-rdma, linux-kernel, netdev, gongyangming, xiaokun,
tangchaofei, oulijun, haifeng.wei, yisen.zhuang, yankejian,
charles.chenxin, linuxarm
In-Reply-To: <1461845396-61306-1-git-send-email-oulijun@huawei.com>
This patch was implementing for Completion Queue(CQ) operations.
A CQ can be used to multiplex work completions from multiple work
queues across queue pairs on the same HCA. CQ as the notification
mechanism for Work Request completions.
CQ operations as follows:
1. create CQ. CQ are created through the Channel Interface,
The maximum number of Completion Queue Entries (CQEs) that
may be outstanding on a CQ must be specified when the CQ
is created.
2. destroy CQ. Destroys the specified CQ. Resources allocated
by the Channel Interface to implement the CQ must be
deallocated during the destroy operation.
3. request completion notification. Requests the CQ event handler
be called when the next completion entry of the specified type
is added to the specified CQ.
4. poll CQ. Polls the specified CQ for a Work Completion.
A Work Completion indicates that a Work Request for a Work
Queue associated with the CQ is done.
Signed-off-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Nenglong Zhao <zhaonenglong@hisilicon.com>
Signed-off-by: Lijun Ou <oulijun@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_cq.c | 358 ++++++++++++++++++++++++++++
drivers/infiniband/hw/hns/hns_roce_device.h | 39 ++-
drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 348 ++++++++++++++++++++++++++-
drivers/infiniband/hw/hns/hns_roce_hw_v1.h | 117 +++++++++
drivers/infiniband/hw/hns/hns_roce_main.c | 9 +
drivers/infiniband/hw/hns/hns_roce_user.h | 4 +
6 files changed, 868 insertions(+), 7 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_cq.c b/drivers/infiniband/hw/hns/hns_roce_cq.c
index f7baf82..3ea1fc3 100644
--- a/drivers/infiniband/hw/hns/hns_roce_cq.c
+++ b/drivers/infiniband/hw/hns/hns_roce_cq.c
@@ -11,6 +11,364 @@
#include <linux/log2.h>
#include <linux/slab.h>
#include "hns_roce_device.h"
+#include "hns_roce_cmd.h"
+#include "hns_roce_icm.h"
+#include "hns_roce_user.h"
+#include "hns_roce_common.h"
+
+static void hns_roce_ib_cq_comp(struct hns_roce_cq *hr_cq)
+{
+ struct ib_cq *ibcq = &hr_cq->ib_cq;
+
+ ibcq->comp_handler(ibcq, ibcq->cq_context);
+}
+
+static void hns_roce_ib_cq_event(struct hns_roce_cq *hr_cq,
+ enum hns_roce_event event_type)
+{
+ struct hns_roce_dev *hr_dev;
+ struct ib_event event;
+ struct ib_cq *ibcq;
+
+ ibcq = &hr_cq->ib_cq;
+ hr_dev = to_hr_dev(ibcq->device);
+
+ if (event_type != HNS_ROCE_EVENT_TYPE_CQ_ID_INVALID &&
+ event_type != HNS_ROCE_EVENT_TYPE_CQ_ACCESS_ERROR &&
+ event_type != HNS_ROCE_EVENT_TYPE_CQ_OVERFLOW) {
+ dev_err(&hr_dev->pdev->dev,
+ "hns_roce_ib: Unexpected event type 0x%x on CQ %06lx\n",
+ event_type, hr_cq->cqn);
+ return;
+ }
+
+ if (ibcq->event_handler) {
+ event.device = ibcq->device;
+ event.event = IB_EVENT_CQ_ERR;
+ event.element.cq = ibcq;
+ ibcq->event_handler(&event, ibcq->cq_context);
+ }
+}
+
+static int hns_roce_sw2hw_cq(struct hns_roce_dev *dev,
+ struct hns_roce_cmd_mailbox *mailbox,
+ unsigned long cq_num)
+{
+ return hns_roce_cmd(dev, mailbox->dma, cq_num, 0,
+ HNS_ROCE_CMD_SW2HW_CQ, HNS_ROCE_CMD_TIME_CLASS_A);
+}
+
+static int hns_roce_cq_alloc(struct hns_roce_dev *hr_dev, int nent,
+ struct hns_roce_mtt *hr_mtt,
+ struct hns_roce_uar *hr_uar,
+ struct hns_roce_cq *hr_cq, int vector,
+ int collapsed)
+{
+ struct hns_roce_cmd_mailbox *mailbox = NULL;
+ struct hns_roce_cq_table *cq_table = NULL;
+ struct device *dev = &hr_dev->pdev->dev;
+ dma_addr_t dma_handle;
+ u64 *mtts = NULL;
+ int ret = 0;
+
+ cq_table = &hr_dev->cq_table;
+
+ /* Get the physical address of cq buf */
+ mtts = hns_roce_table_find(&hr_dev->mr_table.mtt_table,
+ hr_mtt->first_seg, &dma_handle);
+ if (!mtts) {
+ dev_err(dev, "CQ alloc.Failed to find cq buf addr.\n");
+ return -EINVAL;
+ }
+
+ if (vector >= hr_dev->caps.num_comp_vectors) {
+ dev_err(dev, "CQ alloc.Invalid vector.\n");
+ return -EINVAL;
+ }
+ hr_cq->vector = vector;
+
+ ret = hns_roce_bitmap_alloc(&cq_table->bitmap, &hr_cq->cqn);
+ if (ret == -1) {
+ dev_err(dev, "CQ alloc.Failed to alloc index.\n");
+ return -ENOMEM;
+ }
+
+ /* Get CQC memory icm table */
+ ret = hns_roce_table_get(hr_dev, &cq_table->table, hr_cq->cqn);
+ if (ret) {
+ dev_err(dev, "CQ alloc.Failed to get context mem.\n");
+ goto err_out;
+ }
+
+ /* The cq insert radix tree */
+ spin_lock_irq(&cq_table->lock);
+ /* Radix_tree: The associated pointer and long integer key value like */
+ ret = radix_tree_insert(&cq_table->tree, hr_cq->cqn, hr_cq);
+ spin_unlock_irq(&cq_table->lock);
+ if (ret) {
+ dev_err(dev, "CQ alloc.Failed to radix_tree_insert.\n");
+ goto err_put;
+ }
+
+ /* Applicate mailbox memory */
+ mailbox = hns_roce_alloc_cmd_mailbox(hr_dev);
+ if (IS_ERR(mailbox)) {
+ dev_err(dev, "CQ alloc.Failed to alloc mailbox.\n");
+ ret = PTR_ERR(mailbox);
+ goto err_radix;
+ }
+
+ hr_dev->hw->write_cqc(hr_dev, hr_cq, mailbox->buf, mtts, dma_handle,
+ nent, vector);
+
+ /* CQ instructions which sw send to hw be transimited via mailbox */
+ ret = hns_roce_sw2hw_cq(hr_dev, mailbox, hr_cq->cqn);
+ hns_roce_free_cmd_mailbox(hr_dev, mailbox);
+ if (ret) {
+ dev_err(dev, "CQ alloc.Failed to cmd mailbox.\n");
+ goto err_radix;
+ }
+
+ hr_cq->cons_index = 0;
+ hr_cq->uar = hr_uar;
+
+ return 0;
+
+err_radix:
+ spin_lock_irq(&cq_table->lock);
+ radix_tree_delete(&cq_table->tree, hr_cq->cqn);
+ spin_unlock_irq(&cq_table->lock);
+
+err_put:
+ hns_roce_table_put(hr_dev, &cq_table->table, hr_cq->cqn);
+
+err_out:
+ hns_roce_bitmap_free(&cq_table->bitmap, hr_cq->cqn);
+ return ret;
+}
+
+static int hns_roce_hw2sw_cq(struct hns_roce_dev *dev,
+ struct hns_roce_cmd_mailbox *mailbox,
+ unsigned long cq_num)
+{
+ return hns_roce_cmd_box(dev, 0, mailbox ? mailbox->dma : 0, cq_num,
+ mailbox ? 0 : 1, HNS_ROCE_CMD_HW2SW_CQ,
+ HNS_ROCE_CMD_TIME_CLASS_A);
+}
+
+void hns_roce_free_cq(struct hns_roce_dev *hr_dev, struct hns_roce_cq *hr_cq)
+{
+ struct hns_roce_cq_table *cq_table = &hr_dev->cq_table;
+ struct device *dev = &hr_dev->pdev->dev;
+ int ret;
+
+ ret = hns_roce_hw2sw_cq(hr_dev, NULL, hr_cq->cqn);
+ if (ret)
+ dev_err(dev, "HW2SW_CQ failed (%d) for CQN %06lx\n", ret,
+ hr_cq->cqn);
+
+ /* Waiting interrupt process procedure carried out */
+ synchronize_irq(hr_dev->eq_table.eq[hr_cq->vector].irq);
+
+ spin_lock_irq(&cq_table->lock);
+ radix_tree_delete(&cq_table->tree, hr_cq->cqn);
+ spin_unlock_irq(&cq_table->lock);
+
+ hns_roce_table_put(hr_dev, &cq_table->table, hr_cq->cqn);
+ hns_roce_bitmap_free(&cq_table->bitmap, hr_cq->cqn);
+}
+
+static int hns_roce_ib_get_cq_umem(struct hns_roce_dev *hr_dev,
+ struct ib_ucontext *context,
+ struct hns_roce_cq_buf *buf,
+ struct ib_umem **umem, u64 buf_addr, int cqe)
+{
+ int ret;
+
+ /* Get and mapping user space */
+ *umem = ib_umem_get(context, buf_addr, cqe * hr_dev->caps.cq_entry_sz,
+ IB_ACCESS_LOCAL_WRITE, 1);
+ if (IS_ERR(*umem))
+ return PTR_ERR(*umem);
+
+ ret = hns_roce_mtt_init(hr_dev, ib_umem_page_count(*umem),
+ ilog2((unsigned int)(*umem)->page_size),
+ &buf->hr_mtt);
+ if (ret)
+ goto err_buf;
+
+ ret = hns_roce_ib_umem_write_mtt(hr_dev, &buf->hr_mtt, *umem);
+ if (ret)
+ goto err_mtt;
+
+ return 0;
+
+err_mtt:
+ hns_roce_mtt_cleanup(hr_dev, &buf->hr_mtt);
+
+err_buf:
+ ib_umem_release(*umem);
+ return ret;
+}
+
+static int hns_roce_ib_alloc_cq_buf(struct hns_roce_dev *hr_dev,
+ struct hns_roce_cq_buf *buf, u32 nent)
+{
+ int ret;
+
+ ret = hns_roce_buf_alloc(hr_dev, nent * hr_dev->caps.cq_entry_sz,
+ PAGE_SIZE * 2, &buf->hr_buf);
+ if (ret)
+ goto out;
+
+ ret = hns_roce_mtt_init(hr_dev, buf->hr_buf.npages,
+ buf->hr_buf.page_shift, &buf->hr_mtt);
+ if (ret)
+ goto err_buf;
+
+ ret = hns_roce_buf_write_mtt(hr_dev, &buf->hr_mtt, &buf->hr_buf);
+ if (ret)
+ goto err_mtt;
+
+ return 0;
+
+err_mtt:
+ hns_roce_mtt_cleanup(hr_dev, &buf->hr_mtt);
+
+err_buf:
+ hns_roce_buf_free(hr_dev, nent * hr_dev->caps.cq_entry_sz,
+ &buf->hr_buf);
+out:
+ return ret;
+}
+
+static void hns_roce_ib_free_cq_buf(struct hns_roce_dev *hr_dev,
+ struct hns_roce_cq_buf *buf, int cqe)
+{
+ hns_roce_buf_free(hr_dev, (cqe + 1) * hr_dev->caps.cq_entry_sz,
+ &buf->hr_buf);
+}
+
+struct ib_cq *hns_roce_ib_create_cq(struct ib_device *ib_dev,
+ const struct ib_cq_init_attr *attr,
+ struct ib_ucontext *context,
+ struct ib_udata *udata)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(ib_dev);
+ struct device *dev = &hr_dev->pdev->dev;
+ struct hns_roce_ib_create_cq ucmd;
+ struct hns_roce_cq *hr_cq = NULL;
+ struct hns_roce_uar *uar = NULL;
+ int vector = attr->comp_vector;
+ int cq_entries = attr->cqe;
+ int ret = 0;
+
+ if (cq_entries < 1 || cq_entries > hr_dev->caps.max_cqes) {
+ dev_err(dev, "Creat CQ failed. entries=%d, max=%d\n",
+ cq_entries, hr_dev->caps.max_cqes);
+ return ERR_PTR(-EINVAL);
+ }
+
+ hr_cq = kmalloc(sizeof(*hr_cq), GFP_KERNEL);
+ if (!hr_cq)
+ return ERR_PTR(-ENOMEM);
+
+ /* In v1 engine, parameter verification */
+ if (cq_entries < HNS_ROCE_MIN_CQE_NUM)
+ cq_entries = HNS_ROCE_MIN_CQE_NUM;
+
+ cq_entries = roundup_pow_of_two((unsigned int)cq_entries);
+ hr_cq->ib_cq.cqe = cq_entries - 1;
+ mutex_init(&hr_cq->resize_mutex);
+ spin_lock_init(&hr_cq->lock);
+ hr_cq->hr_resize_buf = NULL;
+ hr_cq->resize_umem = NULL;
+
+ if (context) {
+ if (ib_copy_from_udata(&ucmd, udata, sizeof(ucmd))) {
+ dev_err(dev, "Failed to copy_from_udata.\n");
+ ret = -EFAULT;
+ goto err_cq;
+ }
+
+ /* Get user space address, write it into mtt table */
+ ret = hns_roce_ib_get_cq_umem(hr_dev, context, &hr_cq->hr_buf,
+ &hr_cq->umem, ucmd.buf_addr,
+ cq_entries);
+ if (ret) {
+ dev_err(dev, "Failed to get_cq_umem.\n");
+ goto err_cq;
+ }
+
+ /* Get user space parameters */
+ uar = &to_hr_ucontext(context)->uar;
+ } else {
+ /* Init mmt table and write buff address to mtt table */
+ ret = hns_roce_ib_alloc_cq_buf(hr_dev, &hr_cq->hr_buf,
+ cq_entries);
+ if (ret) {
+ dev_err(dev, "Failed to alloc_cq_buf.\n");
+ goto err_cq;
+ }
+
+ uar = &hr_dev->priv_uar;
+ hr_cq->cq_db_l = hr_dev->reg_base + ROCEE_DB_OTHERS_L_0_REG +
+ 0x1000 * uar->index;
+ }
+
+ /* Allocate cq index, fill cq_context */
+ ret = hns_roce_cq_alloc(hr_dev, cq_entries, &hr_cq->hr_buf.hr_mtt,
+ uar, hr_cq, vector, 0);
+ if (ret) {
+ dev_err(dev, "Creat CQ .Failed to cq_alloc.\n");
+ goto err_mtt;
+ }
+
+ /* Get created cq handler and carry out event */
+ hr_cq->comp = hns_roce_ib_cq_comp;
+ hr_cq->event = hns_roce_ib_cq_event;
+ hr_cq->cq_depth = cq_entries;
+
+ if (context) {
+ if (ib_copy_to_udata(udata, &hr_cq->cqn, sizeof(u64))) {
+ ret = -EFAULT;
+ goto err_mtt;
+ }
+ }
+
+ return &hr_cq->ib_cq;
+
+err_mtt:
+ hns_roce_mtt_cleanup(hr_dev, &hr_cq->hr_buf.hr_mtt);
+ if (context)
+ ib_umem_release(hr_cq->umem);
+ else
+ hns_roce_ib_free_cq_buf(hr_dev, &hr_cq->hr_buf,
+ hr_cq->ib_cq.cqe);
+
+err_cq:
+ kfree(hr_cq);
+ return ERR_PTR(ret);
+}
+
+int hns_roce_ib_destroy_cq(struct ib_cq *ib_cq)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(ib_cq->device);
+ struct hns_roce_cq *hr_cq = to_hr_cq(ib_cq);
+
+ hns_roce_free_cq(hr_dev, hr_cq);
+ hns_roce_mtt_cleanup(hr_dev, &hr_cq->hr_buf.hr_mtt);
+
+ if (ib_cq->uobject)
+ ib_umem_release(hr_cq->umem);
+ else
+ /* Free the buff of stored cq */
+ hns_roce_ib_free_cq_buf(hr_dev, &hr_cq->hr_buf, ib_cq->cqe);
+
+ kfree(hr_cq);
+
+ return 0;
+}
void hns_roce_cq_completion(struct hns_roce_dev *hr_dev, u32 cqn)
{
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index d6a0f9e..1beb041 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -31,6 +31,7 @@
#define HNS_ROCE_BA_SIZE (32 * 4096)
/* Hardware specification only for v1 engine */
+#define HNS_ROCE_MIN_CQE_NUM 0x40
#define HNS_ROCE_MIN_WQE_NUM 0x20
#define HNS_ROCE_MAX_IRQ_NUM 34
@@ -124,6 +125,12 @@ enum {
HNS_ROCE_CMD_SUCCESS = 1,
};
+enum {
+ /* RQ&SRQ related operations */
+ HNS_ROCE_OPCODE_SEND_DATA_RECEIVE = 0x06,
+ HNS_ROCE_OPCODE_RDMA_WITH_IMM_RECEIVE = 0x07,
+};
+
#define HNS_ROCE_PORT_DOWN 0
#define HNS_ROCE_PORT_UP 1
@@ -189,9 +196,9 @@ struct hns_roce_icm_table {
};
struct hns_roce_mtt {
- unsigned long first_seg;
- int order;
- int page_shift;
+ unsigned long first_seg;
+ int order;
+ int page_shift;
};
struct hns_roce_mr_table {
@@ -229,20 +236,33 @@ struct hns_roce_buf {
struct hns_roce_cq_buf {
struct hns_roce_buf hr_buf;
+ struct hns_roce_mtt hr_mtt;
+};
+
+struct hns_roce_cq_resize {
+ struct hns_roce_cq_buf hr_buf;
+ int cqe;
};
struct hns_roce_cq {
struct ib_cq ib_cq;
struct hns_roce_cq_buf hr_buf;
/* pointer to store information after resize*/
+ struct hns_roce_cq_resize *hr_resize_buf;
spinlock_t lock;
+ struct mutex resize_mutex;
+ struct ib_umem *umem;
+ struct ib_umem *resize_umem;
void (*comp)(struct hns_roce_cq *);
void (*event)(struct hns_roce_cq *, enum hns_roce_event);
+ struct hns_roce_uar *uar;
u32 cq_depth;
u32 cons_index;
void __iomem *cq_db_l;
+ void __iomem *tptr_addr;
unsigned long cqn;
+ u32 vector;
atomic_t refcount;
struct completion free;
};
@@ -451,6 +471,9 @@ struct hns_roce_hw {
void (*set_mac)(struct hns_roce_dev *hr_dev, u8 phy_port, u8 *addr);
void (*set_mtu)(struct hns_roce_dev *hr_dev, u8 phy_port,
enum ib_mtu mtu);
+ void (*write_cqc)(struct hns_roce_dev *hr_dev,
+ struct hns_roce_cq *hr_cq, void *mb_buf, u64 *mtts,
+ dma_addr_t dma_handle, int nent, u32 vector);
int (*query_qp)(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr,
int qp_attr_mask, struct ib_qp_init_attr *qp_init_attr);
int (*modify_qp)(struct ib_qp *ibqp, const struct ib_qp_attr *attr,
@@ -461,6 +484,8 @@ struct hns_roce_hw {
struct ib_send_wr **bad_wr);
int (*post_recv)(struct ib_qp *qp, struct ib_recv_wr *recv_wr,
struct ib_recv_wr **bad_recv_wr);
+ int (*req_notify_cq)(struct ib_cq *ibcq, enum ib_cq_notify_flags flags);
+ int (*poll_cq)(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc);
void *priv;
};
@@ -484,6 +509,7 @@ struct hns_roce_dev {
u32 vendor_id;
u32 vendor_part_id;
u32 hw_rev;
+ void __iomem *priv_addr;
struct hns_roce_cmdq cmd;
struct hns_roce_bitmap pd_bitmap;
@@ -644,6 +670,13 @@ void hns_roce_release_range_qp(struct hns_roce_dev *hr_dev, int base_qpn,
__be32 send_ieth(struct ib_send_wr *wr);
int to_hr_qp_type(int qp_type);
+struct ib_cq *hns_roce_ib_create_cq(struct ib_device *ib_dev,
+ const struct ib_cq_init_attr *attr,
+ struct ib_ucontext *context,
+ struct ib_udata *udata);
+
+int hns_roce_ib_destroy_cq(struct ib_cq *ib_cq);
+
void hns_roce_cq_completion(struct hns_roce_dev *hr_dev, u32 cqn);
void hns_roce_cq_event(struct hns_roce_dev *hr_dev, u32 cqn, int event_type);
void hns_roce_qp_event(struct hns_roce_dev *hr_dev, u32 qpn, int event_type);
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
index 3f8d6d1..b95e75e 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
@@ -1043,6 +1043,11 @@ static void *get_sw_cqe(struct hns_roce_cq *hr_cq, int n)
!!(n & (hr_cq->ib_cq.cqe + 1))) ? hr_cqe : NULL;
}
+static struct hns_roce_cqe *next_cqe_sw(struct hns_roce_cq *hr_cq)
+{
+ return get_sw_cqe(hr_cq, hr_cq->cons_index);
+}
+
void hns_roce_v1_cq_set_ci(struct hns_roce_cq *hr_cq, u32 cons_index,
spinlock_t *doorbell_lock)
@@ -1118,6 +1123,338 @@ static void hns_roce_v1_cq_clean(struct hns_roce_cq *hr_cq, u32 qpn,
spin_unlock_irq(&hr_cq->lock);
}
+void hns_roce_v1_write_cqc(struct hns_roce_dev *hr_dev,
+ struct hns_roce_cq *hr_cq, void *mb_buf, u64 *mtts,
+ dma_addr_t dma_handle, int nent, u32 vector)
+{
+ struct hns_roce_cq_context *cq_context = NULL;
+ void __iomem *tptr_addr;
+
+ cq_context = mb_buf;
+ memset(cq_context, 0, sizeof(*cq_context));
+
+ tptr_addr = 0;
+ hr_dev->priv_addr = tptr_addr;
+ hr_cq->tptr_addr = tptr_addr;
+
+ /* Register cq_context members */
+ roce_set_field(cq_context->cqc_byte_4,
+ CQ_CONTEXT_CQC_BYTE_4_CQC_STATE_M,
+ CQ_CONTEXT_CQC_BYTE_4_CQC_STATE_S, CQ_STATE_VALID);
+ roce_set_field(cq_context->cqc_byte_4, CQ_CONTEXT_CQC_BYTE_4_CQN_M,
+ CQ_CONTEXT_CQC_BYTE_4_CQN_S, hr_cq->cqn);
+ cq_context->cqc_byte_4 = cpu_to_le32(cq_context->cqc_byte_4);
+
+ cq_context->cq_bt_l = (u32)dma_handle;
+ cq_context->cq_bt_l = cpu_to_le32(cq_context->cq_bt_l);
+
+ roce_set_field(cq_context->cqc_byte_12,
+ CQ_CONTEXT_CQC_BYTE_12_CQ_BT_H_M,
+ CQ_CONTEXT_CQC_BYTE_12_CQ_BT_H_S,
+ ((u64)dma_handle >> ADDR_SHIFT_32));
+ roce_set_field(cq_context->cqc_byte_12,
+ CQ_CONTEXT_CQC_BYTE_12_CQ_CQE_SHIFT_M,
+ CQ_CONTEXT_CQC_BYTE_12_CQ_CQE_SHIFT_S,
+ ilog2((unsigned int)nent));
+ roce_set_field(cq_context->cqc_byte_12, CQ_CONTEXT_CQC_BYTE_12_CEQN_M,
+ CQ_CONTEXT_CQC_BYTE_12_CEQN_S, vector);
+ cq_context->cqc_byte_12 = cpu_to_le32(cq_context->cqc_byte_12);
+
+ cq_context->cur_cqe_ba0_l = (u32)(mtts[0]);
+ cq_context->cur_cqe_ba0_l = cpu_to_le32(cq_context->cur_cqe_ba0_l);
+
+ roce_set_field(cq_context->cqc_byte_20,
+ CQ_CONTEXT_CQC_BYTE_20_CUR_CQE_BA0_H_M,
+ CQ_CONTEXT_CQC_BYTE_20_CUR_CQE_BA0_H_S,
+ cpu_to_le32((mtts[0]) >> ADDR_SHIFT_32));
+ /* Dedicated hardware, directly set 0 */
+ roce_set_field(cq_context->cqc_byte_20,
+ CQ_CONTEXT_CQC_BYTE_20_CQ_CUR_INDEX_M,
+ CQ_CONTEXT_CQC_BYTE_20_CQ_CUR_INDEX_S, 0);
+ roce_set_field(cq_context->cqc_byte_20,
+ CQ_CONTEXT_CQC_BYTE_20_CQE_TPTR_ADDR_H_M,
+ CQ_CONTEXT_CQC_BYTE_20_CQE_TPTR_ADDR_H_S,
+ (u64)tptr_addr >> ADDR_SHIFT_44);
+ cq_context->cqc_byte_20 = cpu_to_le32(cq_context->cqc_byte_20);
+
+ cq_context->cqe_tptr_addr_l = (u32)((u64)tptr_addr >> ADDR_SHIFT_12);
+
+ roce_set_field(cq_context->cqc_byte_32,
+ CQ_CONTEXT_CQC_BYTE_32_CUR_CQE_BA1_H_M,
+ CQ_CONTEXT_CQC_BYTE_32_CUR_CQE_BA1_H_S, 0);
+ roce_set_bit(cq_context->cqc_byte_32,
+ CQ_CONTEXT_CQC_BYTE_32_SE_FLAG_S, 0);
+ roce_set_bit(cq_context->cqc_byte_32,
+ CQ_CONTEXT_CQC_BYTE_32_CE_FLAG_S, 0);
+ roce_set_bit(cq_context->cqc_byte_32,
+ CQ_CONTEXT_CQC_BYTE_32_NOTIFICATION_FLAG_S, 0);
+ roce_set_bit(cq_context->cqc_byte_32,
+ CQ_CQNTEXT_CQC_BYTE_32_TYPE_OF_COMPLETION_NOTIFICATION_S,
+ 0);
+ /*The initial value of cq's ci is 0 */
+ roce_set_field(cq_context->cqc_byte_32,
+ CQ_CONTEXT_CQC_BYTE_32_CQ_CONS_IDX_M,
+ CQ_CONTEXT_CQC_BYTE_32_CQ_CONS_IDX_S, 0);
+ cq_context->cqc_byte_32 = cpu_to_le32(cq_context->cqc_byte_32);
+}
+
+int hns_roce_v1_req_notify_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags)
+{
+ struct hns_roce_cq *hr_cq = to_hr_cq(ibcq);
+ u32 notification_flag;
+ u32 doorbell[2];
+ int ret = 0;
+
+ notification_flag = (flags & IB_CQ_SOLICITED_MASK) ==
+ IB_CQ_SOLICITED ? CQ_DB_REQ_NOT : CQ_DB_REQ_NOT_SOL;
+ /*
+ * flags = 0; Notification Flag = 1, next
+ * flags = 1; Notification Flag = 0, solocited
+ */
+ doorbell[0] = hr_cq->cons_index & ((hr_cq->cq_depth << 1) - 1);
+ roce_set_bit(doorbell[1], ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_HW_SYNS_S, 1);
+ roce_set_field(doorbell[1], ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_CMD_M,
+ ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_CMD_S, 3);
+ roce_set_field(doorbell[1], ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_CMD_MDF_M,
+ ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_CMD_MDF_S, 1);
+ roce_set_field(doorbell[1], ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_INP_H_M,
+ ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_INP_H_S,
+ hr_cq->cqn | notification_flag);
+
+ hns_roce_write64_k(doorbell, hr_cq->cq_db_l);
+
+ return ret;
+}
+
+static int hns_roce_v1_poll_one(struct hns_roce_cq *hr_cq,
+ struct hns_roce_qp **cur_qp, struct ib_wc *wc)
+{
+ int qpn;
+ int is_send;
+ u16 wqe_ctr;
+ u32 status;
+ u32 opcode;
+ struct hns_roce_cqe *cqe;
+ struct hns_roce_qp *hr_qp;
+ struct hns_roce_wq *wq;
+ struct hns_roce_wqe_ctrl_seg *sq_wqe;
+ struct hns_roce_dev *hr_dev = to_hr_dev(hr_cq->ib_cq.device);
+ struct device *dev = &hr_dev->pdev->dev;
+
+ /* Find cqe according consumer index */
+ cqe = next_cqe_sw(hr_cq);
+ if (!cqe)
+ return -EAGAIN;
+
+ ++hr_cq->cons_index;
+ /* Memory barrier */
+ rmb();
+ /* 0->SQ, 1->RQ */
+ is_send = !(roce_get_bit(cqe->cqe_byte_4, CQE_BYTE_4_SQ_RQ_FLAG_S));
+
+ /* Local_qpn in UD cqe is always 1, so it needs to compute new qpn */
+ if (roce_get_field(cqe->cqe_byte_16, CQE_BYTE_16_LOCAL_QPN_M,
+ CQE_BYTE_16_LOCAL_QPN_S) <= 1) {
+ qpn = roce_get_field(cqe->cqe_byte_20, CQE_BYTE_20_PORT_NUM_M,
+ CQE_BYTE_20_PORT_NUM_S) +
+ roce_get_field(cqe->cqe_byte_16, CQE_BYTE_16_LOCAL_QPN_M,
+ CQE_BYTE_16_LOCAL_QPN_S) *
+ HNS_ROCE_MAX_PORTS;
+ } else {
+ qpn = roce_get_field(cqe->cqe_byte_16, CQE_BYTE_16_LOCAL_QPN_M,
+ CQE_BYTE_16_LOCAL_QPN_S);
+ }
+
+ if (!*cur_qp || (qpn & HNS_ROCE_CQE_QPN_MASK) != (*cur_qp)->qpn) {
+ hr_qp = __hns_roce_qp_lookup(hr_dev, qpn);
+ if (unlikely(!hr_qp)) {
+ dev_err(dev, "CQ %06lx with entry for unknown QPN %06x\n",
+ hr_cq->cqn, (qpn & HNS_ROCE_CQE_QPN_MASK));
+ return -EINVAL;
+ }
+
+ *cur_qp = hr_qp;
+ }
+
+ wc->qp = &(*cur_qp)->ibqp;
+ wc->vendor_err = 0;
+
+ status = roce_get_field(cqe->cqe_byte_4,
+ CQE_BYTE_4_STATUS_OF_THE_OPERATION_M,
+ CQE_BYTE_4_STATUS_OF_THE_OPERATION_S) &
+ HNS_ROCE_CQE_STATUS_MASK;
+ switch (status) {
+ case HNS_ROCE_CQE_SUCCESS:
+ wc->status = IB_WC_SUCCESS;
+ break;
+ case HNS_ROCE_CQE_SYNDROME_LOCAL_LENGTH_ERR:
+ wc->status = IB_WC_LOC_LEN_ERR;
+ break;
+ case HNS_ROCE_CQE_SYNDROME_LOCAL_QP_OP_ERR:
+ wc->status = IB_WC_LOC_QP_OP_ERR;
+ break;
+ case HNS_ROCE_CQE_SYNDROME_LOCAL_PROT_ERR:
+ wc->status = IB_WC_LOC_PROT_ERR;
+ break;
+ case HNS_ROCE_CQE_SYNDROME_WR_FLUSH_ERR:
+ wc->status = IB_WC_WR_FLUSH_ERR;
+ break;
+ case HNS_ROCE_CQE_SYNDROME_MEM_MANAGE_OPERATE_ERR:
+ wc->status = IB_WC_MW_BIND_ERR;
+ break;
+ case HNS_ROCE_CQE_SYNDROME_BAD_RESP_ERR:
+ wc->status = IB_WC_BAD_RESP_ERR;
+ break;
+ case HNS_ROCE_CQE_SYNDROME_LOCAL_ACCESS_ERR:
+ wc->status = IB_WC_LOC_ACCESS_ERR;
+ break;
+ case HNS_ROCE_CQE_SYNDROME_REMOTE_INVAL_REQ_ERR:
+ wc->status = IB_WC_REM_INV_REQ_ERR;
+ break;
+ case HNS_ROCE_CQE_SYNDROME_REMOTE_ACCESS_ERR:
+ wc->status = IB_WC_REM_ACCESS_ERR;
+ break;
+ case HNS_ROCE_CQE_SYNDROME_REMOTE_OP_ERR:
+ wc->status = IB_WC_REM_OP_ERR;
+ break;
+ case HNS_ROCE_CQE_SYNDROME_TRANSPORT_RETRY_EXC_ERR:
+ wc->status = IB_WC_RETRY_EXC_ERR;
+ break;
+ case HNS_ROCE_CQE_SYNDROME_RNR_RETRY_EXC_ERR:
+ wc->status = IB_WC_RNR_RETRY_EXC_ERR;
+ break;
+ default:
+ wc->status = IB_WC_GENERAL_ERR;
+ break;
+ }
+
+ /* CQE status error, directly return */
+ if (wc->status != IB_WC_SUCCESS)
+ return 0;
+
+ if (is_send) {
+ /* SQ conrespond to CQE */
+ sq_wqe = get_send_wqe(*cur_qp, roce_get_field(cqe->cqe_byte_4,
+ CQE_BYTE_4_WQE_INDEX_M,
+ CQE_BYTE_4_WQE_INDEX_S));
+ switch (sq_wqe->flag & HNS_ROCE_WQE_OPCODE_MASK) {
+ case HNS_ROCE_WQE_OPCODE_SEND:
+ wc->opcode = IB_WC_SEND;
+ break;
+ case HNS_ROCE_WQE_OPCODE_RDMA_READ:
+ wc->opcode = IB_WC_RDMA_READ;
+ wc->byte_len = le32_to_cpu(cqe->byte_cnt);
+ break;
+ case HNS_ROCE_WQE_OPCODE_RDMA_WRITE:
+ wc->opcode = IB_WC_RDMA_WRITE;
+ break;
+ case HNS_ROCE_WQE_OPCODE_LOCAL_INV:
+ wc->opcode = IB_WC_LOCAL_INV;
+ break;
+ case HNS_ROCE_WQE_OPCODE_UD_SEND:
+ wc->opcode = IB_WC_SEND;
+ break;
+ default:
+ wc->status = IB_WC_GENERAL_ERR;
+ break;
+ }
+ wc->wc_flags = (sq_wqe->flag & HNS_ROCE_WQE_IMM ?
+ IB_WC_WITH_IMM : 0);
+
+ wq = &(*cur_qp)->sq;
+ if ((*cur_qp)->sq_signal_bits) {
+ /*
+ * If sg_signal_bit is 1,
+ * firstly tail pointer updated to wqe
+ * which current cqe correspond to
+ */
+ wqe_ctr = (u16)roce_get_field(cqe->cqe_byte_4,
+ CQE_BYTE_4_WQE_INDEX_M,
+ CQE_BYTE_4_WQE_INDEX_S);
+ wq->tail += (wqe_ctr - (u16)wq->tail) &
+ (wq->wqe_cnt - 1);
+ }
+ wc->wr_id = wq->wrid[wq->tail & (wq->wqe_cnt - 1)];
+ ++wq->tail;
+ } else {
+ /* RQ conrespond to CQE */
+ wc->byte_len = le32_to_cpu(cqe->byte_cnt);
+ opcode = roce_get_field(cqe->cqe_byte_4,
+ CQE_BYTE_4_OPERATION_TYPE_M,
+ CQE_BYTE_4_OPERATION_TYPE_S) &
+ HNS_ROCE_CQE_OPCODE_MASK;
+ switch (opcode) {
+ case HNS_ROCE_OPCODE_RDMA_WITH_IMM_RECEIVE:
+ wc->opcode = IB_WC_RECV_RDMA_WITH_IMM;
+ wc->wc_flags = IB_WC_WITH_IMM;
+ wc->ex.imm_data = le32_to_cpu(cqe->immediate_data);
+ break;
+ case HNS_ROCE_OPCODE_SEND_DATA_RECEIVE:
+ if (roce_get_bit(cqe->cqe_byte_4,
+ CQE_BYTE_4_IMM_INDICATOR_S)) {
+ wc->opcode = IB_WC_RECV;
+ wc->wc_flags = IB_WC_WITH_IMM;
+ wc->ex.imm_data = le32_to_cpu(
+ cqe->immediate_data);
+ } else {
+ wc->opcode = IB_WC_RECV;
+ wc->wc_flags = 0;
+ }
+ break;
+ default:
+ wc->status = IB_WC_GENERAL_ERR;
+ break;
+ }
+
+ /* Update tail pointer, record wr_id */
+ wq = &(*cur_qp)->rq;
+ wc->wr_id = wq->wrid[wq->tail & (wq->wqe_cnt - 1)];
+ ++wq->tail;
+ wc->sl = (u8)roce_get_field(cqe->cqe_byte_20, CQE_BYTE_20_SL_M,
+ CQE_BYTE_20_SL_S);
+ wc->src_qp = (u8)roce_get_field(cqe->cqe_byte_20,
+ CQE_BYTE_20_REMOTE_QPN_M,
+ CQE_BYTE_20_REMOTE_QPN_S);
+ wc->wc_flags |= (roce_get_bit(cqe->cqe_byte_20,
+ CQE_BYTE_20_GRH_PRESENT_S) ?
+ IB_WC_GRH : 0);
+ wc->pkey_index = (u16)roce_get_field(cqe->cqe_byte_28,
+ CQE_BYTE_28_P_KEY_IDX_M,
+ CQE_BYTE_28_P_KEY_IDX_S);
+ }
+
+ return 0;
+}
+
+int hns_roce_v1_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
+{
+ struct hns_roce_cq *hr_cq = to_hr_cq(ibcq);
+ struct hns_roce_qp *cur_qp = NULL;
+ unsigned long flags;
+ int npolled;
+ int ret = 0;
+
+ spin_lock_irqsave(&hr_cq->lock, flags);
+
+ for (npolled = 0; npolled < num_entries; ++npolled) {
+ ret = hns_roce_v1_poll_one(hr_cq, &cur_qp, wc + npolled);
+ if (ret)
+ break;
+ }
+
+ if (npolled) {
+ hns_roce_v1_cq_set_ci(hr_cq, hr_cq->cons_index,
+ &to_hr_dev(ibcq->device)->cq_db_lock);
+ }
+
+ spin_unlock_irqrestore(&hr_cq->lock, flags);
+
+ if (ret == 0 || ret == -EAGAIN)
+ return npolled;
+ else
+ return ret;
+}
+
static int hns_roce_v1_qp_modify(struct hns_roce_dev *hr_dev,
struct hns_roce_mtt *mtt,
enum hns_roce_qp_state cur_state,
@@ -2191,7 +2528,7 @@ static void hns_roce_v1_destroy_qp_common(struct hns_roce_dev *hr_dev,
(enum ib_qp_state)hr_qp->state),
HNS_ROCE_QP_STATE_ERR, NULL,
hr_qp))
- dev_err(dev, "modify QP %06lx to ERR failed.\n",
+ dev_err(dev, "modify QP %06x to ERR failed.\n",
hr_qp->qpn);
/* Record issued doorbell */
@@ -2207,7 +2544,7 @@ static void hns_roce_v1_destroy_qp_common(struct hns_roce_dev *hr_dev,
sdbsendptr_val = roce_readl(hr_dev->reg_base +
ROCEE_SDB_SEND_PTR_REG);
if (!time_before(jiffies, end)) {
- dev_err(dev, "destroy qp(0x%lx) timeout!!!",
+ dev_err(dev, "destroy qp(0x%x) timeout!!!",
hr_qp->qpn);
break;
}
@@ -2228,7 +2565,7 @@ static void hns_roce_v1_destroy_qp_common(struct hns_roce_dev *hr_dev,
sdbinvcnt_val = roce_readl(hr_dev->reg_base +
ROCEE_SDB_INV_CNT_REG);
if (!time_before(jiffies, end)) {
- dev_err(dev, "destroy qp(0x%lx) timeout!!!",
+ dev_err(dev, "destroy qp(0x%x) timeout!!!",
hr_qp->qpn);
dev_err(dev, "SdbInvCnt = 0x%x\n",
sdbinvcnt_val);
@@ -2244,7 +2581,7 @@ static void hns_roce_v1_destroy_qp_common(struct hns_roce_dev *hr_dev,
to_hns_roce_state(
(enum ib_qp_state)hr_qp->state),
HNS_ROCE_QP_STATE_RST, NULL, hr_qp))
- dev_err(dev, "modify QP %06lx to RESET failed.\n",
+ dev_err(dev, "modify QP %06x to RESET failed.\n",
hr_qp->qpn);
}
}
@@ -2307,9 +2644,12 @@ struct hns_roce_hw hns_roce_hw_v1 = {
.set_gid = hns_roce_v1_set_gid,
.set_mac = hns_roce_v1_set_mac,
.set_mtu = hns_roce_v1_set_mtu,
+ .write_cqc = hns_roce_v1_write_cqc,
.modify_qp = hns_roce_v1_modify_qp,
.query_qp = hns_roce_v1_query_qp,
.destroy_qp = hns_roce_v1_destroy_qp,
.post_send = hns_roce_v1_post_send,
.post_recv = hns_roce_v1_post_recv,
+ .req_notify_cq = hns_roce_v1_req_notify_cq,
+ .poll_cq = hns_roce_v1_poll_cq,
};
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.h b/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
index 917985c..ba81540 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
@@ -10,6 +10,8 @@
#ifndef _HNS_ROCE_HW_V1_H
#define _HNS_ROCE_HW_V1_H
+#define CQ_STATE_VALID 2
+
#define HNS_ROCE_V1_MAX_PD_NUM 0x8000
#define HNS_ROCE_V1_MAX_CQ_NUM 0x10000
#define HNS_ROCE_V1_MAX_CQE_NUM 0x8000
@@ -85,6 +87,22 @@
#define HNS_ROCE_ODB_EXTEND_MODE 1
#define HNS_ROCE_CQE_QPN_MASK 0x3ffff
+#define HNS_ROCE_CQE_STATUS_MASK 0x1f
+#define HNS_ROCE_CQE_OPCODE_MASK 0xf
+
+#define HNS_ROCE_CQE_SUCCESS 0x00
+#define HNS_ROCE_CQE_SYNDROME_LOCAL_LENGTH_ERR 0x01
+#define HNS_ROCE_CQE_SYNDROME_LOCAL_QP_OP_ERR 0x02
+#define HNS_ROCE_CQE_SYNDROME_LOCAL_PROT_ERR 0x03
+#define HNS_ROCE_CQE_SYNDROME_WR_FLUSH_ERR 0x04
+#define HNS_ROCE_CQE_SYNDROME_MEM_MANAGE_OPERATE_ERR 0x05
+#define HNS_ROCE_CQE_SYNDROME_BAD_RESP_ERR 0x06
+#define HNS_ROCE_CQE_SYNDROME_LOCAL_ACCESS_ERR 0x07
+#define HNS_ROCE_CQE_SYNDROME_REMOTE_INVAL_REQ_ERR 0x08
+#define HNS_ROCE_CQE_SYNDROME_REMOTE_ACCESS_ERR 0x09
+#define HNS_ROCE_CQE_SYNDROME_REMOTE_OP_ERR 0x0a
+#define HNS_ROCE_CQE_SYNDROME_TRANSPORT_RETRY_EXC_ERR 0x0b
+#define HNS_ROCE_CQE_SYNDROME_RNR_RETRY_EXC_ERR 0x0c
#define QP1C_CFGN_OFFSET 0x28
#define PHY_PORT_OFFSET 0x8
@@ -95,17 +113,114 @@
#define QKEY_VAL 0x80010000
#define SDB_INV_CNT_OFFSET 8
+struct hns_roce_cq_context {
+ u32 cqc_byte_4;
+ u32 cq_bt_l;
+ u32 cqc_byte_12;
+ u32 cur_cqe_ba0_l;
+ u32 cqc_byte_20;
+ u32 cqe_tptr_addr_l;
+ u32 cur_cqe_ba1_l;
+ u32 cqc_byte_32;
+};
+
+#define CQ_CONTEXT_CQC_BYTE_4_CQC_STATE_S 0
+#define CQ_CONTEXT_CQC_BYTE_4_CQC_STATE_M \
+ (((1UL << 2) - 1) << CQ_CONTEXT_CQC_BYTE_4_CQC_STATE_S)
+
+#define CQ_CONTEXT_CQC_BYTE_4_CQN_S 16
+#define CQ_CONTEXT_CQC_BYTE_4_CQN_M \
+ (((1UL << 16) - 1) << CQ_CONTEXT_CQC_BYTE_4_CQN_S)
+
+#define CQ_CONTEXT_CQC_BYTE_12_CQ_BT_H_S 0
+#define CQ_CONTEXT_CQC_BYTE_12_CQ_BT_H_M \
+ (((1UL << 17) - 1) << CQ_CONTEXT_CQC_BYTE_12_CQ_BT_H_S)
+
+#define CQ_CONTEXT_CQC_BYTE_12_CQ_CQE_SHIFT_S 20
+#define CQ_CONTEXT_CQC_BYTE_12_CQ_CQE_SHIFT_M \
+ (((1UL << 4) - 1) << CQ_CONTEXT_CQC_BYTE_12_CQ_CQE_SHIFT_S)
+
+#define CQ_CONTEXT_CQC_BYTE_12_CEQN_S 24
+#define CQ_CONTEXT_CQC_BYTE_12_CEQN_M \
+ (((1UL << 5) - 1) << CQ_CONTEXT_CQC_BYTE_12_CEQN_S)
+
+#define CQ_CONTEXT_CQC_BYTE_20_CUR_CQE_BA0_H_S 0
+#define CQ_CONTEXT_CQC_BYTE_20_CUR_CQE_BA0_H_M \
+ (((1UL << 5) - 1) << CQ_CONTEXT_CQC_BYTE_20_CUR_CQE_BA0_H_S)
+
+#define CQ_CONTEXT_CQC_BYTE_20_CQ_CUR_INDEX_S 16
+#define CQ_CONTEXT_CQC_BYTE_20_CQ_CUR_INDEX_M \
+ (((1UL << 16) - 1) << CQ_CONTEXT_CQC_BYTE_20_CQ_CUR_INDEX_S)
+
+#define CQ_CONTEXT_CQC_BYTE_20_CQE_TPTR_ADDR_H_S 8
+#define CQ_CONTEXT_CQC_BYTE_20_CQE_TPTR_ADDR_H_M \
+ (((1UL << 5) - 1) << CQ_CONTEXT_CQC_BYTE_20_CQE_TPTR_ADDR_H_S)
+
+#define CQ_CONTEXT_CQC_BYTE_32_CUR_CQE_BA1_H_S 0
+#define CQ_CONTEXT_CQC_BYTE_32_CUR_CQE_BA1_H_M \
+ (((1UL << 5) - 1) << CQ_CONTEXT_CQC_BYTE_32_CUR_CQE_BA1_H_S)
+
+#define CQ_CONTEXT_CQC_BYTE_32_SE_FLAG_S 9
+
+#define CQ_CONTEXT_CQC_BYTE_32_CE_FLAG_S 8
+#define CQ_CONTEXT_CQC_BYTE_32_NOTIFICATION_FLAG_S 14
+#define CQ_CQNTEXT_CQC_BYTE_32_TYPE_OF_COMPLETION_NOTIFICATION_S 15
+
+#define CQ_CONTEXT_CQC_BYTE_32_CQ_CONS_IDX_S 16
+#define CQ_CONTEXT_CQC_BYTE_32_CQ_CONS_IDX_M \
+ (((1UL << 16) - 1) << CQ_CONTEXT_CQC_BYTE_32_CQ_CONS_IDX_S)
+
struct hns_roce_cqe {
u32 cqe_byte_4;
+ union {
+ u32 r_key;
+ u32 immediate_data;
+ };
+ u32 byte_cnt;
u32 cqe_byte_16;
+ u32 cqe_byte_20;
+ u32 s_mac_l;
+ u32 cqe_byte_28;
+ u32 reserved;
};
#define CQE_BYTE_4_OWNER_S 7
#define CQE_BYTE_4_SQ_RQ_FLAG_S 14
+#define CQE_BYTE_4_STATUS_OF_THE_OPERATION_S 8
+#define CQE_BYTE_4_STATUS_OF_THE_OPERATION_M \
+ (((1UL << 5) - 1) << CQE_BYTE_4_STATUS_OF_THE_OPERATION_S)
+
+#define CQE_BYTE_4_WQE_INDEX_S 16
+#define CQE_BYTE_4_WQE_INDEX_M (((1UL << 14) - 1) << CQE_BYTE_4_WQE_INDEX_S)
+
+#define CQE_BYTE_4_OPERATION_TYPE_S 0
+#define CQE_BYTE_4_OPERATION_TYPE_M \
+ (((1UL << 4) - 1) << CQE_BYTE_4_OPERATION_TYPE_S)
+
+#define CQE_BYTE_4_IMM_INDICATOR_S 15
+
#define CQE_BYTE_16_LOCAL_QPN_S 0
#define CQE_BYTE_16_LOCAL_QPN_M (((1UL << 24) - 1) << CQE_BYTE_16_LOCAL_QPN_S)
+#define CQE_BYTE_20_PORT_NUM_S 26
+#define CQE_BYTE_20_PORT_NUM_M (((1UL << 3) - 1) << CQE_BYTE_20_PORT_NUM_S)
+
+#define CQE_BYTE_20_SL_S 24
+#define CQE_BYTE_20_SL_M (((1UL << 2) - 1) << CQE_BYTE_20_SL_S)
+
+#define CQE_BYTE_20_REMOTE_QPN_S 0
+#define CQE_BYTE_20_REMOTE_QPN_M \
+ (((1UL << 24) - 1) << CQE_BYTE_20_REMOTE_QPN_S)
+
+#define CQE_BYTE_20_GRH_PRESENT_S 29
+
+#define CQE_BYTE_28_P_KEY_IDX_S 16
+#define CQE_BYTE_28_P_KEY_IDX_M (((1UL << 16) - 1) << CQE_BYTE_28_P_KEY_IDX_S)
+
+#define CQ_DB_REQ_NOT_SOL 0
+#define CQ_DB_REQ_NOT (1 << 16)
+
struct hns_roce_wqe_ctrl_seg {
__be32 sgl_pa_h;
__be32 flag;
@@ -314,6 +429,8 @@ struct hns_roce_sqp_context {
#define HNS_ROCE_WQE_OPCODE_SEND (0<<16)
#define HNS_ROCE_WQE_OPCODE_RDMA_READ (1<<16)
#define HNS_ROCE_WQE_OPCODE_RDMA_WRITE (2<<16)
+#define HNS_ROCE_WQE_OPCODE_LOCAL_INV (4<<16)
+#define HNS_ROCE_WQE_OPCODE_UD_SEND (7<<16)
#define HNS_ROCE_WQE_OPCODE_MASK (15<<16)
struct hns_roce_qp_context {
diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
index b8838b4..2c2cd06 100644
--- a/drivers/infiniband/hw/hns/hns_roce_main.c
+++ b/drivers/infiniband/hw/hns/hns_roce_main.c
@@ -588,6 +588,9 @@ int hns_roce_register_device(struct hns_roce_dev *hr_dev)
(1ULL << IB_USER_VERBS_CMD_QUERY_PORT) |
(1ULL << IB_USER_VERBS_CMD_ALLOC_PD) |
(1ULL << IB_USER_VERBS_CMD_DEALLOC_PD) |
+ (1ULL << IB_USER_VERBS_CMD_CREATE_COMP_CHANNEL) |
+ (1ULL << IB_USER_VERBS_CMD_CREATE_CQ) |
+ (1ULL << IB_USER_VERBS_CMD_DESTROY_CQ) |
(1ULL << IB_USER_VERBS_CMD_CREATE_QP) |
(1ULL << IB_USER_VERBS_CMD_MODIFY_QP) |
(1ULL << IB_USER_VERBS_CMD_QUERY_QP) |
@@ -622,6 +625,12 @@ int hns_roce_register_device(struct hns_roce_dev *hr_dev)
ib_dev->post_send = hr_dev->hw->post_send;
ib_dev->post_recv = hr_dev->hw->post_recv;
+ /* CQ */
+ ib_dev->create_cq = hns_roce_ib_create_cq;
+ ib_dev->destroy_cq = hns_roce_ib_destroy_cq;
+ ib_dev->req_notify_cq = hr_dev->hw->req_notify_cq;
+ ib_dev->poll_cq = hr_dev->hw->poll_cq;
+
ret = ib_register_device(ib_dev, NULL);
if (ret) {
dev_err(dev, "ib_register_device failed!\n");
diff --git a/drivers/infiniband/hw/hns/hns_roce_user.h b/drivers/infiniband/hw/hns/hns_roce_user.h
index 73c3f0c..8222b93 100644
--- a/drivers/infiniband/hw/hns/hns_roce_user.h
+++ b/drivers/infiniband/hw/hns/hns_roce_user.h
@@ -10,6 +10,10 @@
#ifndef _HNS_ROCE_USER_H
#define _HNS_ROCE_USER_H
+struct hns_roce_ib_create_cq {
+ __u64 buf_addr;
+};
+
struct hns_roce_ib_create_qp {
__u64 buf_addr;
__u8 log_sq_bb_count;
--
1.9.1
^ permalink raw reply related
* [PATCH v6 17/21] IB/hns: Add QP operations support
From: Lijun Ou @ 2016-04-28 12:09 UTC (permalink / raw)
To: dledford, sean.hefty, hal.rosenstock, davem, jeffrey.t.kirsher,
jiri, ogerlitz
Cc: linux-rdma, linux-kernel, netdev, gongyangming, xiaokun,
tangchaofei, oulijun, haifeng.wei, yisen.zhuang, yankejian,
charles.chenxin, linuxarm
In-Reply-To: <1461845396-61306-1-git-send-email-oulijun@huawei.com>
This patch was implementing for queue pair operations. QP Consists
of a Send Work Queue and a Receive Work Queue. Send and receive
queues are always created as a pair and remain that way throughout
their lifetime. A Queue Pair is identified by its Queue Pair Number.
QP operations as follows:
1. create QP. When a QP is created, a complete set of initial
attributes must be specified by the Consumer.
2. query QP. Returns the attribute list and current values for
the specified QP.
3. modify QP. modify QP relative attributes by it.
4. destroy QP. When a QP is destroyed, any outstanding Work
Requests are no longer considered to be in the scope of
the Channel Interface. It is the responsibility of the
Consumer to be able to clean up any resources
5. post send request. Builds one or more WQEs for the Send Queue
in the specified QP.
6. post receive request. Builds one or more WQEs for the receive
Queue in the specified QP.
Signed-off-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Nenglong Zhao <zhaonenglong@hisilicon.com>
Signed-off-by: Lijun Ou <oulijun@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_alloc.c | 134 +++
drivers/infiniband/hw/hns/hns_roce_cmd.c | 222 +++-
drivers/infiniband/hw/hns/hns_roce_cmd.h | 54 +-
drivers/infiniband/hw/hns/hns_roce_common.h | 58 +
drivers/infiniband/hw/hns/hns_roce_device.h | 183 ++-
drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 1642 +++++++++++++++++++++++++++
drivers/infiniband/hw/hns/hns_roce_hw_v1.h | 626 ++++++++++
drivers/infiniband/hw/hns/hns_roce_icm.c | 56 +
drivers/infiniband/hw/hns/hns_roce_icm.h | 9 +
drivers/infiniband/hw/hns/hns_roce_main.c | 14 +-
drivers/infiniband/hw/hns/hns_roce_mr.c | 160 +++
drivers/infiniband/hw/hns/hns_roce_qp.c | 765 +++++++++++++
drivers/infiniband/hw/hns/hns_roce_user.h | 6 +
13 files changed, 3911 insertions(+), 18 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_alloc.c b/drivers/infiniband/hw/hns/hns_roce_alloc.c
index 972d2ac..7357a4d 100644
--- a/drivers/infiniband/hw/hns/hns_roce_alloc.c
+++ b/drivers/infiniband/hw/hns/hns_roce_alloc.c
@@ -47,6 +47,45 @@ void hns_roce_bitmap_free(struct hns_roce_bitmap *bitmap, unsigned long obj)
hns_roce_bitmap_free_range(bitmap, obj, 1);
}
+int hns_roce_bitmap_alloc_range(struct hns_roce_bitmap *bitmap, int cnt,
+ int align, unsigned long *obj)
+{
+ int ret = 0;
+ int i;
+
+ if (likely(cnt == 1 && align == 1))
+ return hns_roce_bitmap_alloc(bitmap, obj);
+
+ spin_lock(&bitmap->lock);
+
+ *obj = bitmap_find_next_zero_area(bitmap->table, bitmap->max,
+ bitmap->last, cnt, align - 1);
+ if (*obj >= bitmap->max) {
+ bitmap->top = (bitmap->top + bitmap->max + bitmap->reserved_top)
+ & bitmap->mask;
+ *obj = bitmap_find_next_zero_area(bitmap->table, bitmap->max, 0,
+ cnt, align - 1);
+ }
+
+ if (*obj < bitmap->max) {
+ for (i = 0; i < cnt; i++)
+ set_bit(*obj + i, bitmap->table);
+
+ if (*obj == bitmap->last) {
+ bitmap->last = (*obj + cnt);
+ if (bitmap->last >= bitmap->max)
+ bitmap->last = 0;
+ }
+ *obj |= bitmap->top;
+ } else {
+ ret = -1;
+ }
+
+ spin_unlock(&bitmap->lock);
+
+ return ret;
+}
+
void hns_roce_bitmap_free_range(struct hns_roce_bitmap *bitmap,
unsigned long obj, int cnt)
{
@@ -94,6 +133,101 @@ void hns_roce_bitmap_cleanup(struct hns_roce_bitmap *bitmap)
kfree(bitmap->table);
}
+void hns_roce_buf_free(struct hns_roce_dev *hr_dev, u32 size,
+ struct hns_roce_buf *buf)
+{
+ int i;
+ struct device *dev = &hr_dev->pdev->dev;
+ u32 bits_per_long = BITS_PER_LONG;
+
+ if (buf->nbufs == 1) {
+ dma_free_coherent(dev, size, buf->direct.buf, buf->direct.map);
+ } else {
+ if (bits_per_long == 64)
+ vunmap(buf->direct.buf);
+
+ for (i = 0; i < buf->nbufs; ++i)
+ if (buf->page_list[i].buf)
+ dma_free_coherent(&hr_dev->pdev->dev, PAGE_SIZE,
+ buf->page_list[i].buf,
+ buf->page_list[i].map);
+ kfree(buf->page_list);
+ }
+}
+
+int hns_roce_buf_alloc(struct hns_roce_dev *hr_dev, u32 size, u32 max_direct,
+ struct hns_roce_buf *buf)
+{
+ int i = 0;
+ dma_addr_t t;
+ struct page **pages;
+ struct device *dev = &hr_dev->pdev->dev;
+ u32 bits_per_long = BITS_PER_LONG;
+
+ /* SQ/RQ buf lease than one page, SQ + RQ = 8K */
+ if (size <= max_direct) {
+ buf->nbufs = 1;
+ /* Npages calculated by page_size */
+ buf->npages = 1 << get_order(size);
+ buf->page_shift = PAGE_SHIFT;
+ /* MTT PA must be recorded in 4k alignment, t is 4k aligned */
+ buf->direct.buf = dma_alloc_coherent(dev, size, &t, GFP_KERNEL);
+ if (!buf->direct.buf)
+ return -ENOMEM;
+
+ buf->direct.map = t;
+
+ while (t & ((1 << buf->page_shift) - 1)) {
+ --buf->page_shift;
+ buf->npages *= 2;
+ }
+
+ memset(buf->direct.buf, 0, size);
+ } else {
+ buf->nbufs = (size + PAGE_SIZE - 1) / PAGE_SIZE;
+ buf->npages = buf->nbufs;
+ buf->page_shift = PAGE_SHIFT;
+ buf->page_list = kcalloc(buf->nbufs, sizeof(*buf->page_list),
+ GFP_KERNEL);
+
+ if (!buf->page_list)
+ return -ENOMEM;
+
+ for (i = 0; i < buf->nbufs; ++i) {
+ buf->page_list[i].buf = dma_alloc_coherent(dev,
+ PAGE_SIZE, &t,
+ GFP_KERNEL);
+
+ if (!buf->page_list[i].buf)
+ goto err_free;
+
+ buf->page_list[i].map = t;
+ memset(buf->page_list[i].buf, 0, PAGE_SIZE);
+ }
+ if (bits_per_long == 64) {
+ pages = kmalloc_array(buf->nbufs, sizeof(*pages),
+ GFP_KERNEL);
+ if (!pages)
+ goto err_free;
+
+ for (i = 0; i < buf->nbufs; ++i)
+ pages[i] = virt_to_page(buf->page_list[i].buf);
+
+ buf->direct.buf = vmap(pages, buf->nbufs, VM_MAP,
+ PAGE_KERNEL);
+ kfree(pages);
+ if (!buf->direct.buf)
+ goto err_free;
+ }
+ }
+
+ return 0;
+
+err_free:
+ hns_roce_buf_free(hr_dev, size, buf);
+ return -ENOMEM;
+}
+
void hns_roce_cleanup_bitmap(struct hns_roce_dev *hr_dev)
{
hns_roce_cleanup_qp_table(hr_dev);
diff --git a/drivers/infiniband/hw/hns/hns_roce_cmd.c b/drivers/infiniband/hw/hns/hns_roce_cmd.c
index aa1e0aa..e944b61 100644
--- a/drivers/infiniband/hw/hns/hns_roce_cmd.c
+++ b/drivers/infiniband/hw/hns/hns_roce_cmd.c
@@ -20,7 +20,19 @@
#include "hns_roce_device.h"
#include "hns_roce_cmd.h"
+#define CMD_POLL_TOKEN 0xffff
#define CMD_MAX_NUM 32
+#define STATUS_MASK 0xff
+
+enum {
+ HCR_TOKEN_OFFSET = 0x14,
+ HCR_STATUS_OFFSET = 0x18,
+ HCR_GO_BIT = 15,
+};
+
+enum {
+ GO_BIT_TIMEOUT_MSECS = 10000,
+};
static int hns_roce_status_to_errno(u8 orig_status)
{
@@ -30,6 +42,185 @@ static int hns_roce_status_to_errno(u8 orig_status)
return -EIO;
}
+static int cmd_pending(struct hns_roce_dev *hr_dev)
+{
+ u32 status = roce_readl(hr_dev->cmd.hcr + HCR_TOKEN_OFFSET);
+
+ return (!!(status & (1 << HCR_GO_BIT)));
+}
+
+static int hns_roce_cmd_post(struct hns_roce_dev *hr_dev, u64 in_param,
+ u64 out_param, u32 in_modifier, u8 op_modifier,
+ u16 op, u16 token, int event)
+{
+ struct hns_roce_cmdq *cmd = &hr_dev->cmd;
+ struct device *dev = &hr_dev->pdev->dev;
+ u32 __iomem *hcr = (u32 *)cmd->hcr;
+ int ret = -EAGAIN;
+ unsigned long end;
+ u32 val = 0;
+
+ mutex_lock(&cmd->hcr_mutex);
+
+ end = msecs_to_jiffies(GO_BIT_TIMEOUT_MSECS) + jiffies;
+ while (cmd_pending(hr_dev)) {
+ if (time_after(jiffies, end)) {
+ dev_dbg(dev, "jiffies=%d end=%d\n", (int)jiffies,
+ (int)end);
+ goto out;
+ }
+ cond_resched();
+ }
+
+ roce_set_field(val, ROCEE_MB6_ROCEE_MB_CMD_M, ROCEE_MB6_ROCEE_MB_CMD_S,
+ op);
+ roce_set_field(val, ROCEE_MB6_ROCEE_MB_CMD_MDF_M,
+ ROCEE_MB6_ROCEE_MB_CMD_MDF_S, op_modifier);
+ roce_set_bit(val, ROCEE_MB6_ROCEE_MB_EVENT_S, event);
+ roce_set_bit(val, ROCEE_MB6_ROCEE_MB_HW_RUN_S, 1);
+ roce_set_field(val, ROCEE_MB6_ROCEE_MB_TOKEN_M,
+ ROCEE_MB6_ROCEE_MB_TOKEN_S, token);
+
+ __raw_writeq(cpu_to_le64(in_param), hcr + 0);
+ __raw_writeq(cpu_to_le64(out_param), hcr + 2);
+ __raw_writel(cpu_to_le32(in_modifier), hcr + 4);
+ /* Memory barrier */
+ wmb();
+
+ __raw_writel(cpu_to_le32(val), hcr + 5);
+
+ mmiowb();
+ ret = 0;
+
+out:
+ mutex_unlock(&cmd->hcr_mutex);
+ return ret;
+}
+
+
+static int hns_roce_cmd_poll(struct hns_roce_dev *hr_dev, u64 in_param,
+ u64 *out_param, unsigned long in_modifier,
+ u8 op_modifier, u16 op, unsigned long timeout)
+{
+ struct device *dev = &hr_dev->pdev->dev;
+ u8 __iomem *hcr = hr_dev->cmd.hcr;
+ unsigned long end = 0;
+ u32 status = 0;
+ int ret;
+
+ down(&hr_dev->cmd.poll_sem);
+
+ ret = hns_roce_cmd_post(hr_dev, in_param, out_param ? *out_param : 0,
+ in_modifier, op_modifier, op, CMD_POLL_TOKEN,
+ 0);
+ if (ret) {
+ dev_err(dev, "[cmd_poll]hns_roce_cmd_post failed\n");
+ goto out;
+ }
+
+ end = msecs_to_jiffies(timeout) + jiffies;
+ while (cmd_pending(hr_dev) && time_before(jiffies, end))
+ cond_resched();
+
+ if (cmd_pending(hr_dev)) {
+ dev_err(dev, "[cmd_poll]hw run cmd TIMEDOUT!\n");
+ ret = -ETIMEDOUT;
+ goto out;
+ }
+
+ status = le32_to_cpu((__force __be32)
+ __raw_readl(hcr + HCR_STATUS_OFFSET));
+ if ((status & STATUS_MASK) != 0x1) {
+ dev_err(dev, "mailbox status 0x%x!\n", status);
+ ret = -EBUSY;
+ goto out;
+ }
+
+out:
+ up(&hr_dev->cmd.poll_sem);
+ return ret;
+}
+
+void hns_roce_cmd_event(struct hns_roce_dev *hr_dev, u16 token, u8 status,
+ u64 out_param)
+{
+ struct hns_roce_cmd_context
+ *context = &hr_dev->cmd.context[token & hr_dev->cmd.token_mask];
+
+ if (token != context->token)
+ return;
+
+ context->result = hns_roce_status_to_errno(status);
+ context->out_param = out_param;
+ complete(&context->done);
+}
+
+static int hns_roce_cmd_wait(struct hns_roce_dev *hr_dev, u64 in_param,
+ u64 *out_param, unsigned long in_modifier,
+ u8 op_modifier, u16 op, unsigned long timeout)
+{
+ struct hns_roce_cmdq *cmd = &hr_dev->cmd;
+ struct device *dev = &hr_dev->pdev->dev;
+ struct hns_roce_cmd_context *context;
+ int ret = 0;
+
+ down(&cmd->event_sem);
+
+ spin_lock(&cmd->context_lock);
+ WARN_ON(cmd->free_head < 0);
+ context = &cmd->context[cmd->free_head];
+ context->token += cmd->token_mask + 1;
+ cmd->free_head = context->next;
+ spin_unlock(&cmd->context_lock);
+
+ init_completion(&context->done);
+
+ ret = hns_roce_cmd_post(hr_dev, in_param, out_param ? *out_param : 0,
+ in_modifier, op_modifier, op, context->token,
+ 1);
+ if (ret)
+ goto out;
+
+ /*
+ * It is timeout when wait_for_completion_timeout return 0
+ * The return value is the time limit set in advance
+ * how many seconds showing
+ */
+ if (!wait_for_completion_timeout(&context->done,
+ msecs_to_jiffies(timeout))) {
+ dev_err(dev, "[cmd]wait_for_completion_timeout timeout\n");
+ ret = -EBUSY;
+ goto out;
+ }
+
+ ret = context->result;
+ if (ret) {
+ dev_err(dev, "[cmd]event mod cmd process error!err=%d\n", ret);
+ goto out;
+ }
+
+out:
+ spin_lock(&cmd->context_lock);
+ context->next = cmd->free_head;
+ cmd->free_head = context - cmd->context;
+ spin_unlock(&cmd->context_lock);
+
+ up(&cmd->event_sem);
+ return ret;
+}
+
+int __hns_roce_cmd(struct hns_roce_dev *hr_dev, u64 in_param, u64 *out_param,
+ unsigned long in_modifier, u8 op_modifier, u16 op,
+ unsigned long timeout)
+{
+ if (hr_dev->cmd.use_events)
+ return hns_roce_cmd_wait(hr_dev, in_param, out_param,
+ in_modifier, op_modifier, op, timeout);
+ else
+ return hns_roce_cmd_poll(hr_dev, in_param, out_param,
+ in_modifier, op_modifier, op, timeout);
+}
+
int hns_roce_cmd_init(struct hns_roce_dev *hr_dev)
{
struct device *dev = &hr_dev->pdev->dev;
@@ -103,16 +294,31 @@ void hns_roce_cmd_use_polling(struct hns_roce_dev *hr_dev)
up(&hr_cmd->poll_sem);
}
-void hns_roce_cmd_event(struct hns_roce_dev *hr_dev, u16 token, u8 status,
- u64 out_param)
+struct hns_roce_cmd_mailbox
+ *hns_roce_alloc_cmd_mailbox(struct hns_roce_dev *hr_dev)
{
- struct hns_roce_cmd_context
- *context = &hr_dev->cmd.context[token & hr_dev->cmd.token_mask];
+ struct hns_roce_cmd_mailbox *mailbox;
- if (token != context->token)
+ mailbox = kmalloc(sizeof(*mailbox), GFP_KERNEL);
+ if (!mailbox)
+ return ERR_PTR(-ENOMEM);
+
+ mailbox->buf = dma_pool_alloc(hr_dev->cmd.pool, GFP_KERNEL,
+ &mailbox->dma);
+ if (!mailbox->buf) {
+ kfree(mailbox);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ return mailbox;
+}
+
+void hns_roce_free_cmd_mailbox(struct hns_roce_dev *hr_dev,
+ struct hns_roce_cmd_mailbox *mailbox)
+{
+ if (!mailbox)
return;
- context->result = hns_roce_status_to_errno(status);
- context->out_param = out_param;
- complete(&context->done);
+ dma_pool_free(hr_dev->cmd.pool, mailbox->buf, mailbox->dma);
+ kfree(mailbox);
}
diff --git a/drivers/infiniband/hw/hns/hns_roce_cmd.h b/drivers/infiniband/hw/hns/hns_roce_cmd.h
index 4e102a4..9eb50fb 100644
--- a/drivers/infiniband/hw/hns/hns_roce_cmd.h
+++ b/drivers/infiniband/hw/hns/hns_roce_cmd.h
@@ -13,7 +13,59 @@
#include <linux/dma-mapping.h>
enum {
- HNS_ROCE_MAILBOX_SIZE = 4096
+ /* QP/EE commands */
+ HNS_ROCE_CMD_RST2INIT_QP = 0x19,
+ HNS_ROCE_CMD_INIT2RTR_QP = 0x1a,
+ HNS_ROCE_CMD_RTR2RTS_QP = 0x1b,
+ HNS_ROCE_CMD_RTS2RTS_QP = 0x1c,
+ HNS_ROCE_CMD_2ERR_QP = 0x1e,
+ HNS_ROCE_CMD_RTS2SQD_QP = 0x1f,
+ HNS_ROCE_CMD_SQD2SQD_QP = 0x38,
+ HNS_ROCE_CMD_SQD2RTS_QP = 0x20,
+ HNS_ROCE_CMD_2RST_QP = 0x21,
+ HNS_ROCE_CMD_QUERY_QP = 0x22,
};
+enum {
+ HNS_ROCE_CMD_TIME_CLASS_A = 10000,
+ HNS_ROCE_CMD_TIME_CLASS_C = 10000,
+};
+
+enum {
+ HNS_ROCE_MAILBOX_SIZE = 4096,
+};
+
+struct hns_roce_cmd_mailbox {
+ void *buf;
+ dma_addr_t dma;
+};
+
+int __hns_roce_cmd(struct hns_roce_dev *hr_dev, u64 in_param, u64 *out_param,
+ unsigned long in_modifier, u8 op_modifier, u16 op,
+ unsigned long timeout);
+
+/* Invoke a command with no output parameter */
+static inline int hns_roce_cmd(struct hns_roce_dev *hr_dev, u64 in_param,
+ unsigned long in_modifier, u8 op_modifier,
+ u16 op, unsigned long timeout)
+{
+ return __hns_roce_cmd(hr_dev, in_param, NULL, in_modifier,
+ op_modifier, op, timeout);
+}
+
+/* Invoke a command with an output mailbox */
+static inline int hns_roce_cmd_box(struct hns_roce_dev *hr_dev, u64 in_param,
+ u64 out_param, unsigned long in_modifier,
+ u8 op_modifier, u16 op,
+ unsigned long timeout)
+{
+ return __hns_roce_cmd(hr_dev, in_param, &out_param, in_modifier,
+ op_modifier, op, timeout);
+}
+
+struct hns_roce_cmd_mailbox
+ *hns_roce_alloc_cmd_mailbox(struct hns_roce_dev *hr_dev);
+void hns_roce_free_cmd_mailbox(struct hns_roce_dev *hr_dev,
+ struct hns_roce_cmd_mailbox *mailbox);
+
#endif /* _HNS_ROCE_CMD_H */
diff --git a/drivers/infiniband/hw/hns/hns_roce_common.h b/drivers/infiniband/hw/hns/hns_roce_common.h
index 135203e..740564f 100644
--- a/drivers/infiniband/hw/hns/hns_roce_common.h
+++ b/drivers/infiniband/hw/hns/hns_roce_common.h
@@ -137,6 +137,44 @@
#define ROCEE_BT_CMD_H_ROCEE_BT_CMD_HW_SYNS_S 31
+#define ROCEE_QP1C_CFG0_0_ROCEE_QP1C_QP_ST_S 0
+#define ROCEE_QP1C_CFG0_0_ROCEE_QP1C_QP_ST_M \
+ (((1UL << 3) - 1) << ROCEE_QP1C_CFG0_0_ROCEE_QP1C_QP_ST_S)
+
+#define ROCEE_QP1C_CFG3_0_ROCEE_QP1C_RQ_HEAD_S 0
+#define ROCEE_QP1C_CFG3_0_ROCEE_QP1C_RQ_HEAD_M \
+ (((1UL << 15) - 1) << ROCEE_QP1C_CFG3_0_ROCEE_QP1C_RQ_HEAD_S)
+
+#define ROCEE_MB6_ROCEE_MB_CMD_S 0
+#define ROCEE_MB6_ROCEE_MB_CMD_M \
+ (((1UL << 8) - 1) << ROCEE_MB6_ROCEE_MB_CMD_S)
+
+#define ROCEE_MB6_ROCEE_MB_CMD_MDF_S 8
+#define ROCEE_MB6_ROCEE_MB_CMD_MDF_M \
+ (((1UL << 4) - 1) << ROCEE_MB6_ROCEE_MB_CMD_MDF_S)
+
+#define ROCEE_MB6_ROCEE_MB_EVENT_S 14
+
+#define ROCEE_MB6_ROCEE_MB_HW_RUN_S 15
+
+#define ROCEE_MB6_ROCEE_MB_TOKEN_S 16
+#define ROCEE_MB6_ROCEE_MB_TOKEN_M \
+ (((1UL << 16) - 1) << ROCEE_MB6_ROCEE_MB_TOKEN_S)
+
+#define ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_INP_H_S 0
+#define ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_INP_H_M \
+ (((1UL << 24) - 1) << ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_INP_H_S)
+
+#define ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_CMD_MDF_S 24
+#define ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_CMD_MDF_M \
+ (((1UL << 4) - 1) << ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_CMD_MDF_S)
+
+#define ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_CMD_S 28
+#define ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_CMD_M \
+ (((1UL << 3) - 1) << ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_CMD_S)
+
+#define ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_HW_SYNS_S 31
+
#define ROCEE_SMAC_H_ROCEE_SMAC_H_S 0
#define ROCEE_SMAC_H_ROCEE_SMAC_H_M \
(((1UL << 16) - 1) << ROCEE_SMAC_H_ROCEE_SMAC_H_S)
@@ -176,6 +214,18 @@
#define ROCEE_CAEP_AE_ST_CAEP_AEQ_ALM_OVF_S 0
+#define ROCEE_SDB_ISSUE_PTR_SDB_ISSUE_PTR_S 0
+#define ROCEE_SDB_ISSUE_PTR_SDB_ISSUE_PTR_M \
+ (((1UL << 28) - 1) << ROCEE_SDB_ISSUE_PTR_SDB_ISSUE_PTR_S)
+
+#define ROCEE_SDB_SEND_PTR_SDB_SEND_PTR_S 0
+#define ROCEE_SDB_SEND_PTR_SDB_SEND_PTR_M \
+ (((1UL << 28) - 1) << ROCEE_SDB_SEND_PTR_SDB_SEND_PTR_S)
+
+#define ROCEE_SDB_INV_CNT_SDB_INV_CNT_S 0
+#define ROCEE_SDB_INV_CNT_SDB_INV_CNT_M \
+ (((1UL << 16) - 1) << ROCEE_SDB_INV_CNT_SDB_INV_CNT_S)
+
/*************ROCEE_REG DEFINITION****************/
#define ROCEE_VENDOR_ID_REG 0x0
#define ROCEE_VENDOR_PART_ID_REG 0x4
@@ -195,6 +245,8 @@
#define ROCEE_SMAC_L_0_REG 0x240
#define ROCEE_SMAC_H_0_REG 0x244
+#define ROCEE_QP1C_CFG3_0_REG 0x27C
+
#define ROCEE_CAEP_AEQE_CONS_IDX_REG 0x3AC
#define ROCEE_CAEP_CEQC_CONS_IDX_0_REG 0x3BC
@@ -230,6 +282,9 @@
#define ROCEE_BT_CMD_L_REG 0x200
#define ROCEE_MB1_REG 0x210
+#define ROCEE_DB_SQ_L_0_REG 0x230
+#define ROCEE_DB_OTHERS_L_0_REG 0x238
+#define ROCEE_QP1C_CFG0_0_REG 0x270
#define ROCEE_CAEP_AEQC_AEQE_SHIFT_REG 0x3A0
#define ROCEE_CAEP_CEQC_SHIFT_0_REG 0x3B0
@@ -238,6 +293,9 @@
#define ROCEE_CAEP_AE_MASK_REG 0x6C8
#define ROCEE_CAEP_AE_ST_REG 0x6CC
+#define ROCEE_SDB_ISSUE_PTR_REG 0x758
+#define ROCEE_SDB_SEND_PTR_REG 0x75C
+#define ROCEE_SDB_INV_CNT_REG 0x9A4
#define ROCEE_ECC_UCERR_ALM0_REG 0xB34
#define ROCEE_ECC_CERR_ALM0_REG 0xB40
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index df2bc00..d6a0f9e 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -24,8 +24,15 @@
#define MAC_ADDR_OCTET_NUM 6
#define HNS_ROCE_MAX_MSG_LEN 0x80000000
+#define HNS_ROCE_ALOGN_UP(a, b) ((((a) + (b) - 1) / (b)) * (b))
+
+#define HNS_ROCE_IB_MIN_SQ_STRIDE 6
+
#define HNS_ROCE_BA_SIZE (32 * 4096)
+/* Hardware specification only for v1 engine */
+#define HNS_ROCE_MIN_WQE_NUM 0x20
+
#define HNS_ROCE_MAX_IRQ_NUM 34
#define HNS_ROCE_MAX_PORTS 6
@@ -51,6 +58,16 @@
#define PAGES_SHIFT_16 16
+enum hns_roce_qp_state {
+ HNS_ROCE_QP_STATE_RST = 0,
+ HNS_ROCE_QP_STATE_INIT = 1,
+ HNS_ROCE_QP_STATE_RTR = 2,
+ HNS_ROCE_QP_STATE_RTS = 3,
+ HNS_ROCE_QP_STATE_SQD = 4,
+ HNS_ROCE_QP_STATE_ERR = 5,
+ HNS_ROCE_QP_NUM_STATE
+};
+
enum hns_roce_event {
HNS_ROCE_EVENT_TYPE_PATH_MIG = 0x01,
HNS_ROCE_EVENT_TYPE_PATH_MIG_FAILED = 0x02,
@@ -110,6 +127,10 @@ enum {
#define HNS_ROCE_PORT_DOWN 0
#define HNS_ROCE_PORT_UP 1
+#define HNS_ROCE_MTT_ENTRY_PER_SEG 8
+
+#define PAGE_ADDR_SHIFT 12
+
struct hns_roce_uar {
u64 pfn;
unsigned long index;
@@ -154,17 +175,23 @@ struct hns_roce_buddy {
struct hns_roce_icm_table {
/* ICM type: 0 = qpc 1 = mtt 2 = cqc 3 = srq 4 = other */
- u32 type;
+ u32 type;
/* ICM array elment num */
- unsigned long num_icm;
+ unsigned long num_icm;
/* ICM entry record obj total num */
- unsigned long num_obj;
+ unsigned long num_obj;
/*Single obj size */
- unsigned long obj_size;
- int lowmem;
- int coherent;
- struct mutex mutex;
- struct hns_roce_icm **icm;
+ unsigned long obj_size;
+ int lowmem;
+ int coherent;
+ struct mutex mutex;
+ struct hns_roce_icm **icm;
+};
+
+struct hns_roce_mtt {
+ unsigned long first_seg;
+ int order;
+ int page_shift;
};
struct hns_roce_mr_table {
@@ -174,19 +201,57 @@ struct hns_roce_mr_table {
struct hns_roce_icm_table mtpt_table;
};
+struct hns_roce_wq {
+ u64 *wrid; /* Work request ID */
+ spinlock_t lock;
+ int wqe_cnt; /* WQE num */
+ u32 max_post;
+ int max_gs;
+ int offset;
+ int wqe_shift;/* WQE size */
+ u32 head;
+ u32 tail;
+ void __iomem *db_reg_l;
+};
+
struct hns_roce_buf_list {
void *buf;
dma_addr_t map;
};
+struct hns_roce_buf {
+ struct hns_roce_buf_list direct;
+ struct hns_roce_buf_list *page_list;
+ int nbufs;
+ u32 npages;
+ int page_shift;
+};
+
+struct hns_roce_cq_buf {
+ struct hns_roce_buf hr_buf;
+};
+
struct hns_roce_cq {
+ struct ib_cq ib_cq;
+ struct hns_roce_cq_buf hr_buf;
+ /* pointer to store information after resize*/
+ spinlock_t lock;
void (*comp)(struct hns_roce_cq *);
void (*event)(struct hns_roce_cq *, enum hns_roce_event);
+ u32 cq_depth;
+ u32 cons_index;
+ void __iomem *cq_db_l;
+ unsigned long cqn;
atomic_t refcount;
struct completion free;
};
+struct hns_roce_srq {
+ struct ib_srq ibsrq;
+ int srqn;
+};
+
struct hns_roce_uar_table {
struct hns_roce_bitmap bitmap;
};
@@ -268,13 +333,38 @@ struct hns_roce_cmdq {
struct hns_roce_dev;
struct hns_roce_qp {
+ struct ib_qp ibqp;
+ struct hns_roce_buf hr_buf;
+ struct hns_roce_wq rq;
+ __le64 doorbell_qpn;
+ __le32 sq_signal_bits;
+ u32 sq_next_wqe;
+ int sq_max_wqes_per_wr;
+ int sq_spare_wqes;
+ struct hns_roce_wq sq;
+
+ struct ib_umem *umem;
+ struct hns_roce_mtt mtt;
+ u32 buff_size;
+ struct mutex mutex;
+ u8 port;
+ u8 sl;
+ u8 resp_depth;
+ u8 state;
+ u32 access_flags;
+ u32 pkey_index;
void (*event)(struct hns_roce_qp *,
enum hns_roce_event);
+ unsigned long qpn;
atomic_t refcount;
struct completion free;
};
+struct hns_roce_sqp {
+ struct hns_roce_qp hr_qp;
+};
+
struct hns_roce_ib_iboe {
spinlock_t lock;
struct net_device *netdevs[HNS_ROCE_MAX_PORTS];
@@ -361,6 +451,16 @@ struct hns_roce_hw {
void (*set_mac)(struct hns_roce_dev *hr_dev, u8 phy_port, u8 *addr);
void (*set_mtu)(struct hns_roce_dev *hr_dev, u8 phy_port,
enum ib_mtu mtu);
+ int (*query_qp)(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr,
+ int qp_attr_mask, struct ib_qp_init_attr *qp_init_attr);
+ int (*modify_qp)(struct ib_qp *ibqp, const struct ib_qp_attr *attr,
+ int attr_mask, enum ib_qp_state cur_state,
+ enum ib_qp_state new_state);
+ int (*destroy_qp)(struct ib_qp *ibqp);
+ int (*post_send)(struct ib_qp *ibqp, struct ib_send_wr *wr,
+ struct ib_send_wr **bad_wr);
+ int (*post_recv)(struct ib_qp *qp, struct ib_recv_wr *recv_wr,
+ struct ib_recv_wr **bad_recv_wr);
void *priv;
};
@@ -419,6 +519,26 @@ static inline struct hns_roce_ah *to_hr_ah(struct ib_ah *ibah)
return container_of(ibah, struct hns_roce_ah, ibah);
}
+static inline struct hns_roce_qp *to_hr_qp(struct ib_qp *ibqp)
+{
+ return container_of(ibqp, struct hns_roce_qp, ibqp);
+}
+
+static inline struct hns_roce_cq *to_hr_cq(struct ib_cq *ib_cq)
+{
+ return container_of(ib_cq, struct hns_roce_cq, ib_cq);
+}
+
+static inline struct hns_roce_srq *to_hr_srq(struct ib_srq *ibsrq)
+{
+ return container_of(ibsrq, struct hns_roce_srq, ibsrq);
+}
+
+static inline struct hns_roce_sqp *hr_to_hr_sqp(struct hns_roce_qp *hr_qp)
+{
+ return container_of(hr_qp, struct hns_roce_sqp, hr_qp);
+}
+
static inline void hns_roce_write64_k(__be32 val[2], void __iomem *dest)
{
__raw_writeq(*(u64 *) val, dest);
@@ -431,6 +551,17 @@ static inline struct hns_roce_qp
qpn & (hr_dev->caps.num_qps - 1));
}
+static inline void *hns_roce_buf_offset(struct hns_roce_buf *buf, int offset)
+{
+ u32 bits_per_long_val = BITS_PER_LONG;
+
+ if (bits_per_long_val == 64 || buf->nbufs == 1)
+ return (char *)(buf->direct.buf) + offset;
+ else
+ return (char *)(buf->page_list[offset >> PAGE_SHIFT].buf) +
+ (offset & (PAGE_SIZE - 1));
+}
+
int hns_roce_init_uar_table(struct hns_roce_dev *dev);
int hns_roce_uar_alloc(struct hns_roce_dev *dev, struct hns_roce_uar *uar);
void hns_roce_uar_free(struct hns_roce_dev *dev, struct hns_roce_uar *uar);
@@ -443,6 +574,13 @@ void hns_roce_cmd_event(struct hns_roce_dev *hr_dev, u16 token, u8 status,
int hns_roce_cmd_use_events(struct hns_roce_dev *hr_dev);
void hns_roce_cmd_use_polling(struct hns_roce_dev *hr_dev);
+int hns_roce_mtt_init(struct hns_roce_dev *hr_dev, int npages, int page_shift,
+ struct hns_roce_mtt *mtt);
+void hns_roce_mtt_cleanup(struct hns_roce_dev *hr_dev,
+ struct hns_roce_mtt *mtt);
+int hns_roce_buf_write_mtt(struct hns_roce_dev *hr_dev,
+ struct hns_roce_mtt *mtt, struct hns_roce_buf *buf);
+
int hns_roce_init_pd_table(struct hns_roce_dev *hr_dev);
int hns_roce_init_mr_table(struct hns_roce_dev *hr_dev);
int hns_roce_init_eq_table(struct hns_roce_dev *hr_dev);
@@ -477,6 +615,35 @@ int hns_roce_pd_alloc(struct hns_roce_dev *hr_dev, unsigned long *pdn);
void hns_roce_pd_free(struct hns_roce_dev *hr_dev, unsigned long pdn);
int hns_roce_dealloc_pd(struct ib_pd *pd);
+void hns_roce_buf_free(struct hns_roce_dev *hr_dev, u32 size,
+ struct hns_roce_buf *buf);
+int hns_roce_buf_alloc(struct hns_roce_dev *hr_dev, u32 size, u32 max_direct,
+ struct hns_roce_buf *buf);
+
+int hns_roce_ib_umem_write_mtt(struct hns_roce_dev *hr_dev,
+ struct hns_roce_mtt *mtt, struct ib_umem *umem);
+
+struct ib_qp *hns_roce_create_qp(struct ib_pd *ib_pd,
+ struct ib_qp_init_attr *init_attr,
+ struct ib_udata *udata);
+int hns_roce_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
+ int attr_mask, struct ib_udata *udata);
+void *get_recv_wqe(struct hns_roce_qp *hr_qp, int n);
+void *get_send_wqe(struct hns_roce_qp *hr_qp, int n);
+bool hns_roce_wq_overflow(struct hns_roce_wq *hr_wq, int nreq,
+ struct ib_cq *ib_cq);
+enum hns_roce_qp_state to_hns_roce_state(enum ib_qp_state state);
+void hns_roce_lock_cqs(struct hns_roce_cq *send_cq,
+ struct hns_roce_cq *recv_cq);
+void hns_roce_unlock_cqs(struct hns_roce_cq *send_cq,
+ struct hns_roce_cq *recv_cq);
+void hns_roce_qp_remove(struct hns_roce_dev *hr_dev, struct hns_roce_qp *hr_qp);
+void hns_roce_qp_free(struct hns_roce_dev *hr_dev, struct hns_roce_qp *hr_qp);
+void hns_roce_release_range_qp(struct hns_roce_dev *hr_dev, int base_qpn,
+ int cnt);
+__be32 send_ieth(struct ib_send_wr *wr);
+int to_hr_qp_type(int qp_type);
+
void hns_roce_cq_completion(struct hns_roce_dev *hr_dev, u32 cqn);
void hns_roce_cq_event(struct hns_roce_dev *hr_dev, u32 cqn, int event_type);
void hns_roce_qp_event(struct hns_roce_dev *hr_dev, u32 qpn, int event_type);
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
index 7a5b211..3f8d6d1 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
@@ -19,8 +19,374 @@
#include <linux/platform_device.h>
#include "hns_roce_common.h"
#include "hns_roce_device.h"
+#include "hns_roce_cmd.h"
+#include "hns_roce_icm.h"
#include "hns_roce_hw_v1.h"
+static void set_data_seg(struct hns_roce_wqe_data_seg *dseg, struct ib_sge *sg)
+{
+ dseg->lkey = cpu_to_le32(sg->lkey);
+ dseg->addr = cpu_to_le64(sg->addr);
+ dseg->len = cpu_to_le32(sg->length);
+}
+
+static void set_raddr_seg(struct hns_roce_wqe_raddr_seg *rseg, u64 remote_addr,
+ u32 rkey)
+{
+ rseg->raddr = cpu_to_le64(remote_addr);
+ rseg->rkey = cpu_to_le32(rkey);
+ rseg->len = 0;
+}
+
+int hns_roce_v1_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
+ struct ib_send_wr **bad_wr)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
+ struct hns_roce_ah *ah = to_hr_ah(ud_wr(wr)->ah);
+ struct hns_roce_ud_send_wqe *ud_sq_wqe = NULL;
+ struct hns_roce_wqe_ctrl_seg *ctrl = NULL;
+ struct hns_roce_wqe_data_seg *dseg = NULL;
+ struct hns_roce_qp *qp = to_hr_qp(ibqp);
+ struct device *dev = &hr_dev->pdev->dev;
+ struct hns_roce_sq_db sq_db;
+ int ps_opcode = 0, i = 0;
+ unsigned long flags = 0;
+ void *wqe = NULL;
+ u32 doorbell[2];
+ int nreq = 0;
+ u32 ind = 0;
+ int ret = 0;
+
+ spin_lock_irqsave(&qp->sq.lock, flags);
+
+ ind = qp->sq_next_wqe;
+ for (nreq = 0; wr; ++nreq, wr = wr->next) {
+ if (hns_roce_wq_overflow(&qp->sq, nreq, qp->ibqp.send_cq)) {
+ dev_err(dev, "hns_roce_wq_overflow error\n");
+ ret = -ENOMEM;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(wr->num_sge > qp->sq.max_gs)) {
+ dev_err(dev, "num_sge=%d > qp->sq.max_gs=%d\n",
+ wr->num_sge, qp->sq.max_gs);
+ ret = -EINVAL;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ wqe = get_send_wqe(qp, ind & (qp->sq.wqe_cnt - 1));
+ qp->sq.wrid[(qp->sq.head + nreq) & (qp->sq.wqe_cnt - 1)] =
+ wr->wr_id;
+
+ /* Corresponding to the RC and RD type wqe process separately */
+ if (ibqp->qp_type == IB_QPT_GSI) {
+ ud_sq_wqe = wqe;
+ roce_set_field(ud_sq_wqe->dmac_h,
+ UD_SEND_WQE_U32_4_DMAC_0_M,
+ UD_SEND_WQE_U32_4_DMAC_0_S,
+ ah->av.mac[0]);
+ roce_set_field(ud_sq_wqe->dmac_h,
+ UD_SEND_WQE_U32_4_DMAC_1_M,
+ UD_SEND_WQE_U32_4_DMAC_1_S,
+ ah->av.mac[1]);
+ roce_set_field(ud_sq_wqe->dmac_h,
+ UD_SEND_WQE_U32_4_DMAC_2_M,
+ UD_SEND_WQE_U32_4_DMAC_2_S,
+ ah->av.mac[2]);
+ roce_set_field(ud_sq_wqe->dmac_h,
+ UD_SEND_WQE_U32_4_DMAC_3_M,
+ UD_SEND_WQE_U32_4_DMAC_3_S,
+ ah->av.mac[3]);
+
+ roce_set_field(ud_sq_wqe->u32_8,
+ UD_SEND_WQE_U32_8_DMAC_4_M,
+ UD_SEND_WQE_U32_8_DMAC_4_S,
+ ah->av.mac[4]);
+ roce_set_field(ud_sq_wqe->u32_8,
+ UD_SEND_WQE_U32_8_DMAC_5_M,
+ UD_SEND_WQE_U32_8_DMAC_5_S,
+ ah->av.mac[5]);
+ roce_set_field(ud_sq_wqe->u32_8,
+ UD_SEND_WQE_U32_8_OPERATION_TYPE_M,
+ UD_SEND_WQE_U32_8_OPERATION_TYPE_S,
+ HNS_ROCE_WQE_OPCODE_SEND);
+ roce_set_field(ud_sq_wqe->u32_8,
+ UD_SEND_WQE_U32_8_NUMBER_OF_DATA_SEG_M,
+ UD_SEND_WQE_U32_8_NUMBER_OF_DATA_SEG_S,
+ 2);
+ roce_set_bit(ud_sq_wqe->u32_8,
+ UD_SEND_WQE_U32_8_SEND_GL_ROUTING_HDR_FLAG_S,
+ 1);
+
+ ud_sq_wqe->u32_8 |= (wr->send_flags & IB_SEND_SIGNALED ?
+ cpu_to_le32(HNS_ROCE_WQE_CQ_NOTIFY) : 0) |
+ (wr->send_flags & IB_SEND_SOLICITED ?
+ cpu_to_le32(HNS_ROCE_WQE_SE) : 0) |
+ ((wr->opcode == IB_WR_SEND_WITH_IMM) ?
+ cpu_to_le32(HNS_ROCE_WQE_IMM) : 0);
+
+ roce_set_field(ud_sq_wqe->u32_16,
+ UD_SEND_WQE_U32_16_DEST_QP_M,
+ UD_SEND_WQE_U32_16_DEST_QP_S,
+ ud_wr(wr)->remote_qpn);
+ roce_set_field(ud_sq_wqe->u32_16,
+ UD_SEND_WQE_U32_16_MAX_STATIC_RATE_M,
+ UD_SEND_WQE_U32_16_MAX_STATIC_RATE_S,
+ ah->av.stat_rate);
+
+ roce_set_field(ud_sq_wqe->u32_36,
+ UD_SEND_WQE_U32_36_FLOW_LABEL_M,
+ UD_SEND_WQE_U32_36_FLOW_LABEL_S, 0);
+ roce_set_field(ud_sq_wqe->u32_36,
+ UD_SEND_WQE_U32_36_PRIORITY_M,
+ UD_SEND_WQE_U32_36_PRIORITY_S,
+ ah->av.sl_tclass_flowlabel >>
+ HNS_ROCE_SL_SHIFT);
+ roce_set_field(ud_sq_wqe->u32_36,
+ UD_SEND_WQE_U32_36_SGID_INDEX_M,
+ UD_SEND_WQE_U32_36_SGID_INDEX_S,
+ hns_get_gid_index(hr_dev, qp->port,
+ ah->av.gid_index));
+
+ roce_set_field(ud_sq_wqe->u32_40,
+ UD_SEND_WQE_U32_40_HOP_LIMIT_M,
+ UD_SEND_WQE_U32_40_HOP_LIMIT_S,
+ ah->av.hop_limit);
+ roce_set_field(ud_sq_wqe->u32_40,
+ UD_SEND_WQE_U32_40_TRAFFIC_CLASS_M,
+ UD_SEND_WQE_U32_40_TRAFFIC_CLASS_S, 0);
+
+ memcpy(&ud_sq_wqe->dgid[0], &ah->av.dgid[0], GID_LEN);
+
+ ud_sq_wqe->va0_l = (u32)wr->sg_list[0].addr;
+ ud_sq_wqe->va0_h = (wr->sg_list[0].addr) >>
+ ADDR_SHIFT_32;
+ ud_sq_wqe->l_key0 = wr->sg_list[0].lkey;
+
+ ud_sq_wqe->va1_l = (u32)wr->sg_list[1].addr;
+ ud_sq_wqe->va1_h = (wr->sg_list[1].addr) >>
+ ADDR_SHIFT_32;
+ ud_sq_wqe->l_key1 = wr->sg_list[1].lkey;
+ ind++;
+ } else if (ibqp->qp_type == IB_QPT_RC) {
+ ctrl = wqe;
+ memset(ctrl, 0, sizeof(struct hns_roce_wqe_ctrl_seg));
+ for (i = 0; i < wr->num_sge; i++)
+ ctrl->msg_length += wr->sg_list[i].length;
+
+ ctrl->sgl_pa_h = 0;
+ ctrl->flag = 0;
+ ctrl->imm_data = send_ieth(wr);
+
+ /*Ctrl field, ctrl set type: sig, solic, imm, fence */
+ /* SO wait for conforming application scenarios */
+ ctrl->flag |= (wr->send_flags & IB_SEND_SIGNALED ?
+ cpu_to_le32(HNS_ROCE_WQE_CQ_NOTIFY) : 0) |
+ (wr->send_flags & IB_SEND_SOLICITED ?
+ cpu_to_le32(HNS_ROCE_WQE_SE) : 0) |
+ ((wr->opcode == IB_WR_SEND_WITH_IMM ||
+ wr->opcode == IB_WR_RDMA_WRITE_WITH_IMM) ?
+ cpu_to_le32(HNS_ROCE_WQE_IMM) : 0) |
+ (wr->send_flags & IB_SEND_FENCE ?
+ (cpu_to_le32(HNS_ROCE_WQE_FENCE)) : 0);
+
+ wqe = (struct hns_roce_wqe_ctrl_seg *)wqe +
+ sizeof(struct hns_roce_wqe_ctrl_seg);
+
+ switch (wr->opcode) {
+ case IB_WR_RDMA_READ:
+ ps_opcode = HNS_ROCE_WQE_OPCODE_RDMA_READ;
+ set_raddr_seg(wqe, atomic_wr(wr)->remote_addr,
+ atomic_wr(wr)->rkey);
+ break;
+ case IB_WR_RDMA_WRITE:
+ case IB_WR_RDMA_WRITE_WITH_IMM:
+ ps_opcode = HNS_ROCE_WQE_OPCODE_RDMA_WRITE;
+ set_raddr_seg(wqe, atomic_wr(wr)->remote_addr,
+ atomic_wr(wr)->rkey);
+ break;
+ case IB_WR_SEND:
+ case IB_WR_SEND_WITH_INV:
+ case IB_WR_SEND_WITH_IMM:
+ ps_opcode = HNS_ROCE_WQE_OPCODE_SEND;
+ break;
+ case IB_WR_LOCAL_INV:
+ break;
+ case IB_WR_ATOMIC_CMP_AND_SWP:
+ case IB_WR_ATOMIC_FETCH_AND_ADD:
+ case IB_WR_LSO:
+ default:
+ ps_opcode = HNS_ROCE_WQE_OPCODE_MASK;
+ break;
+ }
+ ctrl->flag |= cpu_to_le32(ps_opcode);
+ wqe = (struct hns_roce_wqe_raddr_seg *)wqe +
+ sizeof(struct hns_roce_wqe_raddr_seg);
+
+ dseg = wqe;
+ if (wr->send_flags & IB_SEND_INLINE && wr->num_sge) {
+ if (ctrl->msg_length >
+ hr_dev->caps.max_sq_inline) {
+ ret = -EINVAL;
+ *bad_wr = wr;
+ dev_err(dev, "inline len(1-%d)=%d, illegal",
+ ctrl->msg_length,
+ hr_dev->caps.max_sq_inline);
+ goto out;
+ }
+ for (i = 0; i < wr->num_sge; i++) {
+ memcpy(wqe, ((void *) (uintptr_t)
+ wr->sg_list[i].addr),
+ wr->sg_list[i].length);
+ wqe = (struct hns_roce_wqe_raddr_seg *)
+ wqe + wr->sg_list[i].length;
+ }
+ ctrl->flag |= HNS_ROCE_WQE_INLINE;
+ } else {
+ /*sqe num is two */
+ for (i = 0; i < wr->num_sge; i++)
+ set_data_seg(dseg + i, wr->sg_list + i);
+
+ ctrl->flag |= cpu_to_le32(wr->num_sge <<
+ HNS_ROCE_WQE_SGE_NUM_BIT);
+ }
+ ind++;
+ } else {
+ dev_dbg(dev, "unSupported QP type\n");
+ break;
+ }
+ }
+
+out:
+ /* Set DB return */
+ if (likely(nreq)) {
+ qp->sq.head += nreq;
+ /* Memory barrier */
+ wmb();
+
+ sq_db.u32_4 = 0;
+ sq_db.u32_8 = 0;
+ roce_set_field(sq_db.u32_4, SQ_DOORBELL_U32_4_SQ_HEAD_M,
+ SQ_DOORBELL_U32_4_SQ_HEAD_S,
+ (qp->sq.head & ((qp->sq.wqe_cnt << 1) - 1)));
+ roce_set_field(sq_db.u32_4, SQ_DOORBELL_U32_4_PORT_M,
+ SQ_DOORBELL_U32_4_PORT_S, qp->port);
+ roce_set_field(sq_db.u32_8, SQ_DOORBELL_U32_8_QPN_M,
+ SQ_DOORBELL_U32_8_QPN_S, qp->doorbell_qpn);
+ roce_set_bit(sq_db.u32_8, SQ_DOORBELL_HW_SYNC_S, 1);
+
+ doorbell[0] = sq_db.u32_4;
+ doorbell[1] = sq_db.u32_8;
+
+ hns_roce_write64_k(doorbell, qp->sq.db_reg_l);
+ qp->sq_next_wqe = ind;
+ }
+
+ spin_unlock_irqrestore(&qp->sq.lock, flags);
+
+ return ret;
+}
+
+int hns_roce_v1_post_recv(struct ib_qp *ibqp, struct ib_recv_wr *wr,
+ struct ib_recv_wr **bad_wr)
+{
+ int ret = 0;
+ int nreq = 0;
+ int ind = 0;
+ int i = 0;
+ u32 reg_val = 0;
+ u32 *addr = NULL;
+ unsigned long flags = 0;
+ struct hns_roce_rq_wqe_ctrl *ctrl = NULL;
+ struct hns_roce_wqe_data_seg *scat = NULL;
+ struct hns_roce_qp *hr_qp = to_hr_qp(ibqp);
+ struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
+ struct device *dev = &hr_dev->pdev->dev;
+
+ spin_lock_irqsave(&hr_qp->rq.lock, flags);
+ ind = hr_qp->rq.head & (hr_qp->rq.wqe_cnt - 1);
+
+ for (nreq = 0; wr; ++nreq, wr = wr->next) {
+ if (hns_roce_wq_overflow(&hr_qp->rq, nreq,
+ hr_qp->ibqp.recv_cq)) {
+ dev_err(dev, "hns_roce_wq_overflow error\n");
+ ret = -ENOMEM;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ if (unlikely(wr->num_sge > hr_qp->rq.max_gs)) {
+ dev_err(dev, "rq:num_sge=%d > qp->sq.max_gs=%d\n",
+ wr->num_sge, hr_qp->rq.max_gs);
+ ret = -EINVAL;
+ *bad_wr = wr;
+ goto out;
+ }
+
+ ctrl = get_recv_wqe(hr_qp, ind);
+
+ roce_set_field(ctrl->rwqe_byte_12,
+ RQ_WQE_CTRL_RWQE_BYTE_12_RWQE_SGE_NUM_M,
+ RQ_WQE_CTRL_RWQE_BYTE_12_RWQE_SGE_NUM_S,
+ wr->num_sge);
+
+ scat = (struct hns_roce_wqe_data_seg *)(ctrl + 1);
+
+ for (i = 0; i < wr->num_sge; i++)
+ set_data_seg(scat + i, wr->sg_list + i);
+
+ hr_qp->rq.wrid[ind] = wr->wr_id;
+
+ ind = (ind + 1) & (hr_qp->rq.wqe_cnt - 1);
+ }
+
+out:
+ if (likely(nreq)) {
+ hr_qp->rq.head += nreq;
+ /* Memory barrier */
+ wmb();
+
+ if (ibqp->qp_type == IB_QPT_GSI) {
+ /* SW update GSI rq header */
+ addr = (u32 *)(to_hr_dev(ibqp->device)->reg_base +
+ ROCEE_QP1C_CFG3_0_REG +
+ QP1C_CFGN_OFFSET * hr_qp->port);
+ reg_val = roce_readl(addr);
+ roce_set_field(reg_val,
+ ROCEE_QP1C_CFG3_0_ROCEE_QP1C_RQ_HEAD_M,
+ ROCEE_QP1C_CFG3_0_ROCEE_QP1C_RQ_HEAD_S,
+ hr_qp->rq.head);
+ roce_writel(reg_val, addr);
+ } else {
+ uint32_t doorbell[2] = {0};
+ struct hns_roce_rq_db rq_db;
+
+ rq_db.u32_4 = 0;
+ rq_db.u32_8 = 0;
+
+ roce_set_field(rq_db.u32_4, RQ_DOORBELL_U32_4_RQ_HEAD_M,
+ RQ_DOORBELL_U32_4_RQ_HEAD_S,
+ hr_qp->rq.head);
+ roce_set_field(rq_db.u32_8, RQ_DOORBELL_U32_8_QPN_M,
+ RQ_DOORBELL_U32_8_QPN_S, hr_qp->qpn);
+ roce_set_field(rq_db.u32_8, RQ_DOORBELL_U32_8_CMD_M,
+ RQ_DOORBELL_U32_8_CMD_S, 1);
+ roce_set_bit(rq_db.u32_8, RQ_DOORBELL_U32_8_HW_SYNC_S,
+ 1);
+
+ doorbell[0] = rq_db.u32_4;
+ doorbell[1] = rq_db.u32_8;
+
+ hns_roce_write64_k(doorbell, hr_qp->rq.db_reg_l);
+ }
+ }
+ spin_unlock_irqrestore(&hr_qp->rq.lock, flags);
+
+ return ret;
+}
+
void hns_roce_set_db_event_mode(struct hns_roce_dev *hr_dev, int sdb_mode,
int odb_mode)
{
@@ -662,6 +1028,1277 @@ void hns_roce_v1_set_mtu(struct hns_roce_dev *hr_dev, u8 phy_port,
phy_port * PHY_PORT_OFFSET);
}
+static void *get_cqe(struct hns_roce_cq *hr_cq, int n)
+{
+ return hns_roce_buf_offset(&hr_cq->hr_buf.hr_buf,
+ n * HNS_ROCE_V1_CQE_ENTRY_SIZE);
+}
+
+static void *get_sw_cqe(struct hns_roce_cq *hr_cq, int n)
+{
+ struct hns_roce_cqe *hr_cqe = get_cqe(hr_cq, n & hr_cq->ib_cq.cqe);
+
+ /* Get cqe when Owner bit is Conversely with the MSB of cons_idx */
+ return (roce_get_bit(hr_cqe->cqe_byte_4, CQE_BYTE_4_OWNER_S) ^
+ !!(n & (hr_cq->ib_cq.cqe + 1))) ? hr_cqe : NULL;
+}
+
+void hns_roce_v1_cq_set_ci(struct hns_roce_cq *hr_cq, u32 cons_index,
+ spinlock_t *doorbell_lock)
+
+{
+ u32 doorbell[2];
+
+ doorbell[0] = cons_index & ((hr_cq->cq_depth << 1) - 1);
+ roce_set_bit(doorbell[1], ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_HW_SYNS_S, 1);
+ roce_set_field(doorbell[1], ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_CMD_M,
+ ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_CMD_S, 3);
+ roce_set_field(doorbell[1], ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_CMD_MDF_M,
+ ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_CMD_MDF_S, 0);
+ roce_set_field(doorbell[1], ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_INP_H_M,
+ ROCEE_DB_OTHERS_H_ROCEE_DB_OTH_INP_H_S, hr_cq->cqn);
+
+ hns_roce_write64_k(doorbell, hr_cq->cq_db_l);
+}
+
+static void hns_roce_v1_clean_cq(struct hns_roce_cq *hr_cq, u32 qpn,
+ struct hns_roce_srq *srq)
+{
+ struct hns_roce_cqe *cqe, *dest;
+ u32 prod_index;
+ int nfreed = 0;
+ u8 owner_bit;
+
+ for (prod_index = hr_cq->cons_index; get_sw_cqe(hr_cq, prod_index);
+ ++prod_index) {
+ if (prod_index == hr_cq->cons_index + hr_cq->ib_cq.cqe)
+ break;
+ }
+
+ /*
+ * Now backwards through the CQ, removing CQ entries
+ * that match our QP by overwriting them with next entries.
+ */
+ while ((int) --prod_index - (int) hr_cq->cons_index >= 0) {
+ cqe = get_cqe(hr_cq, prod_index & hr_cq->ib_cq.cqe);
+ if ((roce_get_field(cqe->cqe_byte_16, CQE_BYTE_16_LOCAL_QPN_M,
+ CQE_BYTE_16_LOCAL_QPN_S) &
+ HNS_ROCE_CQE_QPN_MASK) == qpn) {
+ /* In v1 engine, not support SRQ */
+ ++nfreed;
+ } else if (nfreed) {
+ dest = get_cqe(hr_cq, (prod_index + nfreed) &
+ hr_cq->ib_cq.cqe);
+ owner_bit = roce_get_bit(dest->cqe_byte_4,
+ CQE_BYTE_4_OWNER_S);
+ memcpy(dest, cqe, sizeof(*cqe));
+ roce_set_bit(dest->cqe_byte_4, CQE_BYTE_4_OWNER_S,
+ owner_bit);
+ }
+ }
+
+ if (nfreed) {
+ hr_cq->cons_index += nfreed;
+ /*
+ * Make sure update of buffer contents is done before
+ * updating consumer index.
+ */
+ wmb();
+
+ hns_roce_v1_cq_set_ci(hr_cq, hr_cq->cons_index,
+ &to_hr_dev(hr_cq->ib_cq.device)->cq_db_lock);
+ }
+}
+
+static void hns_roce_v1_cq_clean(struct hns_roce_cq *hr_cq, u32 qpn,
+ struct hns_roce_srq *srq)
+{
+ spin_lock_irq(&hr_cq->lock);
+ hns_roce_v1_clean_cq(hr_cq, qpn, srq);
+ spin_unlock_irq(&hr_cq->lock);
+}
+
+static int hns_roce_v1_qp_modify(struct hns_roce_dev *hr_dev,
+ struct hns_roce_mtt *mtt,
+ enum hns_roce_qp_state cur_state,
+ enum hns_roce_qp_state new_state,
+ struct hns_roce_qp_context *context,
+ struct hns_roce_qp *hr_qp)
+{
+ static const u16
+ op[HNS_ROCE_QP_NUM_STATE][HNS_ROCE_QP_NUM_STATE] = {
+ [HNS_ROCE_QP_STATE_RST] = {
+ [HNS_ROCE_QP_STATE_RST] = HNS_ROCE_CMD_2RST_QP,
+ [HNS_ROCE_QP_STATE_ERR] = HNS_ROCE_CMD_2ERR_QP,
+ [HNS_ROCE_QP_STATE_INIT] = HNS_ROCE_CMD_RST2INIT_QP,
+ },
+ [HNS_ROCE_QP_STATE_INIT] = {
+ [HNS_ROCE_QP_STATE_RST] = HNS_ROCE_CMD_2RST_QP,
+ [HNS_ROCE_QP_STATE_ERR] = HNS_ROCE_CMD_2ERR_QP,
+ /* Note: In v1 engine, HW doesn't support RST2INIT.
+ * We use RST2INIT cmd instead of INIT2INIT.
+ */
+ [HNS_ROCE_QP_STATE_INIT] = HNS_ROCE_CMD_RST2INIT_QP,
+ [HNS_ROCE_QP_STATE_RTR] = HNS_ROCE_CMD_INIT2RTR_QP,
+ },
+ [HNS_ROCE_QP_STATE_RTR] = {
+ [HNS_ROCE_QP_STATE_RST] = HNS_ROCE_CMD_2RST_QP,
+ [HNS_ROCE_QP_STATE_ERR] = HNS_ROCE_CMD_2ERR_QP,
+ [HNS_ROCE_QP_STATE_RTS] = HNS_ROCE_CMD_RTR2RTS_QP,
+ },
+ [HNS_ROCE_QP_STATE_RTS] = {
+ [HNS_ROCE_QP_STATE_RST] = HNS_ROCE_CMD_2RST_QP,
+ [HNS_ROCE_QP_STATE_ERR] = HNS_ROCE_CMD_2ERR_QP,
+ [HNS_ROCE_QP_STATE_RTS] = HNS_ROCE_CMD_RTS2RTS_QP,
+ [HNS_ROCE_QP_STATE_SQD] = HNS_ROCE_CMD_RTS2SQD_QP,
+ },
+ [HNS_ROCE_QP_STATE_SQD] = {
+ [HNS_ROCE_QP_STATE_RST] = HNS_ROCE_CMD_2RST_QP,
+ [HNS_ROCE_QP_STATE_ERR] = HNS_ROCE_CMD_2ERR_QP,
+ [HNS_ROCE_QP_STATE_RTS] = HNS_ROCE_CMD_SQD2RTS_QP,
+ [HNS_ROCE_QP_STATE_SQD] = HNS_ROCE_CMD_SQD2SQD_QP,
+ },
+ [HNS_ROCE_QP_STATE_ERR] = {
+ [HNS_ROCE_QP_STATE_RST] = HNS_ROCE_CMD_2RST_QP,
+ [HNS_ROCE_QP_STATE_ERR] = HNS_ROCE_CMD_2ERR_QP,
+ }
+ };
+
+ struct hns_roce_cmd_mailbox *mailbox;
+ struct device *dev = &hr_dev->pdev->dev;
+ int ret = 0;
+
+ if (cur_state >= HNS_ROCE_QP_NUM_STATE ||
+ new_state >= HNS_ROCE_QP_NUM_STATE ||
+ !op[cur_state][new_state]) {
+ dev_err(dev, "[modify_qp]not support state %d to %d\n",
+ cur_state, new_state);
+ return -EINVAL;
+ }
+
+ if (op[cur_state][new_state] == HNS_ROCE_CMD_2RST_QP)
+ return hns_roce_cmd(hr_dev, 0, hr_qp->qpn, 2,
+ HNS_ROCE_CMD_2RST_QP,
+ HNS_ROCE_CMD_TIME_CLASS_A);
+
+ if (op[cur_state][new_state] == HNS_ROCE_CMD_2ERR_QP)
+ return hns_roce_cmd(hr_dev, 0, hr_qp->qpn, 2,
+ HNS_ROCE_CMD_2ERR_QP,
+ HNS_ROCE_CMD_TIME_CLASS_A);
+
+ mailbox = hns_roce_alloc_cmd_mailbox(hr_dev);
+ if (IS_ERR(mailbox)) {
+ dev_err(dev, "[modify_qp]mailboc alloc failed!\n");
+ return PTR_ERR(mailbox);
+ }
+
+ memcpy(mailbox->buf, context, sizeof(*context));
+
+ ret = hns_roce_cmd(hr_dev, mailbox->dma, hr_qp->qpn, 0,
+ op[cur_state][new_state], HNS_ROCE_CMD_TIME_CLASS_C);
+
+ hns_roce_free_cmd_mailbox(hr_dev, mailbox);
+ return ret;
+}
+
+static int hns_roce_v1_m_sqp(struct ib_qp *ibqp, const struct ib_qp_attr *attr,
+ int attr_mask, enum ib_qp_state cur_state,
+ enum ib_qp_state new_state)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
+ struct hns_roce_qp *hr_qp = to_hr_qp(ibqp);
+ struct hns_roce_sqp_context *context;
+ struct device *dev = &hr_dev->pdev->dev;
+ dma_addr_t dma_handle = 0;
+ int rq_pa_start;
+ u32 reg_val;
+ u64 *mtts;
+ u32 *addr;
+
+ context = kzalloc(sizeof(*context), GFP_KERNEL);
+ if (!context)
+ return -ENOMEM;
+
+ /* Search QP buf's MTTs */
+ mtts = hns_roce_table_find(&hr_dev->mr_table.mtt_table,
+ hr_qp->mtt.first_seg, &dma_handle);
+ if (!mtts) {
+ dev_err(dev, "qp buf pa find failed\n");
+ goto out;
+ }
+
+ if (cur_state == IB_QPS_RESET && new_state == IB_QPS_INIT) {
+ roce_set_field(context->qp1c_bytes_4,
+ QP1C_BYTES_4_SQ_WQE_SHIFT_M,
+ QP1C_BYTES_4_SQ_WQE_SHIFT_S,
+ ilog2((unsigned int)hr_qp->sq.wqe_cnt));
+ roce_set_field(context->qp1c_bytes_4,
+ QP1C_BYTES_4_RQ_WQE_SHIFT_M,
+ QP1C_BYTES_4_RQ_WQE_SHIFT_S,
+ ilog2((unsigned int)hr_qp->rq.wqe_cnt));
+ roce_set_field(context->qp1c_bytes_4, QP1C_BYTES_4_PD_M,
+ QP1C_BYTES_4_PD_S, to_hr_pd(ibqp->pd)->pdn);
+
+ context->sq_rq_bt_l = (u32)(dma_handle);
+ roce_set_field(context->qp1c_bytes_12,
+ QP1C_BYTES_12_SQ_RQ_BT_H_M,
+ QP1C_BYTES_12_SQ_RQ_BT_H_S,
+ ((u32)(dma_handle >> ADDR_SHIFT_32)));
+
+ roce_set_field(context->qp1c_bytes_16, QP1C_BYTES_16_RQ_HEAD_M,
+ QP1C_BYTES_16_RQ_HEAD_S, hr_qp->rq.head);
+ roce_set_field(context->qp1c_bytes_16, QP1C_BYTES_16_PORT_NUM_M,
+ QP1C_BYTES_16_PORT_NUM_S, hr_qp->port);
+ roce_set_bit(context->qp1c_bytes_16,
+ QP1C_BYTES_16_SIGNALING_TYPE_S,
+ hr_qp->sq_signal_bits);
+ roce_set_bit(context->qp1c_bytes_16,
+ QP1C_BYTES_16_LOCAL_ENABLE_E2E_CREDIT_S,
+ hr_qp->sq_signal_bits);
+ roce_set_bit(context->qp1c_bytes_16, QP1C_BYTES_16_RQ_BA_FLG_S,
+ 1);
+ roce_set_bit(context->qp1c_bytes_16, QP1C_BYTES_16_SQ_BA_FLG_S,
+ 1);
+ roce_set_bit(context->qp1c_bytes_16, QP1C_BYTES_16_QP1_ERR_S,
+ 0);
+
+ roce_set_field(context->qp1c_bytes_20, QP1C_BYTES_20_SQ_HEAD_M,
+ QP1C_BYTES_20_SQ_HEAD_S, hr_qp->sq.head);
+ roce_set_field(context->qp1c_bytes_20, QP1C_BYTES_20_PKEY_IDX_M,
+ QP1C_BYTES_20_PKEY_IDX_S, attr->pkey_index);
+
+ rq_pa_start = (u32)hr_qp->rq.offset / PAGE_SIZE;
+ context->cur_rq_wqe_ba_l = (u32)(mtts[rq_pa_start]);
+
+ roce_set_field(context->qp1c_bytes_28,
+ QP1C_BYTES_28_CUR_RQ_WQE_BA_H_M,
+ QP1C_BYTES_28_CUR_RQ_WQE_BA_H_S,
+ (mtts[rq_pa_start]) >> ADDR_SHIFT_32);
+ roce_set_field(context->qp1c_bytes_28,
+ QP1C_BYTES_28_RQ_CUR_IDX_M,
+ QP1C_BYTES_28_RQ_CUR_IDX_S, 0);
+
+ roce_set_field(context->qp1c_bytes_32,
+ QP1C_BYTES_32_RX_CQ_NUM_M,
+ QP1C_BYTES_32_RX_CQ_NUM_S,
+ to_hr_cq(ibqp->recv_cq)->cqn);
+ roce_set_field(context->qp1c_bytes_32,
+ QP1C_BYTES_32_TX_CQ_NUM_M,
+ QP1C_BYTES_32_TX_CQ_NUM_S,
+ to_hr_cq(ibqp->send_cq)->cqn);
+
+ context->cur_sq_wqe_ba_l = (u32)mtts[0];
+
+ roce_set_field(context->qp1c_bytes_40,
+ QP1C_BYTES_40_CUR_SQ_WQE_BA_H_M,
+ QP1C_BYTES_40_CUR_SQ_WQE_BA_H_S,
+ (mtts[0]) >> ADDR_SHIFT_32);
+ roce_set_field(context->qp1c_bytes_40,
+ QP1C_BYTES_40_SQ_CUR_IDX_M,
+ QP1C_BYTES_40_SQ_CUR_IDX_S, 0);
+
+ /* Copy context to QP1C register */
+ addr = (u32 *)(hr_dev->reg_base + ROCEE_QP1C_CFG0_0_REG +
+ hr_qp->port * sizeof(*context));
+
+ roce_writel(context->qp1c_bytes_4, addr);
+ roce_writel(context->sq_rq_bt_l, addr + 1);
+ roce_writel(context->qp1c_bytes_12, addr + 2);
+ roce_writel(context->qp1c_bytes_16, addr + 3);
+ roce_writel(context->qp1c_bytes_20, addr + 4);
+ roce_writel(context->cur_rq_wqe_ba_l, addr + 5);
+ roce_writel(context->qp1c_bytes_28, addr + 6);
+ roce_writel(context->qp1c_bytes_32, addr + 7);
+ roce_writel(context->cur_sq_wqe_ba_l, addr + 8);
+ }
+
+ /* Modify QP1C status */
+ addr = (u32 *)(hr_dev->reg_base + ROCEE_QP1C_CFG0_0_REG +
+ hr_qp->port * sizeof(*context));
+ reg_val = roce_readl(addr);
+ roce_set_field(reg_val, ROCEE_QP1C_CFG0_0_ROCEE_QP1C_QP_ST_M,
+ ROCEE_QP1C_CFG0_0_ROCEE_QP1C_QP_ST_S, new_state);
+ roce_writel(reg_val, addr);
+
+ hr_qp->state = new_state;
+ if (new_state == IB_QPS_RESET) {
+ hns_roce_v1_cq_clean(to_hr_cq(ibqp->recv_cq), hr_qp->qpn,
+ ibqp->srq ? to_hr_srq(ibqp->srq) : NULL);
+ if (ibqp->send_cq != ibqp->recv_cq)
+ hns_roce_v1_cq_clean(to_hr_cq(ibqp->send_cq),
+ hr_qp->qpn, NULL);
+
+ hr_qp->rq.head = 0;
+ hr_qp->rq.tail = 0;
+ hr_qp->sq.head = 0;
+ hr_qp->sq.tail = 0;
+ hr_qp->sq_next_wqe = 0;
+ }
+
+ kfree(context);
+ return 0;
+
+out:
+ kfree(context);
+ return -EINVAL;
+}
+
+static int hns_roce_v1_m_qp(struct ib_qp *ibqp, const struct ib_qp_attr *attr,
+ int attr_mask, enum ib_qp_state cur_state,
+ enum ib_qp_state new_state)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
+ struct hns_roce_qp *hr_qp = to_hr_qp(ibqp);
+ struct device *dev = &hr_dev->pdev->dev;
+ struct hns_roce_qp_context *context;
+ dma_addr_t dma_handle_2 = 0;
+ dma_addr_t dma_handle = 0;
+ int rq_pa_start = 0;
+ u64 *mtts_2 = NULL;
+ int ret = -EINVAL;
+ u64 *mtts = NULL;
+ int port;
+ u8 *dmac;
+ u8 *smac;
+
+ context = kzalloc(sizeof(*context), GFP_KERNEL);
+ if (!context)
+ return -ENOMEM;
+
+ /* Search qp buf's mtts */
+ mtts = hns_roce_table_find(&hr_dev->mr_table.mtt_table,
+ hr_qp->mtt.first_seg, &dma_handle);
+ if (mtts == NULL) {
+ dev_err(dev, "qp buf pa find failed\n");
+ goto out;
+ }
+
+ /* Search IRRL's mtts */
+ mtts_2 = hns_roce_table_find(&hr_dev->qp_table.irrl_table, hr_qp->qpn,
+ &dma_handle_2);
+ if (mtts_2 == NULL) {
+ dev_err(dev, "qp irrl_table find failed\n");
+ goto out;
+ }
+
+ /*
+ *Reset to init
+ * Mandatory param:
+ * IB_QP_STATE | IB_QP_PKEY_INDEX | IB_QP_PORT | IB_QP_ACCESS_FLAGS
+ * Optional param: NA
+ */
+ if (cur_state == IB_QPS_RESET && new_state == IB_QPS_INIT) {
+ roce_set_field(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTES_4_TRANSPORT_SERVICE_TYPE_M,
+ QP_CONTEXT_QPC_BYTES_4_TRANSPORT_SERVICE_TYPE_S,
+ to_hr_qp_type(hr_qp->ibqp.qp_type));
+
+ roce_set_bit(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTE_4_ENABLE_FPMR_S, 0);
+ roce_set_bit(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTE_4_RDMA_READ_ENABLE_S,
+ !!(attr->qp_access_flags & IB_ACCESS_REMOTE_READ));
+ roce_set_bit(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTE_4_RDMA_WRITE_ENABLE_S,
+ !!(attr->qp_access_flags & IB_ACCESS_REMOTE_WRITE)
+ );
+ roce_set_bit(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTE_4_ATOMIC_OPERATION_ENABLE_S,
+ !!(attr->qp_access_flags & IB_ACCESS_REMOTE_ATOMIC)
+ );
+ roce_set_bit(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTE_4_RDMAR_USE_S, 1);
+ roce_set_field(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTES_4_SQ_WQE_SHIFT_M,
+ QP_CONTEXT_QPC_BYTES_4_SQ_WQE_SHIFT_S,
+ ilog2((unsigned int)hr_qp->sq.wqe_cnt));
+ roce_set_field(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTES_4_RQ_WQE_SHIFT_M,
+ QP_CONTEXT_QPC_BYTES_4_RQ_WQE_SHIFT_S,
+ ilog2((unsigned int)hr_qp->rq.wqe_cnt));
+ roce_set_field(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTES_4_PD_M,
+ QP_CONTEXT_QPC_BYTES_4_PD_S,
+ to_hr_pd(ibqp->pd)->pdn);
+ hr_qp->access_flags = attr->qp_access_flags;
+ roce_set_field(context->qpc_bytes_8,
+ QP_CONTEXT_QPC_BYTES_8_TX_COMPLETION_M,
+ QP_CONTEXT_QPC_BYTES_8_TX_COMPLETION_S,
+ to_hr_cq(ibqp->send_cq)->cqn);
+ roce_set_field(context->qpc_bytes_8,
+ QP_CONTEXT_QPC_BYTES_8_RX_COMPLETION_M,
+ QP_CONTEXT_QPC_BYTES_8_RX_COMPLETION_S,
+ to_hr_cq(ibqp->recv_cq)->cqn);
+
+ if (ibqp->srq)
+ roce_set_field(context->qpc_bytes_12,
+ QP_CONTEXT_QPC_BYTES_12_SRQ_NUMBER_M,
+ QP_CONTEXT_QPC_BYTES_12_SRQ_NUMBER_S,
+ to_hr_srq(ibqp->srq)->srqn);
+
+ roce_set_field(context->qpc_bytes_12,
+ QP_CONTEXT_QPC_BYTES_12_P_KEY_INDEX_M,
+ QP_CONTEXT_QPC_BYTES_12_P_KEY_INDEX_S,
+ attr->pkey_index);
+ hr_qp->pkey_index = attr->pkey_index;
+ roce_set_field(context->qpc_bytes_16,
+ QP_CONTEXT_QPC_BYTES_16_QP_NUM_M,
+ QP_CONTEXT_QPC_BYTES_16_QP_NUM_S, hr_qp->qpn);
+
+ } else if (cur_state == IB_QPS_INIT && new_state == IB_QPS_INIT) {
+ roce_set_field(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTES_4_TRANSPORT_SERVICE_TYPE_M,
+ QP_CONTEXT_QPC_BYTES_4_TRANSPORT_SERVICE_TYPE_S,
+ to_hr_qp_type(hr_qp->ibqp.qp_type));
+ roce_set_bit(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTE_4_ENABLE_FPMR_S, 0);
+ if (attr_mask & IB_QP_ACCESS_FLAGS) {
+ roce_set_bit(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTE_4_RDMA_READ_ENABLE_S,
+ !!(attr->qp_access_flags &
+ IB_ACCESS_REMOTE_READ));
+ roce_set_bit(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTE_4_RDMA_WRITE_ENABLE_S,
+ !!(attr->qp_access_flags &
+ IB_ACCESS_REMOTE_WRITE));
+ } else {
+ roce_set_bit(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTE_4_RDMA_READ_ENABLE_S,
+ !!(hr_qp->access_flags &
+ IB_ACCESS_REMOTE_READ));
+ roce_set_bit(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTE_4_RDMA_WRITE_ENABLE_S,
+ !!(hr_qp->access_flags &
+ IB_ACCESS_REMOTE_WRITE));
+ }
+
+ roce_set_bit(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTE_4_RDMAR_USE_S, 1);
+ roce_set_field(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTES_4_SQ_WQE_SHIFT_M,
+ QP_CONTEXT_QPC_BYTES_4_SQ_WQE_SHIFT_S,
+ ilog2((unsigned int)hr_qp->sq.wqe_cnt));
+ roce_set_field(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTES_4_RQ_WQE_SHIFT_M,
+ QP_CONTEXT_QPC_BYTES_4_RQ_WQE_SHIFT_S,
+ ilog2((unsigned int)hr_qp->rq.wqe_cnt));
+ roce_set_field(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTES_4_PD_M,
+ QP_CONTEXT_QPC_BYTES_4_PD_S,
+ to_hr_pd(ibqp->pd)->pdn);
+
+ roce_set_field(context->qpc_bytes_8,
+ QP_CONTEXT_QPC_BYTES_8_TX_COMPLETION_M,
+ QP_CONTEXT_QPC_BYTES_8_TX_COMPLETION_S,
+ to_hr_cq(ibqp->send_cq)->cqn);
+ roce_set_field(context->qpc_bytes_8,
+ QP_CONTEXT_QPC_BYTES_8_RX_COMPLETION_M,
+ QP_CONTEXT_QPC_BYTES_8_RX_COMPLETION_S,
+ to_hr_cq(ibqp->recv_cq)->cqn);
+
+ if (ibqp->srq)
+ roce_set_field(context->qpc_bytes_12,
+ QP_CONTEXT_QPC_BYTES_12_SRQ_NUMBER_M,
+ QP_CONTEXT_QPC_BYTES_12_SRQ_NUMBER_S,
+ to_hr_srq(ibqp->srq)->srqn);
+ if (attr_mask & IB_QP_PKEY_INDEX)
+ roce_set_field(context->qpc_bytes_12,
+ QP_CONTEXT_QPC_BYTES_12_P_KEY_INDEX_M,
+ QP_CONTEXT_QPC_BYTES_12_P_KEY_INDEX_S,
+ attr->pkey_index);
+ else
+ roce_set_field(context->qpc_bytes_12,
+ QP_CONTEXT_QPC_BYTES_12_P_KEY_INDEX_M,
+ QP_CONTEXT_QPC_BYTES_12_P_KEY_INDEX_S,
+ hr_qp->pkey_index);
+
+ roce_set_field(context->qpc_bytes_16,
+ QP_CONTEXT_QPC_BYTES_16_QP_NUM_M,
+ QP_CONTEXT_QPC_BYTES_16_QP_NUM_S, hr_qp->qpn);
+ } else if (cur_state == IB_QPS_INIT && new_state == IB_QPS_RTR) {
+ if ((attr_mask & IB_QP_ALT_PATH) ||
+ (attr_mask & IB_QP_ACCESS_FLAGS) ||
+ (attr_mask & IB_QP_PKEY_INDEX) ||
+ (attr_mask & IB_QP_QKEY)) {
+ dev_err(dev, "INIT2RTR attr_mask error\n");
+ goto out;
+ }
+
+ dmac = (u8 *)attr->ah_attr.dmac;
+
+ context->sq_rq_bt_l = (u32)(dma_handle);
+ roce_set_field(context->qpc_bytes_24,
+ QP_CONTEXT_QPC_BYTES_24_SQ_RQ_BT_H_M,
+ QP_CONTEXT_QPC_BYTES_24_SQ_RQ_BT_H_S,
+ ((u32)(dma_handle >> ADDR_SHIFT_32)));
+ roce_set_bit(context->qpc_bytes_24,
+ QP_CONTEXT_QPC_BYTE_24_REMOTE_ENABLE_E2E_CREDITS_S,
+ 1);
+ roce_set_field(context->qpc_bytes_24,
+ QP_CONTEXT_QPC_BYTES_24_MINIMUM_RNR_NAK_TIMER_M,
+ QP_CONTEXT_QPC_BYTES_24_MINIMUM_RNR_NAK_TIMER_S,
+ attr->min_rnr_timer);
+ context->irrl_ba_l = (u32)(dma_handle_2);
+ roce_set_field(context->qpc_bytes_32,
+ QP_CONTEXT_QPC_BYTES_32_IRRL_BA_H_M,
+ QP_CONTEXT_QPC_BYTES_32_IRRL_BA_H_S,
+ ((u32)(dma_handle_2 >> 32)) &
+ QP_CONTEXT_QPC_BYTES_32_IRRL_BA_H_M);
+ roce_set_field(context->qpc_bytes_32,
+ QP_CONTEXT_QPC_BYTES_32_MIG_STATE_M,
+ QP_CONTEXT_QPC_BYTES_32_MIG_STATE_S, 0);
+ roce_set_bit(context->qpc_bytes_32,
+ QP_CONTEXT_QPC_BYTE_32_LOCAL_ENABLE_E2E_CREDITS_S,
+ 1);
+ roce_set_bit(context->qpc_bytes_32,
+ QP_CONTEXT_QPC_BYTE_32_SIGNALING_TYPE_S,
+ hr_qp->sq_signal_bits);
+
+ for (port = 0; port < hr_dev->caps.num_ports; port++) {
+ smac = (u8 *)hr_dev->dev_addr[port];
+ dev_dbg(dev, "smac: %2x: %2x: %2x: %2x: %2x: %2x\n",
+ smac[0], smac[1], smac[2], smac[3], smac[4],
+ smac[5]);
+ if ((dmac[0] == smac[0]) && (dmac[1] == smac[1]) &&
+ (dmac[2] == smac[2]) && (dmac[3] == smac[3]) &&
+ (dmac[4] == smac[4]) && (dmac[5] == smac[5])) {
+ roce_set_bit(context->qpc_bytes_32,
+ QP_CONTEXT_QPC_BYTE_32_LOOPBACK_INDICATOR_S,
+ 1);
+ break;
+ }
+ }
+
+ if (hr_dev->loop_idc == 0x1)
+ roce_set_bit(context->qpc_bytes_32,
+ QP_CONTEXT_QPC_BYTE_32_LOOPBACK_INDICATOR_S, 1);
+
+ roce_set_bit(context->qpc_bytes_32,
+ QP_CONTEXT_QPC_BYTE_32_GLOBAL_HEADER_S,
+ attr->ah_attr.ah_flags);
+ roce_set_field(context->qpc_bytes_32,
+ QP_CONTEXT_QPC_BYTES_32_RESPONDER_RESOURCES_M,
+ QP_CONTEXT_QPC_BYTES_32_RESPONDER_RESOURCES_S,
+ ilog2((unsigned int)attr->max_dest_rd_atomic));
+
+ roce_set_field(context->qpc_bytes_36,
+ QP_CONTEXT_QPC_BYTES_36_DEST_QP_M,
+ QP_CONTEXT_QPC_BYTES_36_DEST_QP_S,
+ attr->dest_qp_num);
+
+ /* Configure GID index */
+ roce_set_field(context->qpc_bytes_36,
+ QP_CONTEXT_QPC_BYTES_36_SGID_INDEX_M,
+ QP_CONTEXT_QPC_BYTES_36_SGID_INDEX_S,
+ hns_get_gid_index(hr_dev,
+ attr->ah_attr.port_num - 1,
+ attr->ah_attr.grh.sgid_index));
+
+ memcpy(&(context->dmac_l), dmac, 4);
+
+ roce_set_field(context->qpc_bytes_44,
+ QP_CONTEXT_QPC_BYTES_44_DMAC_H_M,
+ QP_CONTEXT_QPC_BYTES_44_DMAC_H_S,
+ *((u16 *)(&dmac[4])));
+ roce_set_field(context->qpc_bytes_44,
+ QP_CONTEXT_QPC_BYTES_44_MAXIMUM_STATIC_RATE_M,
+ QP_CONTEXT_QPC_BYTES_44_MAXIMUM_STATIC_RATE_S,
+ attr->ah_attr.static_rate);
+ roce_set_field(context->qpc_bytes_44,
+ QP_CONTEXT_QPC_BYTES_44_HOPLMT_M,
+ QP_CONTEXT_QPC_BYTES_44_HOPLMT_S,
+ attr->ah_attr.grh.hop_limit);
+
+ roce_set_field(context->qpc_bytes_48,
+ QP_CONTEXT_QPC_BYTES_48_FLOWLABEL_M,
+ QP_CONTEXT_QPC_BYTES_48_FLOWLABEL_S,
+ attr->ah_attr.grh.flow_label);
+ roce_set_field(context->qpc_bytes_48,
+ QP_CONTEXT_QPC_BYTES_48_TCLASS_M,
+ QP_CONTEXT_QPC_BYTES_48_TCLASS_S,
+ attr->ah_attr.grh.traffic_class);
+ roce_set_field(context->qpc_bytes_48,
+ QP_CONTEXT_QPC_BYTES_48_MTU_M,
+ QP_CONTEXT_QPC_BYTES_48_MTU_S, attr->path_mtu);
+
+ memcpy(context->dgid, attr->ah_attr.grh.dgid.raw,
+ sizeof(attr->ah_attr.grh.dgid.raw));
+
+ dev_dbg(dev, "dmac:%x :%lx\n", context->dmac_l,
+ roce_get_field(context->qpc_bytes_44,
+ QP_CONTEXT_QPC_BYTES_44_DMAC_H_M,
+ QP_CONTEXT_QPC_BYTES_44_DMAC_H_S));
+
+ roce_set_field(context->qpc_bytes_68,
+ QP_CONTEXT_QPC_BYTES_68_RQ_HEAD_M,
+ QP_CONTEXT_QPC_BYTES_68_RQ_HEAD_S, 0);
+ roce_set_field(context->qpc_bytes_68,
+ QP_CONTEXT_QPC_BYTES_68_RQ_CUR_INDEX_M,
+ QP_CONTEXT_QPC_BYTES_68_RQ_CUR_INDEX_S, 0);
+
+ rq_pa_start = (u32)hr_qp->rq.offset / PAGE_SIZE;
+ context->cur_rq_wqe_ba_l = (u32)(mtts[rq_pa_start]);
+
+ roce_set_field(context->qpc_bytes_76,
+ QP_CONTEXT_QPC_BYTES_76_CUR_RQ_WQE_BA_H_M,
+ QP_CONTEXT_QPC_BYTES_76_CUR_RQ_WQE_BA_H_S,
+ mtts[rq_pa_start] >> ADDR_SHIFT_32);
+ roce_set_field(context->qpc_bytes_76,
+ QP_CONTEXT_QPC_BYTES_76_RX_REQ_MSN_M,
+ QP_CONTEXT_QPC_BYTES_76_RX_REQ_MSN_S, 0);
+
+ context->rx_rnr_time = 0;
+
+ roce_set_field(context->qpc_bytes_84,
+ QP_CONTEXT_QPC_BYTES_84_LAST_ACK_PSN_M,
+ QP_CONTEXT_QPC_BYTES_84_LAST_ACK_PSN_S,
+ attr->rq_psn - 1);
+ roce_set_field(context->qpc_bytes_84,
+ QP_CONTEXT_QPC_BYTES_84_TRRL_HEAD_M,
+ QP_CONTEXT_QPC_BYTES_84_TRRL_HEAD_S, 0);
+
+ roce_set_field(context->qpc_bytes_88,
+ QP_CONTEXT_QPC_BYTES_88_RX_REQ_EPSN_M,
+ QP_CONTEXT_QPC_BYTES_88_RX_REQ_EPSN_S,
+ attr->rq_psn);
+ roce_set_bit(context->qpc_bytes_88,
+ QP_CONTEXT_QPC_BYTES_88_RX_REQ_PSN_ERR_FLAG_S, 0);
+ roce_set_bit(context->qpc_bytes_88,
+ QP_CONTEXT_QPC_BYTES_88_RX_LAST_OPCODE_FLG_S, 0);
+ roce_set_field(context->qpc_bytes_88,
+ QP_CONTEXT_QPC_BYTES_88_RQ_REQ_LAST_OPERATION_TYPE_M,
+ QP_CONTEXT_QPC_BYTES_88_RQ_REQ_LAST_OPERATION_TYPE_S,
+ 0);
+ roce_set_field(context->qpc_bytes_88,
+ QP_CONTEXT_QPC_BYTES_88_RQ_REQ_RDMA_WR_FLAG_M,
+ QP_CONTEXT_QPC_BYTES_88_RQ_REQ_RDMA_WR_FLAG_S,
+ 0);
+
+ context->dma_length = 0;
+ context->r_key = 0;
+ context->va_l = 0;
+ context->va_h = 0;
+
+ roce_set_field(context->qpc_bytes_108,
+ QP_CONTEXT_QPC_BYTES_108_TRRL_SDB_PSN_M,
+ QP_CONTEXT_QPC_BYTES_108_TRRL_SDB_PSN_S, 0);
+ roce_set_bit(context->qpc_bytes_108,
+ QP_CONTEXT_QPC_BYTES_108_TRRL_SDB_PSN_FLG_S, 0);
+ roce_set_bit(context->qpc_bytes_108,
+ QP_CONTEXT_QPC_BYTES_108_TRRL_TDB_PSN_FLG_S, 0);
+
+ roce_set_field(context->qpc_bytes_112,
+ QP_CONTEXT_QPC_BYTES_112_TRRL_TDB_PSN_M,
+ QP_CONTEXT_QPC_BYTES_112_TRRL_TDB_PSN_S, 0);
+ roce_set_field(context->qpc_bytes_112,
+ QP_CONTEXT_QPC_BYTES_112_TRRL_TAIL_M,
+ QP_CONTEXT_QPC_BYTES_112_TRRL_TAIL_S, 0);
+
+ /* For chip resp ack */
+ roce_set_field(context->qpc_bytes_156,
+ QP_CONTEXT_QPC_BYTES_156_PORT_NUM_M,
+ QP_CONTEXT_QPC_BYTES_156_PORT_NUM_S,
+ hr_qp->port);
+ roce_set_field(context->qpc_bytes_156,
+ QP_CONTEXT_QPC_BYTES_156_SL_M,
+ QP_CONTEXT_QPC_BYTES_156_SL_S, attr->ah_attr.sl);
+ hr_qp->sl = attr->ah_attr.sl;
+ } else if (cur_state == IB_QPS_RTR &&
+ new_state == IB_QPS_RTS) {
+ /* If exist optional param, return error */
+ if ((attr_mask & IB_QP_ALT_PATH) ||
+ (attr_mask & IB_QP_ACCESS_FLAGS) ||
+ (attr_mask & IB_QP_QKEY) ||
+ (attr_mask & IB_QP_PATH_MIG_STATE) ||
+ (attr_mask & IB_QP_CUR_STATE) ||
+ (attr_mask & IB_QP_MIN_RNR_TIMER)) {
+ dev_err(dev, "RTR2RTS attr_mask error\n");
+ goto out;
+ }
+
+ context->rx_cur_sq_wqe_ba_l = (u32)(mtts[0]);
+
+ roce_set_field(context->qpc_bytes_120,
+ QP_CONTEXT_QPC_BYTES_120_RX_CUR_SQ_WQE_BA_H_M,
+ QP_CONTEXT_QPC_BYTES_120_RX_CUR_SQ_WQE_BA_H_S,
+ (mtts[0]) >> ADDR_SHIFT_32);
+
+ roce_set_field(context->qpc_bytes_124,
+ QP_CONTEXT_QPC_BYTES_124_RX_ACK_MSN_M,
+ QP_CONTEXT_QPC_BYTES_124_RX_ACK_MSN_S, 0);
+ roce_set_field(context->qpc_bytes_124,
+ QP_CONTEXT_QPC_BYTES_124_IRRL_MSG_IDX_M,
+ QP_CONTEXT_QPC_BYTES_124_IRRL_MSG_IDX_S, 0);
+
+ roce_set_field(context->qpc_bytes_128,
+ QP_CONTEXT_QPC_BYTES_128_RX_ACK_EPSN_M,
+ QP_CONTEXT_QPC_BYTES_128_RX_ACK_EPSN_S,
+ attr->sq_psn);
+ roce_set_bit(context->qpc_bytes_128,
+ QP_CONTEXT_QPC_BYTES_128_RX_ACK_PSN_ERR_FLG_S, 0);
+ roce_set_field(context->qpc_bytes_128,
+ QP_CONTEXT_QPC_BYTES_128_ACK_LAST_OPERATION_TYPE_M,
+ QP_CONTEXT_QPC_BYTES_128_ACK_LAST_OPERATION_TYPE_S,
+ 0);
+ roce_set_bit(context->qpc_bytes_128,
+ QP_CONTEXT_QPC_BYTES_128_IRRL_PSN_VLD_FLG_S, 0);
+
+ roce_set_field(context->qpc_bytes_132,
+ QP_CONTEXT_QPC_BYTES_132_IRRL_PSN_M,
+ QP_CONTEXT_QPC_BYTES_132_IRRL_PSN_S, 0);
+ roce_set_field(context->qpc_bytes_132,
+ QP_CONTEXT_QPC_BYTES_132_IRRL_TAIL_M,
+ QP_CONTEXT_QPC_BYTES_132_IRRL_TAIL_S, 0);
+
+ roce_set_field(context->qpc_bytes_136,
+ QP_CONTEXT_QPC_BYTES_136_RETRY_MSG_PSN_M,
+ QP_CONTEXT_QPC_BYTES_136_RETRY_MSG_PSN_S,
+ attr->sq_psn);
+ roce_set_field(context->qpc_bytes_136,
+ QP_CONTEXT_QPC_BYTES_136_RETRY_MSG_FPKT_PSN_L_M,
+ QP_CONTEXT_QPC_BYTES_136_RETRY_MSG_FPKT_PSN_L_S,
+ attr->sq_psn);
+
+ roce_set_field(context->qpc_bytes_140,
+ QP_CONTEXT_QPC_BYTES_140_RETRY_MSG_FPKT_PSN_H_M,
+ QP_CONTEXT_QPC_BYTES_140_RETRY_MSG_FPKT_PSN_H_S,
+ (attr->sq_psn >> SQ_PSN_SHIFT));
+ roce_set_field(context->qpc_bytes_140,
+ QP_CONTEXT_QPC_BYTES_140_RETRY_MSG_MSN_M,
+ QP_CONTEXT_QPC_BYTES_140_RETRY_MSG_MSN_S, 0);
+ roce_set_bit(context->qpc_bytes_140,
+ QP_CONTEXT_QPC_BYTES_140_RNR_RETRY_FLG_S, 0);
+
+ roce_set_field(context->qpc_bytes_144,
+ QP_CONTEXT_QPC_BYTES_144_QP_STATE_M,
+ QP_CONTEXT_QPC_BYTES_144_QP_STATE_S,
+ attr->qp_state);
+
+ roce_set_field(context->qpc_bytes_148,
+ QP_CONTEXT_QPC_BYTES_148_CHECK_FLAG_M,
+ QP_CONTEXT_QPC_BYTES_148_CHECK_FLAG_S, 0);
+ roce_set_field(context->qpc_bytes_148,
+ QP_CONTEXT_QPC_BYTES_148_RETRY_COUNT_M,
+ QP_CONTEXT_QPC_BYTES_148_RETRY_COUNT_S, 0);
+ roce_set_field(context->qpc_bytes_148,
+ QP_CONTEXT_QPC_BYTES_148_RNR_RETRY_COUNT_M,
+ QP_CONTEXT_QPC_BYTES_148_RNR_RETRY_COUNT_S, 0);
+ roce_set_field(context->qpc_bytes_148,
+ QP_CONTEXT_QPC_BYTES_148_LSN_M,
+ QP_CONTEXT_QPC_BYTES_148_LSN_S, 0x100);
+
+ context->rnr_retry = 0;
+
+ roce_set_field(context->qpc_bytes_156,
+ QP_CONTEXT_QPC_BYTES_156_RETRY_COUNT_INIT_M,
+ QP_CONTEXT_QPC_BYTES_156_RETRY_COUNT_INIT_S,
+ attr->retry_cnt);
+ roce_set_field(context->qpc_bytes_156,
+ QP_CONTEXT_QPC_BYTES_156_ACK_TIMEOUT_M,
+ QP_CONTEXT_QPC_BYTES_156_ACK_TIMEOUT_S,
+ attr->timeout);
+ roce_set_field(context->qpc_bytes_156,
+ QP_CONTEXT_QPC_BYTES_156_RNR_RETRY_COUNT_INIT_M,
+ QP_CONTEXT_QPC_BYTES_156_RNR_RETRY_COUNT_INIT_S,
+ attr->rnr_retry);
+ roce_set_field(context->qpc_bytes_156,
+ QP_CONTEXT_QPC_BYTES_156_PORT_NUM_M,
+ QP_CONTEXT_QPC_BYTES_156_PORT_NUM_S,
+ hr_qp->port);
+ roce_set_field(context->qpc_bytes_156,
+ QP_CONTEXT_QPC_BYTES_156_SL_M,
+ QP_CONTEXT_QPC_BYTES_156_SL_S, attr->ah_attr.sl);
+ hr_qp->sl = attr->ah_attr.sl;
+ roce_set_field(context->qpc_bytes_156,
+ QP_CONTEXT_QPC_BYTES_156_INITIATOR_DEPTH_M,
+ QP_CONTEXT_QPC_BYTES_156_INITIATOR_DEPTH_S,
+ ilog2((unsigned int)attr->max_rd_atomic));
+ roce_set_field(context->qpc_bytes_156,
+ QP_CONTEXT_QPC_BYTES_156_ACK_REQ_IND_M,
+ QP_CONTEXT_QPC_BYTES_156_ACK_REQ_IND_S, 0);
+ context->pkt_use_len = 0;
+
+ roce_set_field(context->qpc_bytes_164,
+ QP_CONTEXT_QPC_BYTES_164_SQ_PSN_M,
+ QP_CONTEXT_QPC_BYTES_164_SQ_PSN_S, attr->sq_psn);
+ roce_set_field(context->qpc_bytes_164,
+ QP_CONTEXT_QPC_BYTES_164_IRRL_HEAD_M,
+ QP_CONTEXT_QPC_BYTES_164_IRRL_HEAD_S, 0);
+
+ roce_set_field(context->qpc_bytes_168,
+ QP_CONTEXT_QPC_BYTES_168_RETRY_SQ_PSN_M,
+ QP_CONTEXT_QPC_BYTES_168_RETRY_SQ_PSN_S,
+ attr->sq_psn);
+ roce_set_field(context->qpc_bytes_168,
+ QP_CONTEXT_QPC_BYTES_168_SGE_USE_FLA_M,
+ QP_CONTEXT_QPC_BYTES_168_SGE_USE_FLA_S, 0);
+ roce_set_field(context->qpc_bytes_168,
+ QP_CONTEXT_QPC_BYTES_168_DB_TYPE_M,
+ QP_CONTEXT_QPC_BYTES_168_DB_TYPE_S, 0);
+ roce_set_bit(context->qpc_bytes_168,
+ QP_CONTEXT_QPC_BYTES_168_MSG_LP_IND_S, 0);
+ roce_set_bit(context->qpc_bytes_168,
+ QP_CONTEXT_QPC_BYTES_168_CSDB_LP_IND_S, 0);
+ roce_set_bit(context->qpc_bytes_168,
+ QP_CONTEXT_QPC_BYTES_168_QP_ERR_FLG_S, 0);
+ context->sge_use_len = 0;
+
+ roce_set_field(context->qpc_bytes_176,
+ QP_CONTEXT_QPC_BYTES_176_DB_CUR_INDEX_M,
+ QP_CONTEXT_QPC_BYTES_176_DB_CUR_INDEX_S, 0);
+ roce_set_field(context->qpc_bytes_176,
+ QP_CONTEXT_QPC_BYTES_176_RETRY_DB_CUR_INDEX_M,
+ QP_CONTEXT_QPC_BYTES_176_RETRY_DB_CUR_INDEX_S,
+ 0);
+ roce_set_field(context->qpc_bytes_180,
+ QP_CONTEXT_QPC_BYTES_180_SQ_CUR_INDEX_M,
+ QP_CONTEXT_QPC_BYTES_180_SQ_CUR_INDEX_S, 0);
+ roce_set_field(context->qpc_bytes_180,
+ QP_CONTEXT_QPC_BYTES_180_SQ_HEAD_M,
+ QP_CONTEXT_QPC_BYTES_180_SQ_HEAD_S, 0);
+
+ context->tx_cur_sq_wqe_ba_l = (u32)(mtts[0]);
+
+ roce_set_field(context->qpc_bytes_188,
+ QP_CONTEXT_QPC_BYTES_188_TX_CUR_SQ_WQE_BA_H_M,
+ QP_CONTEXT_QPC_BYTES_188_TX_CUR_SQ_WQE_BA_H_S,
+ (mtts[0]) >> ADDR_SHIFT_32);
+ roce_set_bit(context->qpc_bytes_188,
+ QP_CONTEXT_QPC_BYTES_188_PKT_RETRY_FLG_S, 0);
+ roce_set_field(context->qpc_bytes_188,
+ QP_CONTEXT_QPC_BYTES_188_TX_RETRY_CUR_INDEX_M,
+ QP_CONTEXT_QPC_BYTES_188_TX_RETRY_CUR_INDEX_S,
+ 0);
+ } else if ((cur_state == IB_QPS_INIT && new_state == IB_QPS_RESET) ||
+ (cur_state == IB_QPS_INIT && new_state == IB_QPS_ERR) ||
+ (cur_state == IB_QPS_RTR && new_state == IB_QPS_RESET) ||
+ (cur_state == IB_QPS_RTR && new_state == IB_QPS_ERR) ||
+ (cur_state == IB_QPS_RTS && new_state == IB_QPS_RESET) ||
+ (cur_state == IB_QPS_RTS && new_state == IB_QPS_ERR) ||
+ (cur_state == IB_QPS_ERR && new_state == IB_QPS_RESET) ||
+ (cur_state == IB_QPS_ERR && new_state == IB_QPS_ERR)) {
+ roce_set_field(context->qpc_bytes_144,
+ QP_CONTEXT_QPC_BYTES_144_QP_STATE_M,
+ QP_CONTEXT_QPC_BYTES_144_QP_STATE_S,
+ attr->qp_state);
+
+ } else {
+ dev_err(dev, "not support this modify\n");
+ goto out;
+ }
+
+ /* Every status migrate must change state */
+ roce_set_field(context->qpc_bytes_144,
+ QP_CONTEXT_QPC_BYTES_144_QP_STATE_M,
+ QP_CONTEXT_QPC_BYTES_144_QP_STATE_S, attr->qp_state);
+
+ /* SW pass context to HW */
+ ret = hns_roce_v1_qp_modify(hr_dev, &hr_qp->mtt,
+ to_hns_roce_state(cur_state),
+ to_hns_roce_state(new_state), context,
+ hr_qp);
+ if (ret) {
+ dev_err(dev, "hns_roce_qp_modify failed\n");
+ goto out;
+ }
+
+ /*
+ * Use rst2init to instead of init2init with drv,
+ * need to hw to flash RQ HEAD by DB again
+ */
+ if (cur_state == IB_QPS_INIT && new_state == IB_QPS_INIT) {
+ u32 reg_val = 0;
+ u32 *addr = NULL;
+
+ /* Memory barrier */
+ wmb();
+ if (hr_qp->ibqp.qp_type == IB_QPT_GSI) {
+ /* SW update GSI rq header */
+ addr = (u32 *)(hr_dev->reg_base +
+ ROCEE_QP1C_CFG3_0_REG +
+ QP1C_CFGN_OFFSET * hr_qp->port);
+ reg_val = roce_readl(addr);
+ roce_set_field(reg_val,
+ ROCEE_QP1C_CFG3_0_ROCEE_QP1C_RQ_HEAD_M,
+ ROCEE_QP1C_CFG3_0_ROCEE_QP1C_RQ_HEAD_S,
+ hr_qp->rq.head);
+ roce_writel(reg_val, addr);
+ } else {
+ uint32_t doorbell[2] = {0};
+ struct hns_roce_rq_db rq_db;
+
+ rq_db.u32_4 = 0;
+ rq_db.u32_8 = 0;
+
+ roce_set_field(rq_db.u32_4, RQ_DOORBELL_U32_4_RQ_HEAD_M,
+ RQ_DOORBELL_U32_4_RQ_HEAD_S,
+ hr_qp->rq.head);
+ roce_set_field(rq_db.u32_8, RQ_DOORBELL_U32_8_QPN_M,
+ RQ_DOORBELL_U32_8_QPN_S, hr_qp->qpn);
+ roce_set_field(rq_db.u32_8, RQ_DOORBELL_U32_8_CMD_M,
+ RQ_DOORBELL_U32_8_CMD_S, 1);
+ roce_set_bit(rq_db.u32_8, RQ_DOORBELL_U32_8_HW_SYNC_S,
+ 1);
+
+ doorbell[0] = rq_db.u32_4;
+ doorbell[1] = rq_db.u32_8;
+
+ hns_roce_write64_k(doorbell, hr_qp->rq.db_reg_l);
+ }
+ }
+
+ hr_qp->state = new_state;
+
+ if (attr_mask & IB_QP_MAX_DEST_RD_ATOMIC)
+ hr_qp->resp_depth = attr->max_dest_rd_atomic;
+ if (attr_mask & IB_QP_PORT)
+ hr_qp->port = (attr->port_num - 1);
+
+ if (new_state == IB_QPS_RESET && !ibqp->uobject) {
+ hns_roce_v1_cq_clean(to_hr_cq(ibqp->recv_cq), hr_qp->qpn,
+ ibqp->srq ? to_hr_srq(ibqp->srq) : NULL);
+ if (ibqp->send_cq != ibqp->recv_cq)
+ hns_roce_v1_cq_clean(to_hr_cq(ibqp->send_cq),
+ hr_qp->qpn, NULL);
+
+ hr_qp->rq.head = 0;
+ hr_qp->rq.tail = 0;
+ hr_qp->sq.head = 0;
+ hr_qp->sq.tail = 0;
+ hr_qp->sq_next_wqe = 0;
+ }
+out:
+ kfree(context);
+ return ret;
+}
+
+int hns_roce_v1_modify_qp(struct ib_qp *ibqp, const struct ib_qp_attr *attr,
+ int attr_mask, enum ib_qp_state cur_state,
+ enum ib_qp_state new_state)
+{
+
+ if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_SMI)
+ return hns_roce_v1_m_sqp(ibqp, attr, attr_mask, cur_state,
+ new_state);
+ else
+ return hns_roce_v1_m_qp(ibqp, attr, attr_mask, cur_state,
+ new_state);
+}
+
+static enum ib_qp_state to_ib_qp_state(enum hns_roce_qp_state state)
+{
+ switch (state) {
+ case HNS_ROCE_QP_STATE_RST:
+ return IB_QPS_RESET;
+ case HNS_ROCE_QP_STATE_INIT:
+ return IB_QPS_INIT;
+ case HNS_ROCE_QP_STATE_RTR:
+ return IB_QPS_RTR;
+ case HNS_ROCE_QP_STATE_RTS:
+ return IB_QPS_RTS;
+ case HNS_ROCE_QP_STATE_SQD:
+ return IB_QPS_SQD;
+ case HNS_ROCE_QP_STATE_ERR:
+ return IB_QPS_ERR;
+ default:
+ return IB_QPS_ERR;
+ }
+}
+
+static int hns_roce_v1_query_qpc(struct hns_roce_dev *hr_dev,
+ struct hns_roce_qp *hr_qp,
+ struct hns_roce_qp_context *hr_context)
+{
+ struct hns_roce_cmd_mailbox *mailbox;
+ int ret;
+
+ mailbox = hns_roce_alloc_cmd_mailbox(hr_dev);
+ if (IS_ERR(mailbox)) {
+ dev_err(&hr_dev->pdev->dev, "Alloc mailbox failed\n");
+ return PTR_ERR(mailbox);
+ }
+
+ ret = hns_roce_cmd_box(hr_dev, 0, mailbox->dma, hr_qp->qpn, 0,
+ HNS_ROCE_CMD_QUERY_QP,
+ HNS_ROCE_CMD_TIME_CLASS_A);
+ if (!ret)
+ memcpy(hr_context, mailbox->buf, sizeof(*hr_context));
+ else
+ dev_err(&hr_dev->pdev->dev, "QUERY QP cmd process error\n");
+
+ hns_roce_free_cmd_mailbox(hr_dev, mailbox);
+
+ return ret;
+}
+
+int hns_roce_v1_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr,
+ int qp_attr_mask, struct ib_qp_init_attr *qp_init_attr)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
+ struct hns_roce_qp *hr_qp = to_hr_qp(ibqp);
+ struct device *dev = &hr_dev->pdev->dev;
+ struct hns_roce_qp_context *context;
+ int tmp_qp_state = 0;
+ int ret = 0;
+ int state;
+
+ context = kzalloc(sizeof(*context), GFP_KERNEL);
+ if (!context)
+ return -ENOMEM;
+
+ memset(qp_attr, 0, sizeof(*qp_attr));
+ memset(qp_init_attr, 0, sizeof(*qp_init_attr));
+
+ mutex_lock(&hr_qp->mutex);
+
+ if (hr_qp->state == IB_QPS_RESET) {
+ qp_attr->qp_state = IB_QPS_RESET;
+ goto done;
+ }
+
+ ret = hns_roce_v1_query_qpc(hr_dev, hr_qp, context);
+ if (ret) {
+ dev_err(dev, "query qpc error\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
+ state = roce_get_field(context->qpc_bytes_144,
+ QP_CONTEXT_QPC_BYTES_144_QP_STATE_M,
+ QP_CONTEXT_QPC_BYTES_144_QP_STATE_S);
+ tmp_qp_state = (int)to_ib_qp_state((enum hns_roce_qp_state)state);
+ if (tmp_qp_state == -1) {
+ dev_err(dev, "to_ib_qp_state error\n");
+ ret = -EINVAL;
+ goto out;
+ }
+ hr_qp->state = (u8)tmp_qp_state;
+ qp_attr->qp_state = (enum ib_qp_state)hr_qp->state;
+ qp_attr->path_mtu = (enum ib_mtu)roce_get_field(context->qpc_bytes_48,
+ QP_CONTEXT_QPC_BYTES_48_MTU_M,
+ QP_CONTEXT_QPC_BYTES_48_MTU_S);
+ qp_attr->path_mig_state = IB_MIG_ARMED;
+ if (hr_qp->ibqp.qp_type == IB_QPT_UD)
+ qp_attr->qkey = QKEY_VAL;
+
+ qp_attr->rq_psn = roce_get_field(context->qpc_bytes_88,
+ QP_CONTEXT_QPC_BYTES_88_RX_REQ_EPSN_M,
+ QP_CONTEXT_QPC_BYTES_88_RX_REQ_EPSN_S);
+ qp_attr->sq_psn = (u32)roce_get_field(context->qpc_bytes_164,
+ QP_CONTEXT_QPC_BYTES_164_SQ_PSN_M,
+ QP_CONTEXT_QPC_BYTES_164_SQ_PSN_S);
+ qp_attr->dest_qp_num = (u8)roce_get_field(context->qpc_bytes_36,
+ QP_CONTEXT_QPC_BYTES_36_DEST_QP_M,
+ QP_CONTEXT_QPC_BYTES_36_DEST_QP_S);
+ qp_attr->qp_access_flags = ((roce_get_bit(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTE_4_RDMA_READ_ENABLE_S)) << 2) |
+ ((roce_get_bit(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTE_4_RDMA_WRITE_ENABLE_S)) << 1) |
+ ((roce_get_bit(context->qpc_bytes_4,
+ QP_CONTEXT_QPC_BYTE_4_ATOMIC_OPERATION_ENABLE_S)) << 3);
+
+ if (hr_qp->ibqp.qp_type == IB_QPT_RC ||
+ hr_qp->ibqp.qp_type == IB_QPT_UC) {
+ qp_attr->ah_attr.sl = roce_get_field(context->qpc_bytes_156,
+ QP_CONTEXT_QPC_BYTES_156_SL_M,
+ QP_CONTEXT_QPC_BYTES_156_SL_S);
+ qp_attr->ah_attr.grh.flow_label = roce_get_field(
+ context->qpc_bytes_48,
+ QP_CONTEXT_QPC_BYTES_48_FLOWLABEL_M,
+ QP_CONTEXT_QPC_BYTES_48_FLOWLABEL_S);
+ qp_attr->ah_attr.grh.sgid_index = roce_get_field(
+ context->qpc_bytes_36,
+ QP_CONTEXT_QPC_BYTES_36_SGID_INDEX_M,
+ QP_CONTEXT_QPC_BYTES_36_SGID_INDEX_S);
+ qp_attr->ah_attr.grh.hop_limit = roce_get_field(
+ context->qpc_bytes_44,
+ QP_CONTEXT_QPC_BYTES_44_HOPLMT_M,
+ QP_CONTEXT_QPC_BYTES_44_HOPLMT_S);
+ qp_attr->ah_attr.grh.traffic_class = roce_get_field(
+ context->qpc_bytes_48,
+ QP_CONTEXT_QPC_BYTES_48_TCLASS_M,
+ QP_CONTEXT_QPC_BYTES_48_TCLASS_S);
+
+ memcpy(qp_attr->ah_attr.grh.dgid.raw, context->dgid,
+ sizeof(qp_attr->ah_attr.grh.dgid.raw));
+ }
+
+ qp_attr->pkey_index = roce_get_field(context->qpc_bytes_12,
+ QP_CONTEXT_QPC_BYTES_12_P_KEY_INDEX_M,
+ QP_CONTEXT_QPC_BYTES_12_P_KEY_INDEX_S);
+ qp_attr->port_num = (u8)roce_get_field(context->qpc_bytes_156,
+ QP_CONTEXT_QPC_BYTES_156_PORT_NUM_M,
+ QP_CONTEXT_QPC_BYTES_156_PORT_NUM_S) + 1;
+ qp_attr->sq_draining = 0;
+ qp_attr->max_rd_atomic = roce_get_field(context->qpc_bytes_156,
+ QP_CONTEXT_QPC_BYTES_156_INITIATOR_DEPTH_M,
+ QP_CONTEXT_QPC_BYTES_156_INITIATOR_DEPTH_S);
+ qp_attr->max_dest_rd_atomic = roce_get_field(context->qpc_bytes_32,
+ QP_CONTEXT_QPC_BYTES_32_RESPONDER_RESOURCES_M,
+ QP_CONTEXT_QPC_BYTES_32_RESPONDER_RESOURCES_S);
+ qp_attr->min_rnr_timer = (u8)(roce_get_field(context->qpc_bytes_24,
+ QP_CONTEXT_QPC_BYTES_24_MINIMUM_RNR_NAK_TIMER_M,
+ QP_CONTEXT_QPC_BYTES_24_MINIMUM_RNR_NAK_TIMER_S));
+ qp_attr->timeout = (u8)(roce_get_field(context->qpc_bytes_156,
+ QP_CONTEXT_QPC_BYTES_156_ACK_TIMEOUT_M,
+ QP_CONTEXT_QPC_BYTES_156_ACK_TIMEOUT_S));
+ qp_attr->retry_cnt = roce_get_field(context->qpc_bytes_148,
+ QP_CONTEXT_QPC_BYTES_148_RETRY_COUNT_M,
+ QP_CONTEXT_QPC_BYTES_148_RETRY_COUNT_S);
+ qp_attr->rnr_retry = context->rnr_retry;
+
+done:
+ qp_attr->cur_qp_state = qp_attr->qp_state;
+ qp_attr->cap.max_recv_wr = hr_qp->rq.wqe_cnt;
+ qp_attr->cap.max_recv_sge = hr_qp->rq.max_gs;
+
+ if (!ibqp->uobject) {
+ qp_attr->cap.max_send_wr = hr_qp->sq.wqe_cnt;
+ qp_attr->cap.max_send_sge = hr_qp->sq.max_gs;
+ } else {
+ qp_attr->cap.max_send_wr = 0;
+ qp_attr->cap.max_send_sge = 0;
+ }
+
+ qp_init_attr->cap = qp_attr->cap;
+
+out:
+ mutex_unlock(&hr_qp->mutex);
+ kfree(context);
+ return ret;
+}
+
+static void hns_roce_v1_destroy_qp_common(struct hns_roce_dev *hr_dev,
+ struct hns_roce_qp *hr_qp,
+ int is_user)
+{
+ u32 sdbinvcnt;
+ unsigned long end = 0;
+ u32 sdbinvcnt_val;
+ u32 sdbsendptr_val;
+ u32 sdbisusepr_val;
+ struct hns_roce_cq *send_cq, *recv_cq;
+ struct device *dev = &hr_dev->pdev->dev;
+
+ if (hr_qp->ibqp.qp_type == IB_QPT_RC) {
+ if (hr_qp->state != IB_QPS_RESET) {
+ /*
+ * Set qp to ERR,
+ * waiting for hw complete processing all dbs
+ */
+ if (hns_roce_v1_qp_modify(hr_dev, NULL,
+ to_hns_roce_state(
+ (enum ib_qp_state)hr_qp->state),
+ HNS_ROCE_QP_STATE_ERR, NULL,
+ hr_qp))
+ dev_err(dev, "modify QP %06lx to ERR failed.\n",
+ hr_qp->qpn);
+
+ /* Record issued doorbell */
+ sdbisusepr_val = roce_readl(hr_dev->reg_base +
+ ROCEE_SDB_ISSUE_PTR_REG);
+ /*
+ * Query db process status,
+ * until hw process completely
+ */
+ end = msecs_to_jiffies(
+ HNS_ROCE_QP_DESTROY_TIMEOUT_MSECS) + jiffies;
+ do {
+ sdbsendptr_val = roce_readl(hr_dev->reg_base +
+ ROCEE_SDB_SEND_PTR_REG);
+ if (!time_before(jiffies, end)) {
+ dev_err(dev, "destroy qp(0x%lx) timeout!!!",
+ hr_qp->qpn);
+ break;
+ }
+ } while ((short)(roce_get_field(sdbsendptr_val,
+ ROCEE_SDB_SEND_PTR_SDB_SEND_PTR_M,
+ ROCEE_SDB_SEND_PTR_SDB_SEND_PTR_S) -
+ roce_get_field(sdbisusepr_val,
+ ROCEE_SDB_ISSUE_PTR_SDB_ISSUE_PTR_M,
+ ROCEE_SDB_ISSUE_PTR_SDB_ISSUE_PTR_S)
+ ) < 0);
+
+ /* Get list pointer */
+ sdbinvcnt = roce_readl(hr_dev->reg_base +
+ ROCEE_SDB_INV_CNT_REG);
+
+ /* Query db's list status, until hw reversal */
+ do {
+ sdbinvcnt_val = roce_readl(hr_dev->reg_base +
+ ROCEE_SDB_INV_CNT_REG);
+ if (!time_before(jiffies, end)) {
+ dev_err(dev, "destroy qp(0x%lx) timeout!!!",
+ hr_qp->qpn);
+ dev_err(dev, "SdbInvCnt = 0x%x\n",
+ sdbinvcnt_val);
+ break;
+ }
+ } while ((short)(roce_get_field(sdbinvcnt_val,
+ ROCEE_SDB_INV_CNT_SDB_INV_CNT_M,
+ ROCEE_SDB_INV_CNT_SDB_INV_CNT_S) -
+ (sdbinvcnt + SDB_INV_CNT_OFFSET)) < 0);
+
+ /* Modify qp to reset before destroying qp */
+ if (hns_roce_v1_qp_modify(hr_dev, NULL,
+ to_hns_roce_state(
+ (enum ib_qp_state)hr_qp->state),
+ HNS_ROCE_QP_STATE_RST, NULL, hr_qp))
+ dev_err(dev, "modify QP %06lx to RESET failed.\n",
+ hr_qp->qpn);
+ }
+ }
+
+ send_cq = to_hr_cq(hr_qp->ibqp.send_cq);
+ recv_cq = to_hr_cq(hr_qp->ibqp.recv_cq);
+
+ hns_roce_lock_cqs(send_cq, recv_cq);
+
+ if (!is_user) {
+ hns_roce_v1_clean_cq(recv_cq, hr_qp->qpn, hr_qp->ibqp.srq ?
+ to_hr_srq(hr_qp->ibqp.srq) : NULL);
+ if (send_cq != recv_cq)
+ hns_roce_v1_clean_cq(send_cq, hr_qp->qpn, NULL);
+ }
+
+ hns_roce_qp_remove(hr_dev, hr_qp);
+
+ hns_roce_unlock_cqs(send_cq, recv_cq);
+
+ hns_roce_qp_free(hr_dev, hr_qp);
+
+ /* Not special_QP, free their QPN */
+ if ((hr_qp->ibqp.qp_type == IB_QPT_RC) ||
+ (hr_qp->ibqp.qp_type == IB_QPT_UC) ||
+ (hr_qp->ibqp.qp_type == IB_QPT_UD))
+ hns_roce_release_range_qp(hr_dev, hr_qp->qpn, 1);
+
+ hns_roce_mtt_cleanup(hr_dev, &hr_qp->mtt);
+
+ if (is_user) {
+ ib_umem_release(hr_qp->umem);
+ } else {
+ kfree(hr_qp->sq.wrid);
+ kfree(hr_qp->rq.wrid);
+ hns_roce_buf_free(hr_dev, hr_qp->buff_size, &hr_qp->hr_buf);
+ }
+}
+
+int hns_roce_v1_destroy_qp(struct ib_qp *ibqp)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
+ struct hns_roce_qp *hr_qp = to_hr_qp(ibqp);
+
+ hns_roce_v1_destroy_qp_common(hr_dev, hr_qp, !!ibqp->pd->uobject);
+
+ if (hr_qp->ibqp.qp_type == IB_QPT_GSI)
+ kfree(hr_to_hr_sqp(hr_qp));
+ else
+ kfree(hr_qp);
+
+ return 0;
+}
+
struct hns_roce_hw hns_roce_hw_v1 = {
.reset = hns_roce_v1_reset,
.hw_profile = hns_roce_v1_profile,
@@ -670,4 +2307,9 @@ struct hns_roce_hw hns_roce_hw_v1 = {
.set_gid = hns_roce_v1_set_gid,
.set_mac = hns_roce_v1_set_mac,
.set_mtu = hns_roce_v1_set_mtu,
+ .modify_qp = hns_roce_v1_modify_qp,
+ .query_qp = hns_roce_v1_query_qp,
+ .destroy_qp = hns_roce_v1_destroy_qp,
+ .post_send = hns_roce_v1_post_send,
+ .post_recv = hns_roce_v1_post_recv,
};
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.h b/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
index 143ef0b..917985c 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
@@ -84,10 +84,636 @@
#define HNS_ROCE_ODB_EXTEND_MODE 1
+#define HNS_ROCE_CQE_QPN_MASK 0x3ffff
+
+#define QP1C_CFGN_OFFSET 0x28
#define PHY_PORT_OFFSET 0x8
#define ALL_PORT_VAL_OPEN 0x3f
#define POL_TIME_INTERVAL_VAL 0x80
#define SLEEP_TIME_INTERVAL 20
+#define SQ_PSN_SHIFT 8
+#define QKEY_VAL 0x80010000
+#define SDB_INV_CNT_OFFSET 8
+
+struct hns_roce_cqe {
+ u32 cqe_byte_4;
+ u32 cqe_byte_16;
+};
+
+#define CQE_BYTE_4_OWNER_S 7
+#define CQE_BYTE_4_SQ_RQ_FLAG_S 14
+
+#define CQE_BYTE_16_LOCAL_QPN_S 0
+#define CQE_BYTE_16_LOCAL_QPN_M (((1UL << 24) - 1) << CQE_BYTE_16_LOCAL_QPN_S)
+
+struct hns_roce_wqe_ctrl_seg {
+ __be32 sgl_pa_h;
+ __be32 flag;
+ __be32 imm_data;
+ __be32 msg_length;
+};
+
+struct hns_roce_wqe_data_seg {
+ __be64 addr;
+ __be32 lkey;
+ __be32 len;
+};
+
+struct hns_roce_wqe_raddr_seg {
+ __be32 rkey;
+ __be32 len;/* reserved */
+ __be64 raddr;
+};
+
+struct hns_roce_rq_wqe_ctrl {
+
+ u32 rwqe_byte_4;
+ u32 rocee_sgl_ba_l;
+ u32 rwqe_byte_12;
+ u32 reserved[5];
+};
+
+#define RQ_WQE_CTRL_RWQE_BYTE_12_RWQE_SGE_NUM_S 16
+#define RQ_WQE_CTRL_RWQE_BYTE_12_RWQE_SGE_NUM_M \
+ (((1UL << 6) - 1) << RQ_WQE_CTRL_RWQE_BYTE_12_RWQE_SGE_NUM_S)
+
+#define HNS_ROCE_QP_DESTROY_TIMEOUT_MSECS 10000
+
+#define GID_LEN 16
+
+struct hns_roce_ud_send_wqe {
+ u32 dmac_h;
+ u32 u32_8;
+ u32 immediate_data;
+
+ u32 u32_16;
+ union {
+ unsigned char dgid[GID_LEN];
+ struct {
+ u32 u32_20;
+ u32 u32_24;
+ u32 u32_28;
+ u32 u32_32;
+ };
+ };
+
+ u32 u32_36;
+ u32 u32_40;
+
+ u32 va0_l;
+ u32 va0_h;
+ u32 l_key0;
+
+ u32 va1_l;
+ u32 va1_h;
+ u32 l_key1;
+};
+
+#define UD_SEND_WQE_U32_4_DMAC_0_S 0
+#define UD_SEND_WQE_U32_4_DMAC_0_M \
+ (((1UL << 8) - 1) << UD_SEND_WQE_U32_4_DMAC_0_S)
+
+#define UD_SEND_WQE_U32_4_DMAC_1_S 8
+#define UD_SEND_WQE_U32_4_DMAC_1_M \
+ (((1UL << 8) - 1) << UD_SEND_WQE_U32_4_DMAC_1_S)
+
+#define UD_SEND_WQE_U32_4_DMAC_2_S 16
+#define UD_SEND_WQE_U32_4_DMAC_2_M \
+ (((1UL << 8) - 1) << UD_SEND_WQE_U32_4_DMAC_2_S)
+
+#define UD_SEND_WQE_U32_4_DMAC_3_S 24
+#define UD_SEND_WQE_U32_4_DMAC_3_M \
+ (((1UL << 8) - 1) << UD_SEND_WQE_U32_4_DMAC_3_S)
+
+#define UD_SEND_WQE_U32_8_DMAC_4_S 0
+#define UD_SEND_WQE_U32_8_DMAC_4_M \
+ (((1UL << 8) - 1) << UD_SEND_WQE_U32_8_DMAC_4_S)
+
+#define UD_SEND_WQE_U32_8_DMAC_5_S 8
+#define UD_SEND_WQE_U32_8_DMAC_5_M \
+ (((1UL << 8) - 1) << UD_SEND_WQE_U32_8_DMAC_5_S)
+
+#define UD_SEND_WQE_U32_8_OPERATION_TYPE_S 16
+#define UD_SEND_WQE_U32_8_OPERATION_TYPE_M \
+ (((1UL << 4) - 1) << UD_SEND_WQE_U32_8_OPERATION_TYPE_S)
+
+#define UD_SEND_WQE_U32_8_NUMBER_OF_DATA_SEG_S 24
+#define UD_SEND_WQE_U32_8_NUMBER_OF_DATA_SEG_M \
+ (((1UL << 6) - 1) << UD_SEND_WQE_U32_8_NUMBER_OF_DATA_SEG_S)
+
+#define UD_SEND_WQE_U32_8_SEND_GL_ROUTING_HDR_FLAG_S 31
+
+#define UD_SEND_WQE_U32_16_DEST_QP_S 0
+#define UD_SEND_WQE_U32_16_DEST_QP_M \
+ (((1UL << 24) - 1) << UD_SEND_WQE_U32_16_DEST_QP_S)
+
+#define UD_SEND_WQE_U32_16_MAX_STATIC_RATE_S 24
+#define UD_SEND_WQE_U32_16_MAX_STATIC_RATE_M \
+ (((1UL << 8) - 1) << UD_SEND_WQE_U32_16_MAX_STATIC_RATE_S)
+
+#define UD_SEND_WQE_U32_36_FLOW_LABEL_S 0
+#define UD_SEND_WQE_U32_36_FLOW_LABEL_M \
+ (((1UL << 20) - 1) << UD_SEND_WQE_U32_36_FLOW_LABEL_S)
+
+#define UD_SEND_WQE_U32_36_PRIORITY_S 20
+#define UD_SEND_WQE_U32_36_PRIORITY_M \
+ (((1UL << 4) - 1) << UD_SEND_WQE_U32_36_PRIORITY_S)
+
+#define UD_SEND_WQE_U32_36_SGID_INDEX_S 24
+#define UD_SEND_WQE_U32_36_SGID_INDEX_M \
+ (((1UL << 8) - 1) << UD_SEND_WQE_U32_36_SGID_INDEX_S)
+
+#define UD_SEND_WQE_U32_40_HOP_LIMIT_S 0
+#define UD_SEND_WQE_U32_40_HOP_LIMIT_M \
+ (((1UL << 8) - 1) << UD_SEND_WQE_U32_40_HOP_LIMIT_S)
+
+#define UD_SEND_WQE_U32_40_TRAFFIC_CLASS_S 8
+#define UD_SEND_WQE_U32_40_TRAFFIC_CLASS_M \
+ (((1UL << 8) - 1) << UD_SEND_WQE_U32_40_TRAFFIC_CLASS_S)
+
+struct hns_roce_sqp_context {
+ u32 qp1c_bytes_4;
+ u32 sq_rq_bt_l;
+ u32 qp1c_bytes_12;
+ u32 qp1c_bytes_16;
+ u32 qp1c_bytes_20;
+ u32 qp1c_bytes_28;
+ u32 cur_rq_wqe_ba_l;
+ u32 qp1c_bytes_32;
+ u32 cur_sq_wqe_ba_l;
+ u32 qp1c_bytes_40;
+};
+
+#define QP1C_BYTES_4_SQ_WQE_SHIFT_S 8
+#define QP1C_BYTES_4_SQ_WQE_SHIFT_M \
+ (((1UL << 4) - 1) << QP1C_BYTES_4_SQ_WQE_SHIFT_S)
+
+#define QP1C_BYTES_4_RQ_WQE_SHIFT_S 12
+#define QP1C_BYTES_4_RQ_WQE_SHIFT_M \
+ (((1UL << 4) - 1) << QP1C_BYTES_4_RQ_WQE_SHIFT_S)
+
+#define QP1C_BYTES_4_PD_S 16
+#define QP1C_BYTES_4_PD_M (((1UL << 16) - 1) << QP1C_BYTES_4_PD_S)
+
+#define QP1C_BYTES_12_SQ_RQ_BT_H_S 0
+#define QP1C_BYTES_12_SQ_RQ_BT_H_M \
+ (((1UL << 17) - 1) << QP1C_BYTES_12_SQ_RQ_BT_H_S)
+
+#define QP1C_BYTES_16_RQ_HEAD_S 0
+#define QP1C_BYTES_16_RQ_HEAD_M (((1UL << 15) - 1) << QP1C_BYTES_16_RQ_HEAD_S)
+
+#define QP1C_BYTES_16_PORT_NUM_S 16
+#define QP1C_BYTES_16_PORT_NUM_M \
+ (((1UL << 3) - 1) << QP1C_BYTES_16_PORT_NUM_S)
+
+#define QP1C_BYTES_16_SIGNALING_TYPE_S 27
+#define QP1C_BYTES_16_LOCAL_ENABLE_E2E_CREDIT_S 28
+#define QP1C_BYTES_16_RQ_BA_FLG_S 29
+#define QP1C_BYTES_16_SQ_BA_FLG_S 30
+#define QP1C_BYTES_16_QP1_ERR_S 31
+
+#define QP1C_BYTES_20_SQ_HEAD_S 0
+#define QP1C_BYTES_20_SQ_HEAD_M (((1UL << 15) - 1) << QP1C_BYTES_20_SQ_HEAD_S)
+
+#define QP1C_BYTES_20_PKEY_IDX_S 16
+#define QP1C_BYTES_20_PKEY_IDX_M \
+ (((1UL << 16) - 1) << QP1C_BYTES_20_PKEY_IDX_S)
+
+#define QP1C_BYTES_28_CUR_RQ_WQE_BA_H_S 0
+#define QP1C_BYTES_28_CUR_RQ_WQE_BA_H_M \
+ (((1UL << 5) - 1) << QP1C_BYTES_28_CUR_RQ_WQE_BA_H_S)
+
+#define QP1C_BYTES_28_RQ_CUR_IDX_S 16
+#define QP1C_BYTES_28_RQ_CUR_IDX_M \
+ (((1UL << 15) - 1) << QP1C_BYTES_28_RQ_CUR_IDX_S)
+
+#define QP1C_BYTES_32_TX_CQ_NUM_S 0
+#define QP1C_BYTES_32_TX_CQ_NUM_M \
+ (((1UL << 16) - 1) << QP1C_BYTES_32_TX_CQ_NUM_S)
+
+#define QP1C_BYTES_32_RX_CQ_NUM_S 16
+#define QP1C_BYTES_32_RX_CQ_NUM_M \
+ (((1UL << 16) - 1) << QP1C_BYTES_32_RX_CQ_NUM_S)
+
+#define QP1C_BYTES_40_CUR_SQ_WQE_BA_H_S 0
+#define QP1C_BYTES_40_CUR_SQ_WQE_BA_H_M \
+ (((1UL << 5) - 1) << QP1C_BYTES_40_CUR_SQ_WQE_BA_H_S)
+
+#define QP1C_BYTES_40_SQ_CUR_IDX_S 16
+#define QP1C_BYTES_40_SQ_CUR_IDX_M \
+ (((1UL << 15) - 1) << QP1C_BYTES_40_SQ_CUR_IDX_S)
+
+#define HNS_ROCE_WQE_INLINE (1UL<<31)
+#define HNS_ROCE_WQE_SE (1UL<<30)
+
+#define HNS_ROCE_WQE_SGE_NUM_BIT 24
+#define HNS_ROCE_WQE_IMM (1UL<<23)
+#define HNS_ROCE_WQE_FENCE (1UL<<21)
+#define HNS_ROCE_WQE_CQ_NOTIFY (1UL<<20)
+
+#define HNS_ROCE_WQE_OPCODE_SEND (0<<16)
+#define HNS_ROCE_WQE_OPCODE_RDMA_READ (1<<16)
+#define HNS_ROCE_WQE_OPCODE_RDMA_WRITE (2<<16)
+#define HNS_ROCE_WQE_OPCODE_MASK (15<<16)
+
+struct hns_roce_qp_context {
+ u32 qpc_bytes_4;
+ u32 qpc_bytes_8;
+ u32 qpc_bytes_12;
+ u32 qpc_bytes_16;
+ u32 sq_rq_bt_l;
+ u32 qpc_bytes_24;
+ u32 irrl_ba_l;
+ u32 qpc_bytes_32;
+ u32 qpc_bytes_36;
+ u32 dmac_l;
+ u32 qpc_bytes_44;
+ u32 qpc_bytes_48;
+ u8 dgid[16];
+ u32 qpc_bytes_68;
+ u32 cur_rq_wqe_ba_l;
+ u32 qpc_bytes_76;
+ u32 rx_rnr_time;
+ u32 qpc_bytes_84;
+ u32 qpc_bytes_88;
+ union {
+ u32 rx_sge_len;
+ u32 dma_length;
+ };
+ union {
+ u32 rx_sge_num;
+ u32 rx_send_pktn;
+ u32 r_key;
+ };
+ u32 va_l;
+ u32 va_h;
+ u32 qpc_bytes_108;
+ u32 qpc_bytes_112;
+ u32 rx_cur_sq_wqe_ba_l;
+ u32 qpc_bytes_120;
+ u32 qpc_bytes_124;
+ u32 qpc_bytes_128;
+ u32 qpc_bytes_132;
+ u32 qpc_bytes_136;
+ u32 qpc_bytes_140;
+ u32 qpc_bytes_144;
+ u32 qpc_bytes_148;
+ union {
+ u32 rnr_retry;
+ u32 ack_time;
+ };
+ u32 qpc_bytes_156;
+ u32 pkt_use_len;
+ u32 qpc_bytes_164;
+ u32 qpc_bytes_168;
+ union {
+ u32 sge_use_len;
+ u32 pa_use_len;
+ };
+ u32 qpc_bytes_176;
+ u32 qpc_bytes_180;
+ u32 tx_cur_sq_wqe_ba_l;
+ u32 qpc_bytes_188;
+ u32 rvd21;
+};
+
+#define QP_CONTEXT_QPC_BYTES_4_TRANSPORT_SERVICE_TYPE_S 0
+#define QP_CONTEXT_QPC_BYTES_4_TRANSPORT_SERVICE_TYPE_M \
+ (((1UL << 3) - 1) << QP_CONTEXT_QPC_BYTES_4_TRANSPORT_SERVICE_TYPE_S)
+
+#define QP_CONTEXT_QPC_BYTE_4_ENABLE_FPMR_S 3
+#define QP_CONTEXT_QPC_BYTE_4_RDMA_READ_ENABLE_S 4
+#define QP_CONTEXT_QPC_BYTE_4_RDMA_WRITE_ENABLE_S 5
+#define QP_CONTEXT_QPC_BYTE_4_ATOMIC_OPERATION_ENABLE_S 6
+#define QP_CONTEXT_QPC_BYTE_4_RDMAR_USE_S 7
+
+#define QP_CONTEXT_QPC_BYTES_4_SQ_WQE_SHIFT_S 8
+#define QP_CONTEXT_QPC_BYTES_4_SQ_WQE_SHIFT_M \
+ (((1UL << 4) - 1) << QP_CONTEXT_QPC_BYTES_4_SQ_WQE_SHIFT_S)
+
+#define QP_CONTEXT_QPC_BYTES_4_RQ_WQE_SHIFT_S 12
+#define QP_CONTEXT_QPC_BYTES_4_RQ_WQE_SHIFT_M \
+ (((1UL << 4) - 1) << QP_CONTEXT_QPC_BYTES_4_RQ_WQE_SHIFT_S)
+
+#define QP_CONTEXT_QPC_BYTES_4_PD_S 16
+#define QP_CONTEXT_QPC_BYTES_4_PD_M \
+ (((1UL << 16) - 1) << QP_CONTEXT_QPC_BYTES_4_PD_S)
+
+#define QP_CONTEXT_QPC_BYTES_8_TX_COMPLETION_S 0
+#define QP_CONTEXT_QPC_BYTES_8_TX_COMPLETION_M \
+ (((1UL << 16) - 1) << QP_CONTEXT_QPC_BYTES_8_TX_COMPLETION_S)
+
+#define QP_CONTEXT_QPC_BYTES_8_RX_COMPLETION_S 16
+#define QP_CONTEXT_QPC_BYTES_8_RX_COMPLETION_M \
+ (((1UL << 16) - 1) << QP_CONTEXT_QPC_BYTES_8_RX_COMPLETION_S)
+
+#define QP_CONTEXT_QPC_BYTES_12_SRQ_NUMBER_S 0
+#define QP_CONTEXT_QPC_BYTES_12_SRQ_NUMBER_M \
+ (((1UL << 16) - 1) << QP_CONTEXT_QPC_BYTES_12_SRQ_NUMBER_S)
+
+#define QP_CONTEXT_QPC_BYTES_12_P_KEY_INDEX_S 16
+#define QP_CONTEXT_QPC_BYTES_12_P_KEY_INDEX_M \
+ (((1UL << 16) - 1) << QP_CONTEXT_QPC_BYTES_12_P_KEY_INDEX_S)
+
+#define QP_CONTEXT_QPC_BYTES_16_QP_NUM_S 0
+#define QP_CONTEXT_QPC_BYTES_16_QP_NUM_M \
+ (((1UL << 24) - 1) << QP_CONTEXT_QPC_BYTES_16_QP_NUM_S)
+
+#define QP_CONTEXT_QPC_BYTES_24_SQ_RQ_BT_H_S 0
+#define QP_CONTEXT_QPC_BYTES_24_SQ_RQ_BT_H_M \
+ (((1UL << 17) - 1) << QP_CONTEXT_QPC_BYTES_24_SQ_RQ_BT_H_S)
+
+#define QP_CONTEXT_QPC_BYTES_24_MINIMUM_RNR_NAK_TIMER_S 18
+#define QP_CONTEXT_QPC_BYTES_24_MINIMUM_RNR_NAK_TIMER_M \
+ (((1UL << 5) - 1) << QP_CONTEXT_QPC_BYTES_24_MINIMUM_RNR_NAK_TIMER_S)
+
+#define QP_CONTEXT_QPC_BYTE_24_REMOTE_ENABLE_E2E_CREDITS_S 23
+
+#define QP_CONTEXT_QPC_BYTES_32_IRRL_BA_H_S 0
+#define QP_CONTEXT_QPC_BYTES_32_IRRL_BA_H_M \
+ (((1UL << 17) - 1) << QP_CONTEXT_QPC_BYTES_32_IRRL_BA_H_S)
+
+#define QP_CONTEXT_QPC_BYTES_32_MIG_STATE_S 18
+#define QP_CONTEXT_QPC_BYTES_32_MIG_STATE_M \
+ (((1UL << 2) - 1) << QP_CONTEXT_QPC_BYTES_32_MIG_STATE_S)
+
+#define QP_CONTEXT_QPC_BYTE_32_LOCAL_ENABLE_E2E_CREDITS_S 20
+#define QP_CONTEXT_QPC_BYTE_32_SIGNALING_TYPE_S 21
+#define QP_CONTEXT_QPC_BYTE_32_LOOPBACK_INDICATOR_S 22
+#define QP_CONTEXT_QPC_BYTE_32_GLOBAL_HEADER_S 23
+
+#define QP_CONTEXT_QPC_BYTES_32_RESPONDER_RESOURCES_S 24
+#define QP_CONTEXT_QPC_BYTES_32_RESPONDER_RESOURCES_M \
+ (((1UL << 8) - 1) << QP_CONTEXT_QPC_BYTES_32_RESPONDER_RESOURCES_S)
+
+#define QP_CONTEXT_QPC_BYTES_36_DEST_QP_S 0
+#define QP_CONTEXT_QPC_BYTES_36_DEST_QP_M \
+ (((1UL << 24) - 1) << QP_CONTEXT_QPC_BYTES_36_DEST_QP_S)
+
+#define QP_CONTEXT_QPC_BYTES_36_SGID_INDEX_S 24
+#define QP_CONTEXT_QPC_BYTES_36_SGID_INDEX_M \
+ (((1UL << 8) - 1) << QP_CONTEXT_QPC_BYTES_36_SGID_INDEX_S)
+
+#define QP_CONTEXT_QPC_BYTES_44_DMAC_H_S 0
+#define QP_CONTEXT_QPC_BYTES_44_DMAC_H_M \
+ (((1UL << 16) - 1) << QP_CONTEXT_QPC_BYTES_44_DMAC_H_S)
+
+#define QP_CONTEXT_QPC_BYTES_44_MAXIMUM_STATIC_RATE_S 16
+#define QP_CONTEXT_QPC_BYTES_44_MAXIMUM_STATIC_RATE_M \
+ (((1UL << 8) - 1) << QP_CONTEXT_QPC_BYTES_44_MAXIMUM_STATIC_RATE_S)
+
+#define QP_CONTEXT_QPC_BYTES_44_HOPLMT_S 24
+#define QP_CONTEXT_QPC_BYTES_44_HOPLMT_M \
+ (((1UL << 8) - 1) << QP_CONTEXT_QPC_BYTES_44_HOPLMT_S)
+
+#define QP_CONTEXT_QPC_BYTES_48_FLOWLABEL_S 0
+#define QP_CONTEXT_QPC_BYTES_48_FLOWLABEL_M \
+ (((1UL << 20) - 1) << QP_CONTEXT_QPC_BYTES_48_FLOWLABEL_S)
+
+#define QP_CONTEXT_QPC_BYTES_48_TCLASS_S 20
+#define QP_CONTEXT_QPC_BYTES_48_TCLASS_M \
+ (((1UL << 8) - 1) << QP_CONTEXT_QPC_BYTES_48_TCLASS_S)
+
+#define QP_CONTEXT_QPC_BYTES_48_MTU_S 28
+#define QP_CONTEXT_QPC_BYTES_48_MTU_M \
+ (((1UL << 4) - 1) << QP_CONTEXT_QPC_BYTES_48_MTU_S)
+
+#define QP_CONTEXT_QPC_BYTES_68_RQ_HEAD_S 0
+#define QP_CONTEXT_QPC_BYTES_68_RQ_HEAD_M \
+ (((1UL << 15) - 1) << QP_CONTEXT_QPC_BYTES_68_RQ_HEAD_S)
+
+#define QP_CONTEXT_QPC_BYTES_68_RQ_CUR_INDEX_S 16
+#define QP_CONTEXT_QPC_BYTES_68_RQ_CUR_INDEX_M \
+ (((1UL << 15) - 1) << QP_CONTEXT_QPC_BYTES_68_RQ_CUR_INDEX_S)
+
+#define QP_CONTEXT_QPC_BYTES_76_CUR_RQ_WQE_BA_H_S 0
+#define QP_CONTEXT_QPC_BYTES_76_CUR_RQ_WQE_BA_H_M \
+ (((1UL << 5) - 1) << QP_CONTEXT_QPC_BYTES_76_CUR_RQ_WQE_BA_H_S)
+
+#define QP_CONTEXT_QPC_BYTES_76_RX_REQ_MSN_S 8
+#define QP_CONTEXT_QPC_BYTES_76_RX_REQ_MSN_M \
+ (((1UL << 24) - 1) << QP_CONTEXT_QPC_BYTES_76_RX_REQ_MSN_S)
+
+#define QP_CONTEXT_QPC_BYTES_84_LAST_ACK_PSN_S 0
+#define QP_CONTEXT_QPC_BYTES_84_LAST_ACK_PSN_M \
+ (((1UL << 24) - 1) << QP_CONTEXT_QPC_BYTES_84_LAST_ACK_PSN_S)
+
+#define QP_CONTEXT_QPC_BYTES_84_TRRL_HEAD_S 24
+#define QP_CONTEXT_QPC_BYTES_84_TRRL_HEAD_M \
+ (((1UL << 8) - 1) << QP_CONTEXT_QPC_BYTES_84_TRRL_HEAD_S)
+
+#define QP_CONTEXT_QPC_BYTES_88_RX_REQ_EPSN_S 0
+#define QP_CONTEXT_QPC_BYTES_88_RX_REQ_EPSN_M \
+ (((1UL << 24) - 1) << QP_CONTEXT_QPC_BYTES_88_RX_REQ_EPSN_S)
+
+#define QP_CONTEXT_QPC_BYTES_88_RX_REQ_PSN_ERR_FLAG_S 24
+#define QP_CONTEXT_QPC_BYTES_88_RX_LAST_OPCODE_FLG_S 25
+
+#define QP_CONTEXT_QPC_BYTES_88_RQ_REQ_LAST_OPERATION_TYPE_S 26
+#define QP_CONTEXT_QPC_BYTES_88_RQ_REQ_LAST_OPERATION_TYPE_M \
+ (((1UL << 2) - 1) << \
+ QP_CONTEXT_QPC_BYTES_88_RQ_REQ_LAST_OPERATION_TYPE_S)
+
+#define QP_CONTEXT_QPC_BYTES_88_RQ_REQ_RDMA_WR_FLAG_S 29
+#define QP_CONTEXT_QPC_BYTES_88_RQ_REQ_RDMA_WR_FLAG_M \
+ (((1UL << 2) - 1) << QP_CONTEXT_QPC_BYTES_88_RQ_REQ_RDMA_WR_FLAG_S)
+
+#define QP_CONTEXT_QPC_BYTES_108_TRRL_SDB_PSN_S 0
+#define QP_CONTEXT_QPC_BYTES_108_TRRL_SDB_PSN_M \
+ (((1UL << 24) - 1) << QP_CONTEXT_QPC_BYTES_108_TRRL_SDB_PSN_S)
+
+#define QP_CONTEXT_QPC_BYTES_108_TRRL_SDB_PSN_FLG_S 24
+#define QP_CONTEXT_QPC_BYTES_108_TRRL_TDB_PSN_FLG_S 25
+
+#define QP_CONTEXT_QPC_BYTES_112_TRRL_TDB_PSN_S 0
+#define QP_CONTEXT_QPC_BYTES_112_TRRL_TDB_PSN_M \
+ (((1UL << 24) - 1) << QP_CONTEXT_QPC_BYTES_112_TRRL_TDB_PSN_S)
+
+#define QP_CONTEXT_QPC_BYTES_112_TRRL_TAIL_S 24
+#define QP_CONTEXT_QPC_BYTES_112_TRRL_TAIL_M \
+ (((1UL << 8) - 1) << QP_CONTEXT_QPC_BYTES_112_TRRL_TAIL_S)
+
+#define QP_CONTEXT_QPC_BYTES_120_RX_CUR_SQ_WQE_BA_H_S 0
+#define QP_CONTEXT_QPC_BYTES_120_RX_CUR_SQ_WQE_BA_H_M \
+ (((1UL << 5) - 1) << QP_CONTEXT_QPC_BYTES_120_RX_CUR_SQ_WQE_BA_H_S)
+
+#define QP_CONTEXT_QPC_BYTES_124_RX_ACK_MSN_S 0
+#define QP_CONTEXT_QPC_BYTES_124_RX_ACK_MSN_M \
+ (((1UL << 15) - 1) << QP_CONTEXT_QPC_BYTES_124_RX_ACK_MSN_S)
+
+#define QP_CONTEXT_QPC_BYTES_124_IRRL_MSG_IDX_S 16
+#define QP_CONTEXT_QPC_BYTES_124_IRRL_MSG_IDX_M \
+ (((1UL << 15) - 1) << QP_CONTEXT_QPC_BYTES_124_IRRL_MSG_IDX_S)
+
+#define QP_CONTEXT_QPC_BYTES_128_RX_ACK_EPSN_S 0
+#define QP_CONTEXT_QPC_BYTES_128_RX_ACK_EPSN_M \
+ (((1UL << 24) - 1) << QP_CONTEXT_QPC_BYTES_128_RX_ACK_EPSN_S)
+
+#define QP_CONTEXT_QPC_BYTES_128_RX_ACK_PSN_ERR_FLG_S 24
+
+#define QP_CONTEXT_QPC_BYTES_128_ACK_LAST_OPERATION_TYPE_S 25
+#define QP_CONTEXT_QPC_BYTES_128_ACK_LAST_OPERATION_TYPE_M \
+ (((1UL << 2) - 1) << QP_CONTEXT_QPC_BYTES_128_ACK_LAST_OPERATION_TYPE_S)
+
+#define QP_CONTEXT_QPC_BYTES_128_IRRL_PSN_VLD_FLG_S 27
+
+#define QP_CONTEXT_QPC_BYTES_132_IRRL_PSN_S 0
+#define QP_CONTEXT_QPC_BYTES_132_IRRL_PSN_M \
+ (((1UL << 24) - 1) << QP_CONTEXT_QPC_BYTES_132_IRRL_PSN_S)
+
+#define QP_CONTEXT_QPC_BYTES_132_IRRL_TAIL_S 24
+#define QP_CONTEXT_QPC_BYTES_132_IRRL_TAIL_M \
+ (((1UL << 8) - 1) << QP_CONTEXT_QPC_BYTES_132_IRRL_TAIL_S)
+
+#define QP_CONTEXT_QPC_BYTES_136_RETRY_MSG_PSN_S 0
+#define QP_CONTEXT_QPC_BYTES_136_RETRY_MSG_PSN_M \
+ (((1UL << 24) - 1) << QP_CONTEXT_QPC_BYTES_136_RETRY_MSG_PSN_S)
+
+#define QP_CONTEXT_QPC_BYTES_136_RETRY_MSG_FPKT_PSN_L_S 24
+#define QP_CONTEXT_QPC_BYTES_136_RETRY_MSG_FPKT_PSN_L_M \
+ (((1UL << 8) - 1) << QP_CONTEXT_QPC_BYTES_136_RETRY_MSG_FPKT_PSN_L_S)
+
+#define QP_CONTEXT_QPC_BYTES_140_RETRY_MSG_FPKT_PSN_H_S 0
+#define QP_CONTEXT_QPC_BYTES_140_RETRY_MSG_FPKT_PSN_H_M \
+ (((1UL << 16) - 1) << QP_CONTEXT_QPC_BYTES_140_RETRY_MSG_FPKT_PSN_H_S)
+
+#define QP_CONTEXT_QPC_BYTES_140_RETRY_MSG_MSN_S 16
+#define QP_CONTEXT_QPC_BYTES_140_RETRY_MSG_MSN_M \
+ (((1UL << 15) - 1) << QP_CONTEXT_QPC_BYTES_140_RETRY_MSG_MSN_S)
+
+#define QP_CONTEXT_QPC_BYTES_140_RNR_RETRY_FLG_S 31
+
+#define QP_CONTEXT_QPC_BYTES_144_QP_STATE_S 0
+#define QP_CONTEXT_QPC_BYTES_144_QP_STATE_M \
+ (((1UL << 3) - 1) << QP_CONTEXT_QPC_BYTES_144_QP_STATE_S)
+
+#define QP_CONTEXT_QPC_BYTES_148_CHECK_FLAG_S 0
+#define QP_CONTEXT_QPC_BYTES_148_CHECK_FLAG_M \
+ (((1UL << 2) - 1) << QP_CONTEXT_QPC_BYTES_148_CHECK_FLAG_S)
+
+#define QP_CONTEXT_QPC_BYTES_148_RETRY_COUNT_S 2
+#define QP_CONTEXT_QPC_BYTES_148_RETRY_COUNT_M \
+ (((1UL << 3) - 1) << QP_CONTEXT_QPC_BYTES_148_RETRY_COUNT_S)
+
+#define QP_CONTEXT_QPC_BYTES_148_RNR_RETRY_COUNT_S 5
+#define QP_CONTEXT_QPC_BYTES_148_RNR_RETRY_COUNT_M \
+ (((1UL << 3) - 1) << QP_CONTEXT_QPC_BYTES_148_RNR_RETRY_COUNT_S)
+
+#define QP_CONTEXT_QPC_BYTES_148_LSN_S 8
+#define QP_CONTEXT_QPC_BYTES_148_LSN_M \
+ (((1UL << 16) - 1) << QP_CONTEXT_QPC_BYTES_148_LSN_S)
+
+#define QP_CONTEXT_QPC_BYTES_156_RETRY_COUNT_INIT_S 0
+#define QP_CONTEXT_QPC_BYTES_156_RETRY_COUNT_INIT_M \
+ (((1UL << 3) - 1) << QP_CONTEXT_QPC_BYTES_156_RETRY_COUNT_INIT_S)
+
+#define QP_CONTEXT_QPC_BYTES_156_ACK_TIMEOUT_S 3
+#define QP_CONTEXT_QPC_BYTES_156_ACK_TIMEOUT_M \
+ (((1UL << 5) - 1) << QP_CONTEXT_QPC_BYTES_156_ACK_TIMEOUT_S)
+
+#define QP_CONTEXT_QPC_BYTES_156_RNR_RETRY_COUNT_INIT_S 8
+#define QP_CONTEXT_QPC_BYTES_156_RNR_RETRY_COUNT_INIT_M \
+ (((1UL << 3) - 1) << QP_CONTEXT_QPC_BYTES_156_RNR_RETRY_COUNT_INIT_S)
+
+#define QP_CONTEXT_QPC_BYTES_156_PORT_NUM_S 11
+#define QP_CONTEXT_QPC_BYTES_156_PORT_NUM_M \
+ (((1UL << 3) - 1) << QP_CONTEXT_QPC_BYTES_156_PORT_NUM_S)
+
+#define QP_CONTEXT_QPC_BYTES_156_SL_S 14
+#define QP_CONTEXT_QPC_BYTES_156_SL_M \
+ (((1UL << 2) - 1) << QP_CONTEXT_QPC_BYTES_156_SL_S)
+
+#define QP_CONTEXT_QPC_BYTES_156_INITIATOR_DEPTH_S 16
+#define QP_CONTEXT_QPC_BYTES_156_INITIATOR_DEPTH_M \
+ (((1UL << 8) - 1) << QP_CONTEXT_QPC_BYTES_156_INITIATOR_DEPTH_S)
+
+#define QP_CONTEXT_QPC_BYTES_156_ACK_REQ_IND_S 24
+#define QP_CONTEXT_QPC_BYTES_156_ACK_REQ_IND_M \
+ (((1UL << 2) - 1) << QP_CONTEXT_QPC_BYTES_156_ACK_REQ_IND_S)
+
+#define QP_CONTEXT_QPC_BYTES_164_SQ_PSN_S 0
+#define QP_CONTEXT_QPC_BYTES_164_SQ_PSN_M \
+ (((1UL << 24) - 1) << QP_CONTEXT_QPC_BYTES_164_SQ_PSN_S)
+
+#define QP_CONTEXT_QPC_BYTES_164_IRRL_HEAD_S 24
+#define QP_CONTEXT_QPC_BYTES_164_IRRL_HEAD_M \
+ (((1UL << 8) - 1) << QP_CONTEXT_QPC_BYTES_164_IRRL_HEAD_S)
+
+#define QP_CONTEXT_QPC_BYTES_168_RETRY_SQ_PSN_S 0
+#define QP_CONTEXT_QPC_BYTES_168_RETRY_SQ_PSN_M \
+ (((1UL << 24) - 1) << QP_CONTEXT_QPC_BYTES_168_RETRY_SQ_PSN_S)
+
+#define QP_CONTEXT_QPC_BYTES_168_SGE_USE_FLA_S 24
+#define QP_CONTEXT_QPC_BYTES_168_SGE_USE_FLA_M \
+ (((1UL << 2) - 1) << QP_CONTEXT_QPC_BYTES_168_SGE_USE_FLA_S)
+
+#define QP_CONTEXT_QPC_BYTES_168_DB_TYPE_S 26
+#define QP_CONTEXT_QPC_BYTES_168_DB_TYPE_M \
+ (((1UL << 2) - 1) << QP_CONTEXT_QPC_BYTES_168_DB_TYPE_S)
+
+#define QP_CONTEXT_QPC_BYTES_168_MSG_LP_IND_S 28
+#define QP_CONTEXT_QPC_BYTES_168_CSDB_LP_IND_S 29
+#define QP_CONTEXT_QPC_BYTES_168_QP_ERR_FLG_S 30
+
+#define QP_CONTEXT_QPC_BYTES_176_DB_CUR_INDEX_S 0
+#define QP_CONTEXT_QPC_BYTES_176_DB_CUR_INDEX_M \
+ (((1UL << 15) - 1) << QP_CONTEXT_QPC_BYTES_176_DB_CUR_INDEX_S)
+
+#define QP_CONTEXT_QPC_BYTES_176_RETRY_DB_CUR_INDEX_S 16
+#define QP_CONTEXT_QPC_BYTES_176_RETRY_DB_CUR_INDEX_M \
+ (((1UL << 15) - 1) << QP_CONTEXT_QPC_BYTES_176_RETRY_DB_CUR_INDEX_S)
+
+#define QP_CONTEXT_QPC_BYTES_180_SQ_HEAD_S 0
+#define QP_CONTEXT_QPC_BYTES_180_SQ_HEAD_M \
+ (((1UL << 15) - 1) << QP_CONTEXT_QPC_BYTES_180_SQ_HEAD_S)
+
+#define QP_CONTEXT_QPC_BYTES_180_SQ_CUR_INDEX_S 16
+#define QP_CONTEXT_QPC_BYTES_180_SQ_CUR_INDEX_M \
+ (((1UL << 15) - 1) << QP_CONTEXT_QPC_BYTES_180_SQ_CUR_INDEX_S)
+
+#define QP_CONTEXT_QPC_BYTES_188_TX_CUR_SQ_WQE_BA_H_S 0
+#define QP_CONTEXT_QPC_BYTES_188_TX_CUR_SQ_WQE_BA_H_M \
+ (((1UL << 5) - 1) << QP_CONTEXT_QPC_BYTES_188_TX_CUR_SQ_WQE_BA_H_S)
+
+#define QP_CONTEXT_QPC_BYTES_188_PKT_RETRY_FLG_S 8
+
+#define QP_CONTEXT_QPC_BYTES_188_TX_RETRY_CUR_INDEX_S 16
+#define QP_CONTEXT_QPC_BYTES_188_TX_RETRY_CUR_INDEX_M \
+ (((1UL << 15) - 1) << QP_CONTEXT_QPC_BYTES_188_TX_RETRY_CUR_INDEX_S)
+
+struct hns_roce_rq_db {
+ u32 u32_4;
+ u32 u32_8;
+};
+
+#define RQ_DOORBELL_U32_4_RQ_HEAD_S 0
+#define RQ_DOORBELL_U32_4_RQ_HEAD_M \
+ (((1UL << 15) - 1) << RQ_DOORBELL_U32_4_RQ_HEAD_S)
+
+#define RQ_DOORBELL_U32_8_QPN_S 0
+#define RQ_DOORBELL_U32_8_QPN_M (((1UL << 24) - 1) << RQ_DOORBELL_U32_8_QPN_S)
+
+#define RQ_DOORBELL_U32_8_CMD_S 28
+#define RQ_DOORBELL_U32_8_CMD_M (((1UL << 3) - 1) << RQ_DOORBELL_U32_8_CMD_S)
+
+#define RQ_DOORBELL_U32_8_HW_SYNC_S 31
+
+struct hns_roce_sq_db {
+ u32 u32_4;
+ u32 u32_8;
+};
+
+#define SQ_DOORBELL_U32_4_SQ_HEAD_S 0
+#define SQ_DOORBELL_U32_4_SQ_HEAD_M \
+ (((1UL << 15) - 1) << SQ_DOORBELL_U32_4_SQ_HEAD_S)
+
+#define SQ_DOORBELL_U32_4_PORT_S 18
+#define SQ_DOORBELL_U32_4_PORT_M (((1UL << 3) - 1) << SQ_DOORBELL_U32_4_PORT_S)
+
+#define SQ_DOORBELL_U32_8_QPN_S 0
+#define SQ_DOORBELL_U32_8_QPN_M (((1UL << 24) - 1) << SQ_DOORBELL_U32_8_QPN_S)
+
+#define SQ_DOORBELL_HW_SYNC_S 31
struct hns_roce_ext_db {
int esdb_almept;
diff --git a/drivers/infiniband/hw/hns/hns_roce_icm.c b/drivers/infiniband/hw/hns/hns_roce_icm.c
index 0e854fc..f86d94b 100644
--- a/drivers/infiniband/hw/hns/hns_roce_icm.c
+++ b/drivers/infiniband/hw/hns/hns_roce_icm.c
@@ -14,6 +14,7 @@
#include <linux/slab.h>
#include <linux/scatterlist.h>
#include "hns_roce_device.h"
+#include "hns_roce_cmd.h"
#include "hns_roce_icm.h"
#include "hns_roce_common.h"
@@ -403,6 +404,50 @@ void hns_roce_table_put(struct hns_roce_dev *hr_dev,
mutex_unlock(&table->mutex);
}
+void *hns_roce_table_find(struct hns_roce_icm_table *table, unsigned long obj,
+ dma_addr_t *dma_handle)
+{
+ struct hns_roce_icm_chunk *chunk;
+ unsigned long idx;
+ int i;
+ int offset, dma_offset;
+ struct hns_roce_icm *icm;
+ struct page *page = NULL;
+
+ if (!table->lowmem)
+ return NULL;
+
+ mutex_lock(&table->mutex);
+ idx = (obj & (table->num_obj - 1)) * table->obj_size;
+ icm = table->icm[idx / HNS_ROCE_TABLE_CHUNK_SIZE];
+ dma_offset = offset = idx % HNS_ROCE_TABLE_CHUNK_SIZE;
+
+ if (!icm)
+ goto out;
+
+ list_for_each_entry(chunk, &icm->chunk_list, list) {
+ for (i = 0; i < chunk->npages; ++i) {
+ if (dma_handle && dma_offset >= 0) {
+ if (sg_dma_len(&chunk->mem[i]) >
+ (u32)dma_offset)
+ *dma_handle = sg_dma_address(
+ &chunk->mem[i]) + dma_offset;
+ dma_offset -= sg_dma_len(&chunk->mem[i]);
+ }
+
+ if (chunk->mem[i].length > (u32)offset) {
+ page = sg_page(&chunk->mem[i]);
+ goto out;
+ }
+ offset -= chunk->mem[i].length;
+ }
+ }
+
+out:
+ mutex_unlock(&table->mutex);
+ return page ? lowmem_page_address(page) + offset : NULL;
+}
+
int hns_roce_table_get_range(struct hns_roce_dev *hr_dev,
struct hns_roce_icm_table *table,
unsigned long start, unsigned long end)
@@ -428,6 +473,17 @@ fail:
return ret;
}
+void hns_roce_table_put_range(struct hns_roce_dev *hr_dev,
+ struct hns_roce_icm_table *table,
+ unsigned long start, unsigned long end)
+{
+ unsigned long i;
+
+ for (i = start; i <= end;
+ i += HNS_ROCE_TABLE_CHUNK_SIZE / table->obj_size)
+ hns_roce_table_put(hr_dev, table, i);
+}
+
int hns_roce_init_icm_table(struct hns_roce_dev *hr_dev,
struct hns_roce_icm_table *table, u32 type,
unsigned long obj_size, unsigned long nobj,
diff --git a/drivers/infiniband/hw/hns/hns_roce_icm.h b/drivers/infiniband/hw/hns/hns_roce_icm.h
index e1cf60b..b38b352 100644
--- a/drivers/infiniband/hw/hns/hns_roce_icm.h
+++ b/drivers/infiniband/hw/hns/hns_roce_icm.h
@@ -30,6 +30,10 @@ enum {
((256 - sizeof(struct list_head) - 2 * sizeof(int)) / \
(sizeof(struct scatterlist)))
+enum {
+ HNS_ROCE_ICM_PAGE_SHIFT = 12,
+};
+
struct hns_roce_icm_chunk {
struct list_head list;
int npages;
@@ -54,9 +58,14 @@ int hns_roce_table_get(struct hns_roce_dev *hr_dev,
struct hns_roce_icm_table *table, unsigned long obj);
void hns_roce_table_put(struct hns_roce_dev *hr_dev,
struct hns_roce_icm_table *table, unsigned long obj);
+void *hns_roce_table_find(struct hns_roce_icm_table *table, unsigned long obj,
+ dma_addr_t *dma_handle);
int hns_roce_table_get_range(struct hns_roce_dev *hr_dev,
struct hns_roce_icm_table *table,
unsigned long start, unsigned long end);
+void hns_roce_table_put_range(struct hns_roce_dev *hr_dev,
+ struct hns_roce_icm_table *table,
+ unsigned long start, unsigned long end);
int hns_roce_init_icm_table(struct hns_roce_dev *hr_dev,
struct hns_roce_icm_table *table, u32 type,
unsigned long obj_size, unsigned long nobj,
diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
index 7eb88bb..b8838b4 100644
--- a/drivers/infiniband/hw/hns/hns_roce_main.c
+++ b/drivers/infiniband/hw/hns/hns_roce_main.c
@@ -587,7 +587,11 @@ int hns_roce_register_device(struct hns_roce_dev *hr_dev)
(1ULL << IB_USER_VERBS_CMD_QUERY_DEVICE) |
(1ULL << IB_USER_VERBS_CMD_QUERY_PORT) |
(1ULL << IB_USER_VERBS_CMD_ALLOC_PD) |
- (1ULL << IB_USER_VERBS_CMD_DEALLOC_PD);
+ (1ULL << IB_USER_VERBS_CMD_DEALLOC_PD) |
+ (1ULL << IB_USER_VERBS_CMD_CREATE_QP) |
+ (1ULL << IB_USER_VERBS_CMD_MODIFY_QP) |
+ (1ULL << IB_USER_VERBS_CMD_QUERY_QP) |
+ (1ULL << IB_USER_VERBS_CMD_DESTROY_QP);
/* HCA||device||port */
ib_dev->modify_device = hns_roce_modify_device;
@@ -610,6 +614,14 @@ int hns_roce_register_device(struct hns_roce_dev *hr_dev)
ib_dev->query_ah = hns_roce_query_ah;
ib_dev->destroy_ah = hns_roce_destroy_ah;
+ /* QP */
+ ib_dev->create_qp = hns_roce_create_qp;
+ ib_dev->modify_qp = hns_roce_modify_qp;
+ ib_dev->query_qp = hr_dev->hw->query_qp;
+ ib_dev->destroy_qp = hr_dev->hw->destroy_qp;
+ ib_dev->post_send = hr_dev->hw->post_send;
+ ib_dev->post_recv = hr_dev->hw->post_recv;
+
ret = ib_register_device(ib_dev, NULL);
if (ret) {
dev_err(dev, "ib_register_device failed!\n");
diff --git a/drivers/infiniband/hw/hns/hns_roce_mr.c b/drivers/infiniband/hw/hns/hns_roce_mr.c
index 3e0fc1b..d51b119 100644
--- a/drivers/infiniband/hw/hns/hns_roce_mr.c
+++ b/drivers/infiniband/hw/hns/hns_roce_mr.c
@@ -12,6 +12,7 @@
#include <linux/slab.h>
#include <linux/platform_device.h>
#include "hns_roce_device.h"
+#include "hns_roce_icm.h"
static int hns_roce_buddy_alloc(struct hns_roce_buddy *buddy, int order,
unsigned long *seg)
@@ -137,6 +138,127 @@ static int hns_roce_alloc_mtt_range(struct hns_roce_dev *hr_dev, int order,
return 0;
}
+int hns_roce_mtt_init(struct hns_roce_dev *hr_dev, int npages, int page_shift,
+ struct hns_roce_mtt *mtt)
+{
+ int ret = 0;
+ int i;
+
+ /* Page num is zero, correspond to DMA memory register */
+ if (!npages) {
+ mtt->order = -1;
+ mtt->page_shift = HNS_ROCE_ICM_PAGE_SHIFT;
+ return 0;
+ }
+
+ /* Note: if page_shift is zero, FAST memory regsiter */
+ mtt->page_shift = page_shift;
+
+ /* Compute MTT entry necessary */
+ for (mtt->order = 0, i = HNS_ROCE_MTT_ENTRY_PER_SEG; i < npages;
+ i <<= 1)
+ ++mtt->order;
+
+ /* Allocate MTT entry */
+ ret = hns_roce_alloc_mtt_range(hr_dev, mtt->order, &mtt->first_seg);
+ if (ret == -1)
+ return -ENOMEM;
+
+ return 0;
+}
+
+void hns_roce_mtt_cleanup(struct hns_roce_dev *hr_dev, struct hns_roce_mtt *mtt)
+{
+ struct hns_roce_mr_table *mr_table = &hr_dev->mr_table;
+
+ if (mtt->order < 0)
+ return;
+
+ hns_roce_buddy_free(&mr_table->mtt_buddy, mtt->first_seg, mtt->order);
+ hns_roce_table_put_range(hr_dev, &mr_table->mtt_table, mtt->first_seg,
+ mtt->first_seg + (1 << mtt->order) - 1);
+}
+
+static int hns_roce_write_mtt_chunk(struct hns_roce_dev *hr_dev,
+ struct hns_roce_mtt *mtt, u32 start_index,
+ u32 npages, u64 *page_list)
+{
+ u32 i = 0;
+ __le64 *mtts = NULL;
+ dma_addr_t dma_handle;
+ u32 s = start_index * sizeof(u64);
+
+ /* All MTTs must fit in the same page */
+ if (start_index / (PAGE_SIZE / sizeof(u64)) !=
+ (start_index + npages - 1) / (PAGE_SIZE / sizeof(u64)))
+ return -EINVAL;
+
+ if (start_index & (HNS_ROCE_MTT_ENTRY_PER_SEG - 1))
+ return -EINVAL;
+
+ mtts = hns_roce_table_find(&hr_dev->mr_table.mtt_table,
+ mtt->first_seg + s / hr_dev->caps.mtt_entry_sz,
+ &dma_handle);
+ if (!mtts)
+ return -ENOMEM;
+
+ /* Save page addr, low 12 bits : 0 */
+ for (i = 0; i < npages; ++i)
+ mtts[i] = (cpu_to_le64(page_list[i])) >> PAGE_ADDR_SHIFT;
+
+ return 0;
+}
+
+int hns_roce_write_mtt(struct hns_roce_dev *hr_dev, struct hns_roce_mtt *mtt,
+ u32 start_index, u32 npages, u64 *page_list)
+{
+ int chunk;
+ int ret;
+
+ if (mtt->order < 0)
+ return -EINVAL;
+
+ while (npages > 0) {
+ chunk = min_t(int, PAGE_SIZE / sizeof(u64), npages);
+
+ ret = hns_roce_write_mtt_chunk(hr_dev, mtt, start_index, chunk,
+ page_list);
+ if (ret)
+ return ret;
+
+ npages -= chunk;
+ start_index += chunk;
+ page_list += chunk;
+ }
+
+ return 0;
+}
+
+int hns_roce_buf_write_mtt(struct hns_roce_dev *hr_dev,
+ struct hns_roce_mtt *mtt, struct hns_roce_buf *buf)
+{
+ u32 i = 0;
+ int ret = 0;
+ u64 *page_list = NULL;
+
+ page_list = kmalloc_array(buf->npages, sizeof(*page_list), GFP_KERNEL);
+ if (!page_list)
+ return -ENOMEM;
+
+ for (i = 0; i < buf->npages; ++i) {
+ if (buf->nbufs == 1)
+ page_list[i] = buf->direct.map + (i << buf->page_shift);
+ else
+ page_list[i] = buf->page_list[i].map;
+
+ }
+ ret = hns_roce_write_mtt(hr_dev, mtt, 0, buf->npages, page_list);
+
+ kfree(page_list);
+
+ return ret;
+}
+
int hns_roce_init_mr_table(struct hns_roce_dev *hr_dev)
{
struct hns_roce_mr_table *mr_table = &hr_dev->mr_table;
@@ -185,3 +307,41 @@ void hns_roce_cleanup_mr_table(struct hns_roce_dev *hr_dev)
hns_roce_bitmap_cleanup(&mr_table->mtpt_bitmap);
}
+int hns_roce_ib_umem_write_mtt(struct hns_roce_dev *hr_dev,
+ struct hns_roce_mtt *mtt, struct ib_umem *umem)
+{
+ struct scatterlist *sg;
+ int i, k, entry;
+ int ret = 0;
+ u64 *pages;
+ u32 n;
+ int len;
+
+ pages = (u64 *) __get_free_page(GFP_KERNEL);
+ if (!pages)
+ return -ENOMEM;
+
+ i = n = 0;
+
+ for_each_sg(umem->sg_head.sgl, sg, umem->nmap, entry) {
+ len = sg_dma_len(sg) >> mtt->page_shift;
+ for (k = 0; k < len; ++k) {
+ pages[i++] = sg_dma_address(sg) + umem->page_size * k;
+ if (i == PAGE_SIZE / sizeof(u64)) {
+ ret = hns_roce_write_mtt(hr_dev, mtt, n, i,
+ pages);
+ if (ret)
+ goto out;
+ n += i;
+ i = 0;
+ }
+ }
+ }
+
+ if (i)
+ ret = hns_roce_write_mtt(hr_dev, mtt, n, i, pages);
+
+out:
+ free_page((unsigned long) pages);
+ return ret;
+}
diff --git a/drivers/infiniband/hw/hns/hns_roce_qp.c b/drivers/infiniband/hw/hns/hns_roce_qp.c
index e8d396b..5bd665d 100644
--- a/drivers/infiniband/hw/hns/hns_roce_qp.c
+++ b/drivers/infiniband/hw/hns/hns_roce_qp.c
@@ -11,8 +11,12 @@
#include <linux/slab.h>
#include <rdma/ib_cache.h>
#include <rdma/ib_pack.h>
+#include "hns_roce_common.h"
#include "hns_roce_device.h"
+#include "hns_roce_icm.h"
+#include "hns_roce_user.h"
+#define DB_REG_OFFSET 0x1000
#define SQP_NUM 12
void hns_roce_qp_event(struct hns_roce_dev *hr_dev, u32 qpn, int event_type)
@@ -40,6 +44,767 @@ void hns_roce_qp_event(struct hns_roce_dev *hr_dev, u32 qpn, int event_type)
complete(&qp->free);
}
+static void hns_roce_ib_qp_event(struct hns_roce_qp *hr_qp,
+ enum hns_roce_event type)
+{
+ struct ib_event event;
+ struct ib_qp *ibqp = &hr_qp->ibqp;
+
+ if (ibqp->event_handler) {
+ event.device = ibqp->device;
+ event.element.qp = ibqp;
+ switch (type) {
+ case HNS_ROCE_EVENT_TYPE_PATH_MIG:
+ event.event = IB_EVENT_PATH_MIG;
+ break;
+ case HNS_ROCE_EVENT_TYPE_COMM_EST:
+ event.event = IB_EVENT_COMM_EST;
+ break;
+ case HNS_ROCE_EVENT_TYPE_SQ_DRAINED:
+ event.event = IB_EVENT_SQ_DRAINED;
+ break;
+ case HNS_ROCE_EVENT_TYPE_SRQ_LAST_WQE_REACH:
+ event.event = IB_EVENT_QP_LAST_WQE_REACHED;
+ break;
+ case HNS_ROCE_EVENT_TYPE_WQ_CATAS_ERROR:
+ event.event = IB_EVENT_QP_FATAL;
+ break;
+ case HNS_ROCE_EVENT_TYPE_PATH_MIG_FAILED:
+ event.event = IB_EVENT_PATH_MIG_ERR;
+ break;
+ case HNS_ROCE_EVENT_TYPE_INV_REQ_LOCAL_WQ_ERROR:
+ event.event = IB_EVENT_QP_REQ_ERR;
+ break;
+ case HNS_ROCE_EVENT_TYPE_LOCAL_WQ_ACCESS_ERROR:
+ event.event = IB_EVENT_QP_ACCESS_ERR;
+ break;
+ default:
+ dev_dbg(ibqp->device->dma_device, "roce_ib: Unexpected event type %d on QP %06lx\n",
+ type, hr_qp->qpn);
+ return;
+ }
+ ibqp->event_handler(&event, ibqp->qp_context);
+ }
+}
+
+int hns_roce_reserve_range_qp(struct hns_roce_dev *hr_dev, int cnt, int align,
+ unsigned long *base)
+{
+ struct hns_roce_qp_table *qp_table = &hr_dev->qp_table;
+ int ret = 0;
+ unsigned long qpn;
+
+ ret = hns_roce_bitmap_alloc_range(&qp_table->bitmap, cnt, align, &qpn);
+ if (ret == -1)
+ return -ENOMEM;
+
+ *base = qpn;
+
+ return 0;
+}
+
+enum hns_roce_qp_state to_hns_roce_state(enum ib_qp_state state)
+{
+ switch (state) {
+ case IB_QPS_RESET:
+ return HNS_ROCE_QP_STATE_RST;
+ case IB_QPS_INIT:
+ return HNS_ROCE_QP_STATE_INIT;
+ case IB_QPS_RTR:
+ return HNS_ROCE_QP_STATE_RTR;
+ case IB_QPS_RTS:
+ return HNS_ROCE_QP_STATE_RTS;
+ case IB_QPS_SQD:
+ return HNS_ROCE_QP_STATE_SQD;
+ case IB_QPS_ERR:
+ return HNS_ROCE_QP_STATE_ERR;
+ default:
+ return HNS_ROCE_QP_NUM_STATE;
+ }
+}
+
+int hns_roce_gsi_qp_alloc(struct hns_roce_dev *hr_dev, unsigned long qpn,
+ struct hns_roce_qp *hr_qp)
+{
+ struct hns_roce_qp_table *qp_table = &hr_dev->qp_table;
+ int ret;
+
+ if (!qpn)
+ return -EINVAL;
+
+ hr_qp->qpn = qpn;
+
+ spin_lock_irq(&qp_table->lock);
+ ret = radix_tree_insert(&hr_dev->qp_table_tree,
+ hr_qp->qpn & (hr_dev->caps.num_qps - 1), hr_qp);
+ spin_unlock_irq(&qp_table->lock);
+ if (ret) {
+ dev_err(&hr_dev->pdev->dev, "QPC radix_tree_insert failed\n");
+ goto err_put_irrl;
+ }
+
+ atomic_set(&hr_qp->refcount, 1);
+ init_completion(&hr_qp->free);
+
+ return 0;
+
+err_put_irrl:
+
+ return ret;
+}
+
+int hns_roce_qp_alloc(struct hns_roce_dev *hr_dev, unsigned long qpn,
+ struct hns_roce_qp *hr_qp)
+{
+ struct hns_roce_qp_table *qp_table = &hr_dev->qp_table;
+ struct device *dev = &hr_dev->pdev->dev;
+ int ret;
+
+ if (!qpn)
+ return -EINVAL;
+
+ hr_qp->qpn = qpn;
+
+ /* Alloc memory for QPC */
+ ret = hns_roce_table_get(hr_dev, &qp_table->qp_table, hr_qp->qpn);
+ if (ret) {
+ dev_err(dev, "QPC table get failed\n");
+ goto err_out;
+ }
+
+ /* Alloc memory for IRRL */
+ ret = hns_roce_table_get(hr_dev, &qp_table->irrl_table, hr_qp->qpn);
+ if (ret) {
+ dev_err(dev, "IRRL table get failed\n");
+ goto err_put_qp;
+ }
+
+ spin_lock_irq(&qp_table->lock);
+ ret = radix_tree_insert(&hr_dev->qp_table_tree,
+ hr_qp->qpn & (hr_dev->caps.num_qps - 1), hr_qp);
+ spin_unlock_irq(&qp_table->lock);
+ if (ret) {
+ dev_err(dev, "QPC radix_tree_insert failed\n");
+ goto err_put_irrl;
+ }
+
+ atomic_set(&hr_qp->refcount, 1);
+ init_completion(&hr_qp->free);
+
+ return 0;
+
+err_put_irrl:
+ hns_roce_table_put(hr_dev, &qp_table->irrl_table, hr_qp->qpn);
+
+err_put_qp:
+ hns_roce_table_put(hr_dev, &qp_table->qp_table, hr_qp->qpn);
+
+err_out:
+ return ret;
+}
+
+void hns_roce_qp_remove(struct hns_roce_dev *hr_dev, struct hns_roce_qp *hr_qp)
+{
+ struct hns_roce_qp_table *qp_table = &hr_dev->qp_table;
+ unsigned long flags;
+
+ spin_lock_irqsave(&qp_table->lock, flags);
+ radix_tree_delete(&hr_dev->qp_table_tree,
+ hr_qp->qpn & (hr_dev->caps.num_qps - 1));
+ spin_unlock_irqrestore(&qp_table->lock, flags);
+}
+
+void hns_roce_qp_free(struct hns_roce_dev *hr_dev, struct hns_roce_qp *hr_qp)
+{
+ struct hns_roce_qp_table *qp_table = &hr_dev->qp_table;
+
+ if (atomic_dec_and_test(&hr_qp->refcount))
+ complete(&hr_qp->free);
+ wait_for_completion(&hr_qp->free);
+
+ if ((hr_qp->ibqp.qp_type) != IB_QPT_GSI) {
+ hns_roce_table_put(hr_dev, &qp_table->irrl_table, hr_qp->qpn);
+ hns_roce_table_put(hr_dev, &qp_table->qp_table, hr_qp->qpn);
+ }
+}
+
+void hns_roce_release_range_qp(struct hns_roce_dev *hr_dev, int base_qpn,
+ int cnt)
+{
+ struct hns_roce_qp_table *qp_table = &hr_dev->qp_table;
+
+ if (base_qpn < (hr_dev->caps.sqp_start + 2 * hr_dev->caps.num_ports))
+ return;
+
+ hns_roce_bitmap_free_range(&qp_table->bitmap, base_qpn, cnt);
+}
+
+int hns_roce_set_rq_size(struct hns_roce_dev *hr_dev, struct ib_qp_cap *cap,
+ int is_user, int has_srq, struct hns_roce_qp *hr_qp)
+{
+ u32 max_cnt;
+ struct device *dev = &hr_dev->pdev->dev;
+
+ /* Check the validity of QP support capacity */
+ if (cap->max_recv_wr > hr_dev->caps.max_wqes ||
+ cap->max_recv_sge > hr_dev->caps.max_rq_sg) {
+ dev_err(dev, "RQ WR or sge error!max_recv_wr=%d max_recv_sge=%d\n",
+ cap->max_recv_wr, cap->max_recv_sge);
+ return -EINVAL;
+ }
+
+ /* If srq exit, set zero for relative number of rq */
+ if (has_srq) {
+ if (cap->max_recv_wr) {
+ dev_dbg(dev, "srq no need config max_recv_wr\n");
+ return -EINVAL;
+ }
+
+ hr_qp->rq.wqe_cnt = hr_qp->rq.max_gs = 0;
+ } else {
+ if (is_user && (!cap->max_recv_wr || !cap->max_recv_sge)) {
+ dev_err(dev, "user space no need config max_recv_wr max_recv_sge\n");
+ return -EINVAL;
+ }
+
+ /* In v1 engine, parameter verification procession */
+ max_cnt = cap->max_recv_wr > HNS_ROCE_MIN_WQE_NUM ?
+ cap->max_recv_wr : HNS_ROCE_MIN_WQE_NUM;
+ hr_qp->rq.wqe_cnt = roundup_pow_of_two(max_cnt);
+
+ if ((u32)hr_qp->rq.wqe_cnt > hr_dev->caps.max_wqes) {
+ dev_err(dev, "hns_roce_set_rq_size rq.wqe_cnt too large\n");
+ return -EINVAL;
+ }
+
+ max_cnt = max(1U, cap->max_recv_sge);
+ hr_qp->rq.max_gs = roundup_pow_of_two(max_cnt);
+ /* WQE is fixed for 64B */
+ hr_qp->rq.wqe_shift = ilog2(hr_dev->caps.max_rq_desc_sz);
+ }
+
+ cap->max_recv_wr = hr_qp->rq.max_post = hr_qp->rq.wqe_cnt;
+ cap->max_recv_sge = hr_qp->rq.max_gs;
+
+ return 0;
+}
+
+int hns_roce_set_user_sq_size(struct hns_roce_dev *hr_dev,
+ struct hns_roce_qp *hr_qp,
+ struct hns_roce_ib_create_qp *ucmd)
+{
+ u32 roundup_sq_stride = roundup_pow_of_two(hr_dev->caps.max_sq_desc_sz);
+ u8 max_sq_stride = ilog2(roundup_sq_stride);
+
+ /* Sanity check SQ size before proceeding */
+ if ((u32)(1 << ucmd->log_sq_bb_count) > hr_dev->caps.max_wqes ||
+ ucmd->log_sq_stride > max_sq_stride ||
+ ucmd->log_sq_stride < HNS_ROCE_IB_MIN_SQ_STRIDE) {
+ dev_err(&hr_dev->pdev->dev, "check SQ size error!\n");
+ return -EINVAL;
+ }
+
+ hr_qp->sq.wqe_cnt = 1 << ucmd->log_sq_bb_count;
+ hr_qp->sq.wqe_shift = ucmd->log_sq_stride;
+
+ /* Get buf size, SQ and RQ are aligned to page_szie */
+ hr_qp->buff_size = HNS_ROCE_ALOGN_UP((hr_qp->rq.wqe_cnt <<
+ hr_qp->rq.wqe_shift), PAGE_SIZE) +
+ HNS_ROCE_ALOGN_UP((hr_qp->sq.wqe_cnt <<
+ hr_qp->sq.wqe_shift), PAGE_SIZE);
+
+ hr_qp->sq.offset = 0;
+ hr_qp->rq.offset = HNS_ROCE_ALOGN_UP((hr_qp->sq.wqe_cnt <<
+ hr_qp->sq.wqe_shift), PAGE_SIZE);
+
+ return 0;
+}
+
+int hns_roce_set_kernel_sq_size(struct hns_roce_dev *hr_dev,
+ struct ib_qp_cap *cap, enum ib_qp_type type,
+ struct hns_roce_qp *hr_qp)
+{
+ struct device *dev = &hr_dev->pdev->dev;
+ u32 max_cnt;
+ (void)type;
+
+ if (cap->max_send_wr > hr_dev->caps.max_wqes ||
+ cap->max_send_sge > hr_dev->caps.max_sq_sg ||
+ cap->max_inline_data > hr_dev->caps.max_sq_inline) {
+ dev_err(dev, "hns_roce_set_kernel_sq_size error1\n");
+ return -EINVAL;
+ }
+
+ hr_qp->sq.wqe_shift = ilog2(hr_dev->caps.max_sq_desc_sz);
+ hr_qp->sq_max_wqes_per_wr = 1;
+ hr_qp->sq_spare_wqes = 0;
+
+ /* In v1 engine, parameter verification procession */
+ max_cnt = cap->max_send_wr > HNS_ROCE_MIN_WQE_NUM ?
+ cap->max_send_wr : HNS_ROCE_MIN_WQE_NUM;
+ hr_qp->sq.wqe_cnt = roundup_pow_of_two(max_cnt);
+ if ((u32)hr_qp->sq.wqe_cnt > hr_dev->caps.max_wqes) {
+ dev_err(dev, "hns_roce_set_kernel_sq_size sq.wqe_cnt too large\n");
+ return -EINVAL;
+ }
+
+ /* Get data_seg numbers */
+ max_cnt = max(1U, cap->max_send_sge);
+ hr_qp->sq.max_gs = roundup_pow_of_two(max_cnt);
+
+ /* Get buf size, SQ and RQ are aligned to page_szie */
+ hr_qp->buff_size = HNS_ROCE_ALOGN_UP((hr_qp->rq.wqe_cnt <<
+ hr_qp->rq.wqe_shift), PAGE_SIZE) +
+ HNS_ROCE_ALOGN_UP((hr_qp->sq.wqe_cnt <<
+ hr_qp->sq.wqe_shift), PAGE_SIZE);
+ hr_qp->sq.offset = 0;
+ hr_qp->rq.offset = HNS_ROCE_ALOGN_UP((hr_qp->sq.wqe_cnt <<
+ hr_qp->sq.wqe_shift), PAGE_SIZE);
+
+ /* Get wr and sge number which send */
+ cap->max_send_wr = hr_qp->sq.max_post = hr_qp->sq.wqe_cnt;
+ cap->max_send_sge = hr_qp->sq.max_gs;
+
+ /* We don't support inline sends for kernel QPs (yet) */
+ cap->max_inline_data = 0;
+
+ return 0;
+}
+
+static int hns_roce_create_qp_common(struct hns_roce_dev *hr_dev,
+ struct ib_pd *ib_pd,
+ struct ib_qp_init_attr *init_attr,
+ struct ib_udata *udata, unsigned long sqpn,
+ struct hns_roce_qp *hr_qp)
+{
+ struct device *dev = &hr_dev->pdev->dev;
+ struct hns_roce_ib_create_qp ucmd;
+ unsigned long qpn = 0;
+ int ret = 0;
+
+ mutex_init(&hr_qp->mutex);
+ spin_lock_init(&hr_qp->sq.lock);
+ spin_lock_init(&hr_qp->rq.lock);
+
+ hr_qp->state = IB_QPS_RESET;
+
+ if (init_attr->sq_sig_type == IB_SIGNAL_ALL_WR)
+ hr_qp->sq_signal_bits = IB_SIGNAL_ALL_WR;
+ else
+ hr_qp->sq_signal_bits = IB_SIGNAL_REQ_WR;
+
+ ret = hns_roce_set_rq_size(hr_dev, &init_attr->cap, !!ib_pd->uobject,
+ !!init_attr->srq, hr_qp);
+ if (ret) {
+ dev_err(dev, "hns_roce_set_rq_size failed\n");
+ goto err_out;
+ }
+
+ if (ib_pd->uobject) {
+ if (ib_copy_from_udata(&ucmd, udata, sizeof(ucmd))) {
+ dev_err(dev, "ib_copy_from_udata error for create qp\n");
+ ret = -EFAULT;
+ goto err_out;
+ }
+
+ ret = hns_roce_set_user_sq_size(hr_dev, hr_qp, &ucmd);
+ if (ret) {
+ dev_err(dev, "hns_roce_set_user_sq_size error for create qp\n");
+ goto err_out;
+ }
+
+ hr_qp->umem = ib_umem_get(ib_pd->uobject->context,
+ ucmd.buf_addr, hr_qp->buff_size, 0,
+ 0);
+ if (IS_ERR(hr_qp->umem)) {
+ dev_err(dev, "ib_umem_get error for create qp\n");
+ ret = PTR_ERR(hr_qp->umem);
+ goto err_out;
+ }
+
+ ret = hns_roce_mtt_init(hr_dev, ib_umem_page_count(hr_qp->umem),
+ ilog2((unsigned int)hr_qp->umem->page_size),
+ &hr_qp->mtt);
+ if (ret) {
+ dev_err(dev, "hns_roce_mtt_init error for create qp\n");
+ goto err_buf;
+ }
+
+ ret = hns_roce_ib_umem_write_mtt(hr_dev, &hr_qp->mtt,
+ hr_qp->umem);
+ if (ret) {
+ dev_err(dev, "hns_roce_ib_umem_write_mtt error for create qp\n");
+ goto err_mtt;
+ }
+ } else {
+ if (init_attr->create_flags &
+ IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK) {
+ dev_err(dev, "init_attr->create_flags error!\n");
+ ret = -EINVAL;
+ goto err_out;
+ }
+
+ if (init_attr->create_flags & IB_QP_CREATE_IPOIB_UD_LSO) {
+ dev_err(dev, "init_attr->create_flags error!\n");
+ ret = -EINVAL;
+ goto err_out;
+ }
+
+ /* Set SQ size */
+ ret = hns_roce_set_kernel_sq_size(hr_dev, &init_attr->cap,
+ init_attr->qp_type, hr_qp);
+ if (ret) {
+ dev_err(dev, "hns_roce_set_kernel_sq_size error!\n");
+ goto err_out;
+ }
+
+ /* QP doorbell register address */
+ hr_qp->sq.db_reg_l = hr_dev->reg_base + ROCEE_DB_SQ_L_0_REG +
+ DB_REG_OFFSET * hr_dev->priv_uar.index;
+ hr_qp->rq.db_reg_l = hr_dev->reg_base +
+ ROCEE_DB_OTHERS_L_0_REG +
+ DB_REG_OFFSET * hr_dev->priv_uar.index;
+
+ /* Allocate QP buf */
+ if (hns_roce_buf_alloc(hr_dev, hr_qp->buff_size, PAGE_SIZE * 2,
+ &hr_qp->hr_buf)) {
+ dev_err(dev, "hns_roce_buf_alloc error!\n");
+ ret = -ENOMEM;
+ goto err_out;
+ }
+
+ /* Write MTT */
+ ret = hns_roce_mtt_init(hr_dev, hr_qp->hr_buf.npages,
+ hr_qp->hr_buf.page_shift, &hr_qp->mtt);
+ if (ret) {
+ dev_err(dev, "hns_roce_mtt_init error for kernel create qp\n");
+ goto err_buf;
+ }
+
+ ret = hns_roce_buf_write_mtt(hr_dev, &hr_qp->mtt,
+ &hr_qp->hr_buf);
+ if (ret) {
+ dev_err(dev, "hns_roce_buf_write_mtt error for kernel create qp\n");
+ goto err_mtt;
+ }
+
+ hr_qp->sq.wrid = kmalloc_array(hr_qp->sq.wqe_cnt, sizeof(u64),
+ GFP_KERNEL);
+ hr_qp->rq.wrid = kmalloc_array(hr_qp->rq.wqe_cnt, sizeof(u64),
+ GFP_KERNEL);
+ if (!hr_qp->sq.wrid || !hr_qp->rq.wrid) {
+ ret = -ENOMEM;
+ dev_err(dev, "wrid buf alloc failed!\n");
+ goto err_wrid;
+ }
+ }
+
+ if (sqpn) {
+ qpn = sqpn;
+ } else {
+ /* Get QPN */
+ ret = hns_roce_reserve_range_qp(hr_dev, 1, 1, &qpn);
+ if (ret) {
+ dev_err(dev, "hns_roce_reserve_range_qp alloc qpn error\n");
+ goto err_wrid;
+ }
+ }
+
+ if ((init_attr->qp_type) == IB_QPT_GSI) {
+ ret = hns_roce_gsi_qp_alloc(hr_dev, qpn, hr_qp);
+ if (ret) {
+ dev_err(dev, "hns_roce_qp_alloc failed!\n");
+ goto err_qpn;
+ }
+ } else {
+ ret = hns_roce_qp_alloc(hr_dev, qpn, hr_qp);
+ if (ret) {
+ dev_err(dev, "hns_roce_qp_alloc failed!\n");
+ goto err_qpn;
+ }
+ }
+
+ if (sqpn)
+ hr_qp->doorbell_qpn = 1;
+ else
+ hr_qp->doorbell_qpn = cpu_to_le64(hr_qp->qpn);
+
+ hr_qp->event = hns_roce_ib_qp_event;
+
+ return 0;
+
+err_qpn:
+ if (!sqpn)
+ hns_roce_release_range_qp(hr_dev, qpn, 1);
+
+err_wrid:
+ kfree(hr_qp->sq.wrid);
+ kfree(hr_qp->rq.wrid);
+
+err_mtt:
+ hns_roce_mtt_cleanup(hr_dev, &hr_qp->mtt);
+
+err_buf:
+ if (ib_pd->uobject)
+ ib_umem_release(hr_qp->umem);
+ else
+ hns_roce_buf_free(hr_dev, hr_qp->buff_size, &hr_qp->hr_buf);
+
+err_out:
+ return ret;
+}
+
+struct ib_qp *hns_roce_create_qp(struct ib_pd *pd,
+ struct ib_qp_init_attr *init_attr,
+ struct ib_udata *udata)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(pd->device);
+ struct device *dev = &hr_dev->pdev->dev;
+ struct hns_roce_sqp *hr_sqp;
+ struct hns_roce_qp *hr_qp;
+ int ret;
+
+ switch (init_attr->qp_type) {
+ case IB_QPT_RC: {
+ hr_qp = kzalloc(sizeof(*hr_qp), GFP_KERNEL);
+ if (!hr_qp) {
+ dev_err(dev, "hr_qp alloc failed!\n");
+ return ERR_PTR(-ENOMEM);
+ }
+
+ ret = hns_roce_create_qp_common(hr_dev, pd, init_attr, udata, 0,
+ hr_qp);
+ if (ret) {
+ dev_err(dev, "Create RC QP failed\n");
+ kfree(hr_qp);
+ return ERR_PTR(ret);
+ }
+
+ hr_qp->ibqp.qp_num = hr_qp->qpn;
+
+ break;
+ }
+ case IB_QPT_GSI: {
+ /* Userspace is not allowed to create special QPs: */
+ if (pd->uobject) {
+ dev_err(dev, "not support usr space GSI\n");
+ return ERR_PTR(-EINVAL);
+ }
+
+ hr_sqp = kzalloc(sizeof(*hr_sqp), GFP_KERNEL);
+ if (!hr_sqp)
+ return ERR_PTR(-ENOMEM);
+
+ hr_qp = &hr_sqp->hr_qp;
+
+ ret = hns_roce_create_qp_common(hr_dev, pd, init_attr, udata,
+ hr_dev->caps.sqp_start +
+ hr_dev->caps.num_ports +
+ init_attr->port_num - 1, hr_qp);
+ if (ret) {
+ dev_err(dev, "Create GSI QP failed!\n");
+ kfree(hr_sqp);
+ return ERR_PTR(ret);
+ }
+
+ hr_qp->port = (init_attr->port_num - 1);
+ hr_qp->ibqp.qp_num = hr_dev->caps.sqp_start +
+ hr_dev->caps.num_ports +
+ init_attr->port_num - 1;
+ break;
+ }
+ default:{
+ dev_err(dev, "not support QP type %d\n", init_attr->qp_type);
+ return ERR_PTR(-EINVAL);
+ }
+ }
+
+ return &hr_qp->ibqp;
+}
+
+int to_hr_qp_type(int qp_type)
+{
+ int transport_type;
+
+ if (qp_type == IB_QPT_RC)
+ transport_type = SERV_TYPE_RC;
+ else if (qp_type == IB_QPT_UC)
+ transport_type = SERV_TYPE_UC;
+ else if (qp_type == IB_QPT_UD)
+ transport_type = SERV_TYPE_UD;
+ else if (qp_type == IB_QPT_GSI)
+ transport_type = SERV_TYPE_UD;
+ else
+ transport_type = -1;
+
+ return transport_type;
+}
+
+int hns_roce_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
+ int attr_mask, struct ib_udata *udata)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
+ struct hns_roce_qp *hr_qp = to_hr_qp(ibqp);
+ enum ib_qp_state cur_state, new_state;
+ struct device *dev = &hr_dev->pdev->dev;
+ int ret = -EINVAL;
+ int p;
+
+ mutex_lock(&hr_qp->mutex);
+
+ cur_state = attr_mask & IB_QP_CUR_STATE ?
+ attr->cur_qp_state : (enum ib_qp_state)hr_qp->state;
+ new_state = attr_mask & IB_QP_STATE ?
+ attr->qp_state : cur_state;
+
+ if (!ib_modify_qp_is_ok(cur_state, new_state, ibqp->qp_type, attr_mask,
+ IB_LINK_LAYER_ETHERNET)) {
+ dev_err(dev, "ib_modify_qp_is_ok failed\n");
+ goto out;
+ }
+
+ if ((attr_mask & IB_QP_PORT) &&
+ (attr->port_num == 0 || attr->port_num > hr_dev->caps.num_ports)) {
+ dev_err(dev, "attr port_num invalid.attr->port_num=%d\n",
+ attr->port_num);
+ goto out;
+ }
+
+ if (attr_mask & IB_QP_PKEY_INDEX) {
+ p = attr_mask & IB_QP_PORT ? attr->port_num : (hr_qp->port + 1);
+ if (attr->pkey_index >= hr_dev->caps.pkey_table_len[p]) {
+ dev_dbg(dev, "attr pkey_index invalid.attr->pkey_index=%d\n",
+ attr->pkey_index);
+ goto out;
+ }
+ }
+
+ if (attr_mask & IB_QP_MAX_QP_RD_ATOMIC &&
+ attr->max_rd_atomic > hr_dev->caps.max_qp_init_rdma) {
+ dev_dbg(dev, "attr max_rd_atomic invalid.attr->max_rd_atomic=%d\n",
+ attr->max_rd_atomic);
+ goto out;
+ }
+
+ if (attr_mask & IB_QP_MAX_DEST_RD_ATOMIC &&
+ attr->max_dest_rd_atomic > hr_dev->caps.max_qp_dest_rdma) {
+ dev_dbg(dev, "attr max_dest_rd_atomic invalid.attr->max_dest_rd_atomic=%d\n",
+ attr->max_dest_rd_atomic);
+ goto out;
+ }
+
+ if (cur_state == new_state && cur_state == IB_QPS_RESET) {
+ ret = -EPERM;
+ dev_dbg(dev, "cur_state=%d new_state=%d\n", cur_state,
+ new_state);
+ goto out;
+ }
+
+ ret = hr_dev->hw->modify_qp(ibqp, attr, attr_mask, cur_state,
+ new_state);
+
+out:
+ mutex_unlock(&hr_qp->mutex);
+
+ return ret;
+}
+
+void hns_roce_lock_cqs(struct hns_roce_cq *send_cq, struct hns_roce_cq *recv_cq)
+ __acquires(&send_cq->lock) __acquires(&recv_cq->lock)
+{
+ if (send_cq == recv_cq) {
+ spin_lock_irq(&send_cq->lock);
+ __acquire(&recv_cq->lock);
+ } else if (send_cq->cqn < recv_cq->cqn) {
+ spin_lock_irq(&send_cq->lock);
+ spin_lock_nested(&recv_cq->lock, SINGLE_DEPTH_NESTING);
+ } else {
+ spin_lock_irq(&recv_cq->lock);
+ spin_lock_nested(&send_cq->lock, SINGLE_DEPTH_NESTING);
+ }
+}
+
+void hns_roce_unlock_cqs(struct hns_roce_cq *send_cq,
+ struct hns_roce_cq *recv_cq) __releases(&send_cq->lock)
+ __releases(&recv_cq->lock)
+{
+ if (send_cq == recv_cq) {
+ __release(&recv_cq->lock);
+ spin_unlock_irq(&send_cq->lock);
+ } else if (send_cq->cqn < recv_cq->cqn) {
+ spin_unlock(&recv_cq->lock);
+ spin_unlock_irq(&send_cq->lock);
+ } else {
+ spin_unlock(&send_cq->lock);
+ spin_unlock_irq(&recv_cq->lock);
+ }
+}
+
+__be32 send_ieth(struct ib_send_wr *wr)
+{
+ switch (wr->opcode) {
+ case IB_WR_SEND_WITH_IMM:
+ case IB_WR_RDMA_WRITE_WITH_IMM:
+ return cpu_to_le32(wr->ex.imm_data);
+ case IB_WR_SEND_WITH_INV:
+ return cpu_to_le32(wr->ex.invalidate_rkey);
+ default:
+ return 0;
+ }
+}
+
+static void *get_wqe(struct hns_roce_qp *hr_qp, int offset)
+{
+
+ return hns_roce_buf_offset(&hr_qp->hr_buf, offset);
+}
+
+void *get_recv_wqe(struct hns_roce_qp *hr_qp, int n)
+{
+ struct ib_qp *ibqp = &hr_qp->ibqp;
+ struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
+
+ if ((n < 0) || (n > hr_qp->rq.wqe_cnt)) {
+ dev_err(&hr_dev->pdev->dev, "rq wqe index:%d,rq wqe cnt:%d\r\n",
+ n, hr_qp->rq.wqe_cnt);
+ return NULL;
+ }
+
+ return get_wqe(hr_qp, hr_qp->rq.offset + (n << hr_qp->rq.wqe_shift));
+}
+
+void *get_send_wqe(struct hns_roce_qp *hr_qp, int n)
+{
+ struct ib_qp *ibqp = &hr_qp->ibqp;
+ struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
+
+ if ((n < 0) || (n > hr_qp->sq.wqe_cnt)) {
+ dev_err(&hr_dev->pdev->dev, "sq wqe index:%d,sq wqe cnt:%d\r\n",
+ n, hr_qp->sq.wqe_cnt);
+ return NULL;
+ }
+
+ return get_wqe(hr_qp, hr_qp->sq.offset + (n << hr_qp->sq.wqe_shift));
+}
+
+bool hns_roce_wq_overflow(struct hns_roce_wq *hr_wq, int nreq,
+ struct ib_cq *ib_cq)
+{
+ struct hns_roce_cq *hr_cq;
+ u32 cur;
+
+ cur = hr_wq->head - hr_wq->tail;
+ if (likely(cur + nreq < hr_wq->max_post))
+ return 0;
+
+ hr_cq = to_hr_cq(ib_cq);
+ spin_lock(&hr_cq->lock);
+ cur = hr_wq->head - hr_wq->tail;
+ spin_unlock(&hr_cq->lock);
+
+ return cur + nreq >= hr_wq->max_post;
+}
+
int hns_roce_init_qp_table(struct hns_roce_dev *hr_dev)
{
struct hns_roce_qp_table *qp_table = &hr_dev->qp_table;
diff --git a/drivers/infiniband/hw/hns/hns_roce_user.h b/drivers/infiniband/hw/hns/hns_roce_user.h
index 457e77a..73c3f0c 100644
--- a/drivers/infiniband/hw/hns/hns_roce_user.h
+++ b/drivers/infiniband/hw/hns/hns_roce_user.h
@@ -10,6 +10,12 @@
#ifndef _HNS_ROCE_USER_H
#define _HNS_ROCE_USER_H
+struct hns_roce_ib_create_qp {
+ __u64 buf_addr;
+ __u8 log_sq_bb_count;
+ __u8 log_sq_stride;
+};
+
struct hns_roce_ib_alloc_ucontext_resp {
__u32 qp_tab_size;
};
--
1.9.1
^ permalink raw reply related
* [PATCH v6 16/21] IB/hns: Add ah operations support
From: Lijun Ou @ 2016-04-28 12:09 UTC (permalink / raw)
To: dledford, sean.hefty, hal.rosenstock, davem, jeffrey.t.kirsher,
jiri, ogerlitz
Cc: linux-rdma, linux-kernel, netdev, gongyangming, xiaokun,
tangchaofei, oulijun, haifeng.wei, yisen.zhuang, yankejian,
charles.chenxin, linuxarm
In-Reply-To: <1461845396-61306-1-git-send-email-oulijun@huawei.com>
This patch was for implementing of address handle operations.
It includes three verbs that create ah, query ah and destroy
ah. They is completed independently by RoCE driver.
Signed-off-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Nenglong Zhao <zhaonenglong@hisilicon.com>
Signed-off-by: Lijun Ou <oulijun@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_ah.c | 109 ++++++++++++++++++++++++++++
drivers/infiniband/hw/hns/hns_roce_device.h | 31 +++++++-
drivers/infiniband/hw/hns/hns_roce_main.c | 5 ++
3 files changed, 144 insertions(+), 1 deletion(-)
create mode 100644 drivers/infiniband/hw/hns/hns_roce_ah.c
diff --git a/drivers/infiniband/hw/hns/hns_roce_ah.c b/drivers/infiniband/hw/hns/hns_roce_ah.c
new file mode 100644
index 0000000..9d0eb61
--- /dev/null
+++ b/drivers/infiniband/hw/hns/hns_roce_ah.c
@@ -0,0 +1,109 @@
+/*
+ * Copyright (c) 2016 Hisilicon Limited.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#include <linux/inet.h>
+#include <linux/init.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include <rdma/ib_addr.h>
+#include <rdma/ib_cache.h>
+#include "hns_roce_common.h"
+#include "hns_roce_device.h"
+
+#define HNS_ROCE_PORT_NUM_SHIFT 24
+#define HNS_ROCE_VLAN_SL_BIT_MASK 7
+#define HNS_ROCE_VLAN_SL_SHIFT 13
+
+struct ib_ah *hns_roce_create_ah(struct ib_pd *ibpd, struct ib_ah_attr *ah_attr)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(ibpd->device);
+ struct device *dev = &hr_dev->pdev->dev;
+ struct ib_gid_attr gid_attr;
+ struct hns_roce_ah *ah;
+ u16 vlan_tag = 0xffff;
+ struct in6_addr in6;
+ union ib_gid sgid;
+ int ret;
+
+ ah = kzalloc(sizeof(*ah), GFP_ATOMIC);
+ if (!ah)
+ return ERR_PTR(-ENOMEM);
+
+ /* Get mac address */
+ memcpy(&in6, ah_attr->grh.dgid.raw, sizeof(ah_attr->grh.dgid.raw));
+ if (rdma_is_multicast_addr(&in6))
+ rdma_get_mcast_mac(&in6, ah->av.mac);
+ else
+ memcpy(ah->av.mac, ah_attr->dmac, sizeof(ah_attr->dmac));
+
+ /* Get source gid */
+ ret = ib_get_cached_gid(ibpd->device, ah_attr->port_num,
+ ah_attr->grh.sgid_index, &sgid, &gid_attr);
+ if (ret) {
+ dev_err(dev, "get sgid failed! ret = %d\n", ret);
+ kfree(ah);
+ return ERR_PTR(ret);
+ }
+
+ if (gid_attr.ndev) {
+ if (is_vlan_dev(gid_attr.ndev))
+ vlan_tag = vlan_dev_vlan_id(gid_attr.ndev);
+ dev_put(gid_attr.ndev);
+ }
+
+ if (vlan_tag < 0x1000)
+ vlan_tag |= (ah_attr->sl & HNS_ROCE_VLAN_SL_BIT_MASK) <<
+ HNS_ROCE_VLAN_SL_SHIFT;
+
+ ah->av.port_pd = cpu_to_be32(to_hr_pd(ibpd)->pdn | (ah_attr->port_num <<
+ HNS_ROCE_PORT_NUM_SHIFT));
+ ah->av.gid_index = ah_attr->grh.sgid_index;
+ ah->av.vlan = cpu_to_le16(vlan_tag);
+ dev_dbg(dev, "gid_index = 0x%x,vlan = 0x%x\n", ah->av.gid_index,
+ ah->av.vlan);
+
+ if (ah_attr->static_rate)
+ ah->av.stat_rate = IB_RATE_10_GBPS;
+
+ memcpy(ah->av.dgid, ah_attr->grh.dgid.raw, HNS_ROCE_GID_SIZE);
+ ah->av.sl_tclass_flowlabel = cpu_to_le32(ah_attr->sl <<
+ HNS_ROCE_SL_SHIFT);
+
+ return &ah->ibah;
+}
+
+int hns_roce_query_ah(struct ib_ah *ibah, struct ib_ah_attr *ah_attr)
+{
+ struct hns_roce_ah *ah = to_hr_ah(ibah);
+
+ memset(ah_attr, 0, sizeof(*ah_attr));
+
+ ah_attr->sl = le32_to_cpu(ah->av.sl_tclass_flowlabel) >>
+ HNS_ROCE_SL_SHIFT;
+ ah_attr->port_num = le32_to_cpu(ah->av.port_pd) >>
+ HNS_ROCE_PORT_NUM_SHIFT;
+ ah_attr->static_rate = ah->av.stat_rate;
+ ah_attr->ah_flags = IB_AH_GRH;
+ ah_attr->grh.traffic_class = le32_to_cpu(ah->av.sl_tclass_flowlabel) >>
+ HNS_ROCE_TCLASS_SHIFT;
+ ah_attr->grh.flow_label = le32_to_cpu(ah->av.sl_tclass_flowlabel) &
+ HNS_ROCE_FLOW_LABLE_MASK;
+ ah_attr->grh.hop_limit = ah->av.hop_limit;
+ ah_attr->grh.sgid_index = ah->av.gid_index;
+ memcpy(ah_attr->grh.dgid.raw, ah->av.dgid, HNS_ROCE_GID_SIZE);
+
+ return 0;
+}
+
+int hns_roce_destroy_ah(struct ib_ah *ah)
+{
+ kfree(to_hr_ah(ah));
+
+ return 0;
+}
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index 4640c34..df2bc00 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -24,7 +24,6 @@
#define MAC_ADDR_OCTET_NUM 6
#define HNS_ROCE_MAX_MSG_LEN 0x80000000
-
#define HNS_ROCE_BA_SIZE (32 * 4096)
#define HNS_ROCE_MAX_IRQ_NUM 34
@@ -35,6 +34,11 @@
#define HNS_ROCE_AEQE_VEC_NUM 1
#define HNS_ROCE_AEQE_OF_VEC_NUM 1
+/* 4G/4K = 1M */
+#define HNS_ROCE_SL_SHIFT 29
+#define HNS_ROCE_TCLASS_SHIFT 20
+#define HNS_ROCE_FLOW_LABLE_MASK 0xfffff
+
#define HNS_ROCE_MAX_PORTS 6
#define HNS_ROCE_MAX_GID_NUM 16
#define HNS_ROCE_GID_SIZE 16
@@ -208,6 +212,22 @@ struct hns_roce_raq_table {
struct hns_roce_buf_list *e_raq_buf;
};
+struct hns_roce_av {
+ __le32 port_pd;
+ u8 gid_index;
+ u8 stat_rate;
+ u8 hop_limit;
+ __le32 sl_tclass_flowlabel;
+ u8 dgid[HNS_ROCE_GID_SIZE];
+ u8 mac[6];
+ __le16 vlan;
+};
+
+struct hns_roce_ah {
+ struct ib_ah ibah;
+ struct hns_roce_av av;
+};
+
struct hns_roce_cmd_context {
struct completion done;
int result;
@@ -394,6 +414,11 @@ static inline struct hns_roce_pd *to_hr_pd(struct ib_pd *ibpd)
return container_of(ibpd, struct hns_roce_pd, ibpd);
}
+static inline struct hns_roce_ah *to_hr_ah(struct ib_ah *ibah)
+{
+ return container_of(ibah, struct hns_roce_ah, ibah);
+}
+
static inline void hns_roce_write64_k(__be32 val[2], void __iomem *dest)
{
__raw_writeq(*(u64 *) val, dest);
@@ -441,6 +466,10 @@ int hns_roce_bitmap_alloc_range(struct hns_roce_bitmap *bitmap, int cnt,
void hns_roce_bitmap_free_range(struct hns_roce_bitmap *bitmap,
unsigned long obj, int cnt);
+struct ib_ah *hns_roce_create_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr);
+int hns_roce_query_ah(struct ib_ah *ibah, struct ib_ah_attr *ah_attr);
+int hns_roce_destroy_ah(struct ib_ah *ah);
+
struct ib_pd *hns_roce_alloc_pd(struct ib_device *ib_dev,
struct ib_ucontext *context,
struct ib_udata *udata);
diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
index 2352120..7eb88bb 100644
--- a/drivers/infiniband/hw/hns/hns_roce_main.c
+++ b/drivers/infiniband/hw/hns/hns_roce_main.c
@@ -605,6 +605,11 @@ int hns_roce_register_device(struct hns_roce_dev *hr_dev)
ib_dev->alloc_pd = hns_roce_alloc_pd;
ib_dev->dealloc_pd = hns_roce_dealloc_pd;
+ /* AH */
+ ib_dev->create_ah = hns_roce_create_ah;
+ ib_dev->query_ah = hns_roce_query_ah;
+ ib_dev->destroy_ah = hns_roce_destroy_ah;
+
ret = ib_register_device(ib_dev, NULL);
if (ret) {
dev_err(dev, "ib_register_device failed!\n");
--
1.9.1
^ permalink raw reply related
* [PATCH v6 15/21] IB/hns: Add PD operations support
From: Lijun Ou @ 2016-04-28 12:09 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA,
sean.hefty-ral2JQCrhuEAvxtiuMwx3w,
hal.rosenstock-Re5JQEeQqe8AvxtiuMwx3w,
davem-fT/PcQaiUtIeIZ0/mPfg9Q,
jeffrey.t.kirsher-ral2JQCrhuEAvxtiuMwx3w,
jiri-VPRAkNaXOzVWk0Htik3J/w, ogerlitz-VPRAkNaXOzVWk0Htik3J/w
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
netdev-u79uwXL29TY76Z2rM5mHXA,
gongyangming-hv44wF8Li93QT0dZR+AlfA,
xiaokun-hv44wF8Li93QT0dZR+AlfA,
tangchaofei-hv44wF8Li93QT0dZR+AlfA,
oulijun-hv44wF8Li93QT0dZR+AlfA,
haifeng.wei-hv44wF8Li93QT0dZR+AlfA,
yisen.zhuang-hv44wF8Li93QT0dZR+AlfA,
yankejian-hv44wF8Li93QT0dZR+AlfA,
charles.chenxin-hv44wF8Li93QT0dZR+AlfA,
linuxarm-hv44wF8Li93QT0dZR+AlfA
In-Reply-To: <1461845396-61306-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
This patch added the verbs to operate PD. It mainly includes
the functions of allocating PD and deallocating PD.
Signed-off-by: Wei Hu <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Nenglong Zhao <zhaonenglong-C8/M+/jPZTeaMJb+Lgu22Q@public.gmane.org>
Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
drivers/infiniband/hw/hns/hns_roce_device.h | 17 ++++++++
drivers/infiniband/hw/hns/hns_roce_main.c | 8 +++-
drivers/infiniband/hw/hns/hns_roce_pd.c | 62 +++++++++++++++++++++++++++++
3 files changed, 86 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index a3ad4a9..4640c34 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -116,6 +116,11 @@ struct hns_roce_ucontext {
struct hns_roce_uar uar;
};
+struct hns_roce_pd {
+ struct ib_pd ibpd;
+ unsigned long pdn;
+};
+
struct hns_roce_bitmap {
/* Bitmap Traversal last a bit which is 1 */
unsigned long last;
@@ -384,6 +389,11 @@ static inline struct hns_roce_ucontext
return container_of(ibucontext, struct hns_roce_ucontext, ibucontext);
}
+static inline struct hns_roce_pd *to_hr_pd(struct ib_pd *ibpd)
+{
+ return container_of(ibpd, struct hns_roce_pd, ibpd);
+}
+
static inline void hns_roce_write64_k(__be32 val[2], void __iomem *dest)
{
__raw_writeq(*(u64 *) val, dest);
@@ -431,6 +441,13 @@ int hns_roce_bitmap_alloc_range(struct hns_roce_bitmap *bitmap, int cnt,
void hns_roce_bitmap_free_range(struct hns_roce_bitmap *bitmap,
unsigned long obj, int cnt);
+struct ib_pd *hns_roce_alloc_pd(struct ib_device *ib_dev,
+ struct ib_ucontext *context,
+ struct ib_udata *udata);
+int hns_roce_pd_alloc(struct hns_roce_dev *hr_dev, unsigned long *pdn);
+void hns_roce_pd_free(struct hns_roce_dev *hr_dev, unsigned long pdn);
+int hns_roce_dealloc_pd(struct ib_pd *pd);
+
void hns_roce_cq_completion(struct hns_roce_dev *hr_dev, u32 cqn);
void hns_roce_cq_event(struct hns_roce_dev *hr_dev, u32 cqn, int event_type);
void hns_roce_qp_event(struct hns_roce_dev *hr_dev, u32 qpn, int event_type);
diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
index 4a39444..2352120 100644
--- a/drivers/infiniband/hw/hns/hns_roce_main.c
+++ b/drivers/infiniband/hw/hns/hns_roce_main.c
@@ -585,7 +585,9 @@ int hns_roce_register_device(struct hns_roce_dev *hr_dev)
ib_dev->uverbs_cmd_mask =
(1ULL << IB_USER_VERBS_CMD_GET_CONTEXT) |
(1ULL << IB_USER_VERBS_CMD_QUERY_DEVICE) |
- (1ULL << IB_USER_VERBS_CMD_QUERY_PORT);
+ (1ULL << IB_USER_VERBS_CMD_QUERY_PORT) |
+ (1ULL << IB_USER_VERBS_CMD_ALLOC_PD) |
+ (1ULL << IB_USER_VERBS_CMD_DEALLOC_PD);
/* HCA||device||port */
ib_dev->modify_device = hns_roce_modify_device;
@@ -599,6 +601,10 @@ int hns_roce_register_device(struct hns_roce_dev *hr_dev)
ib_dev->dealloc_ucontext = hns_roce_dealloc_ucontext;
ib_dev->mmap = hns_roce_mmap;
+ /* PD */
+ ib_dev->alloc_pd = hns_roce_alloc_pd;
+ ib_dev->dealloc_pd = hns_roce_dealloc_pd;
+
ret = ib_register_device(ib_dev, NULL);
if (ret) {
dev_err(dev, "ib_register_device failed!\n");
diff --git a/drivers/infiniband/hw/hns/hns_roce_pd.c b/drivers/infiniband/hw/hns/hns_roce_pd.c
index 90b0f33..6485378 100644
--- a/drivers/infiniband/hw/hns/hns_roce_pd.c
+++ b/drivers/infiniband/hw/hns/hns_roce_pd.c
@@ -17,6 +17,28 @@
#include "hns_roce_common.h"
#include "hns_roce_device.h"
+int hns_roce_pd_alloc(struct hns_roce_dev *hr_dev, unsigned long *pdn)
+{
+ struct device *dev = &hr_dev->pdev->dev;
+ unsigned long pd_number;
+ int ret = 0;
+
+ ret = hns_roce_bitmap_alloc(&hr_dev->pd_bitmap, &pd_number);
+ if (ret == -1) {
+ dev_err(dev, "alloc pdn from pdbitmap failed\n");
+ return -ENOMEM;
+ }
+
+ *pdn = pd_number;
+
+ return 0;
+}
+
+void hns_roce_pd_free(struct hns_roce_dev *hr_dev, unsigned long pdn)
+{
+ hns_roce_bitmap_free(&hr_dev->pd_bitmap, pdn);
+}
+
int hns_roce_init_pd_table(struct hns_roce_dev *hr_dev)
{
return hns_roce_bitmap_init(&hr_dev->pd_bitmap, hr_dev->caps.num_pds,
@@ -29,6 +51,46 @@ void hns_roce_cleanup_pd_table(struct hns_roce_dev *hr_dev)
hns_roce_bitmap_cleanup(&hr_dev->pd_bitmap);
}
+struct ib_pd *hns_roce_alloc_pd(struct ib_device *ib_dev,
+ struct ib_ucontext *context,
+ struct ib_udata *udata)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(ib_dev);
+ struct device *dev = &hr_dev->pdev->dev;
+ struct hns_roce_pd *pd;
+ int ret;
+
+ pd = kmalloc(sizeof(*pd), GFP_KERNEL);
+ if (!pd)
+ return ERR_PTR(-ENOMEM);
+
+ ret = hns_roce_pd_alloc(to_hr_dev(ib_dev), &pd->pdn);
+ if (ret) {
+ kfree(pd);
+ dev_err(dev, "[alloc_pd]hns_roce_pd_alloc failed!\n");
+ return ERR_PTR(ret);
+ }
+
+ if (context) {
+ if (ib_copy_to_udata(udata, &pd->pdn, sizeof(u64))) {
+ hns_roce_pd_free(to_hr_dev(ib_dev), pd->pdn);
+ dev_err(dev, "[alloc_pd]ib_copy_to_udata failed!\n");
+ kfree(pd);
+ return ERR_PTR(-EFAULT);
+ }
+ }
+
+ return &pd->ibpd;
+}
+
+int hns_roce_dealloc_pd(struct ib_pd *pd)
+{
+ hns_roce_pd_free(to_hr_dev(pd->device), to_hr_pd(pd)->pdn);
+ kfree(to_hr_pd(pd));
+
+ return 0;
+}
+
int hns_roce_uar_alloc(struct hns_roce_dev *hr_dev, struct hns_roce_uar *uar)
{
struct resource *res;
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH v6 14/21] IB/hns: Add operations support for IB device and port
From: Lijun Ou @ 2016-04-28 12:09 UTC (permalink / raw)
To: dledford, sean.hefty, hal.rosenstock, davem, jeffrey.t.kirsher,
jiri, ogerlitz
Cc: linux-rdma, linux-kernel, netdev, gongyangming, xiaokun,
tangchaofei, oulijun, haifeng.wei, yisen.zhuang, yankejian,
charles.chenxin, linuxarm
In-Reply-To: <1461845396-61306-1-git-send-email-oulijun@huawei.com>
This patch mainly registered some relative verbs for the kernel.
These operation functions will be called by user. For example:
1. modify device
2. query device
3. query_port
4. modify_port
and so on.
Signed-off-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Nenglong Zhao <zhaonenglong@hisilicon.com>
Signed-off-by: Lijun Ou <oulijun@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_common.h | 4 +
drivers/infiniband/hw/hns/hns_roce_device.h | 21 +++
drivers/infiniband/hw/hns/hns_roce_main.c | 228 ++++++++++++++++++++++++++++
drivers/infiniband/hw/hns/hns_roce_user.h | 17 +++
4 files changed, 270 insertions(+)
create mode 100644 drivers/infiniband/hw/hns/hns_roce_user.h
diff --git a/drivers/infiniband/hw/hns/hns_roce_common.h b/drivers/infiniband/hw/hns/hns_roce_common.h
index 45b5804..135203e 100644
--- a/drivers/infiniband/hw/hns/hns_roce_common.h
+++ b/drivers/infiniband/hw/hns/hns_roce_common.h
@@ -10,6 +10,10 @@
#ifndef _HNS_ROCE_COMMON_H
#define _HNS_ROCE_COMMON_H
+#ifndef assert
+#define assert(cond)
+#endif
+
#define roce_writel(value, addr) writel((value), (addr))
#define roce_readl(addr) readl((addr))
#define roce_raw_write(value, addr) \
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index 240a801..a3ad4a9 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -22,6 +22,8 @@
#define DRV_NAME "hns_roce"
#define MAC_ADDR_OCTET_NUM 6
+#define HNS_ROCE_MAX_MSG_LEN 0x80000000
+
#define HNS_ROCE_BA_SIZE (32 * 4096)
@@ -35,7 +37,10 @@
#define HNS_ROCE_MAX_PORTS 6
#define HNS_ROCE_MAX_GID_NUM 16
+#define HNS_ROCE_GID_SIZE 16
+#define PKEY_ID 0xffff
+#define NODE_DESC_SIZE 64
#define ADDR_SHIFT_12 12
#define ADDR_SHIFT_32 32
#define ADDR_SHIFT_44 44
@@ -106,6 +111,11 @@ struct hns_roce_uar {
unsigned long index;
};
+struct hns_roce_ucontext {
+ struct ib_ucontext ibucontext;
+ struct hns_roce_uar uar;
+};
+
struct hns_roce_bitmap {
/* Bitmap Traversal last a bit which is 1 */
unsigned long last;
@@ -363,6 +373,17 @@ struct hns_roce_dev {
struct hns_roce_hw *hw;
};
+static inline struct hns_roce_dev *to_hr_dev(struct ib_device *ib_dev)
+{
+ return container_of(ib_dev, struct hns_roce_dev, ib_dev);
+}
+
+static inline struct hns_roce_ucontext
+ *to_hr_ucontext(struct ib_ucontext *ibucontext)
+{
+ return container_of(ibucontext, struct hns_roce_ucontext, ibucontext);
+}
+
static inline void hns_roce_write64_k(__be32 val[2], void __iomem *dest)
{
__raw_writeq(*(u64 *) val, dest);
diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
index c115cb3..4a39444 100644
--- a/drivers/infiniband/hw/hns/hns_roce_main.c
+++ b/drivers/infiniband/hw/hns/hns_roce_main.c
@@ -41,6 +41,7 @@
#include <rdma/ib_verbs.h>
#include "hns_roce_common.h"
#include "hns_roce_device.h"
+#include "hns_roce_user.h"
#include "hns_roce_icm.h"
/**
@@ -341,6 +342,217 @@ int hns_roce_setup_mtu_gids(struct hns_roce_dev *hr_dev)
return ret;
}
+static int hns_roce_query_device(struct ib_device *ib_dev,
+ struct ib_device_attr *props,
+ struct ib_udata *uhw)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(ib_dev);
+
+ memset(props, 0, sizeof(*props));
+
+ props->fw_ver = hr_dev->fw_ver;
+ props->sys_image_guid = hr_dev->sys_image_guid;
+ props->max_mr_size = (u64)(~(0ULL));
+ props->page_size_cap = hr_dev->caps.page_size_cap;
+ props->vendor_id = hr_dev->vendor_id;
+ props->vendor_part_id = hr_dev->vendor_part_id;
+ props->hw_ver = hr_dev->hw_rev;
+ props->max_qp = hr_dev->caps.num_qps;
+ props->max_qp_wr = hr_dev->caps.max_wqes;
+ props->device_cap_flags = IB_DEVICE_PORT_ACTIVE_EVENT |
+ IB_DEVICE_RC_RNR_NAK_GEN |
+ IB_DEVICE_LOCAL_DMA_LKEY;
+ props->max_sge = hr_dev->caps.max_sq_sg;
+ props->max_sge_rd = 1;
+ props->max_cq = hr_dev->caps.num_cqs;
+ props->max_cqe = hr_dev->caps.max_cqes;
+ props->max_mr = hr_dev->caps.num_mtpts;
+ props->max_pd = hr_dev->caps.num_pds;
+ props->max_qp_rd_atom = hr_dev->caps.max_qp_dest_rdma;
+ props->max_qp_init_rd_atom = hr_dev->caps.max_qp_init_rdma;
+ props->atomic_cap = IB_ATOMIC_NONE;
+ props->max_pkeys = 1;
+ props->local_ca_ack_delay = hr_dev->caps.local_ca_ack_delay;
+
+ return 0;
+}
+
+static int hns_roce_query_port(struct ib_device *ib_dev, u8 port_num,
+ struct ib_port_attr *props)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(ib_dev);
+ struct device *dev = &hr_dev->pdev->dev;
+ struct net_device *net_dev;
+ unsigned long flags;
+ enum ib_mtu mtu;
+ u8 port;
+
+ assert(port_num > 0);
+ port = port_num - 1;
+
+ memset(props, 0, sizeof(*props));
+
+ props->max_mtu = hr_dev->caps.max_mtu;
+ props->gid_tbl_len = hr_dev->caps.gid_table_len[port];
+ props->port_cap_flags = IB_PORT_CM_SUP | IB_PORT_REINIT_SUP |
+ IB_PORT_VENDOR_CLASS_SUP |
+ IB_PORT_BOOT_MGMT_SUP;
+ props->max_msg_sz = HNS_ROCE_MAX_MSG_LEN;
+ props->pkey_tbl_len = 1;
+ props->active_width = IB_WIDTH_4X;
+ props->active_speed = 1;
+
+ spin_lock_irqsave(&hr_dev->iboe.lock, flags);
+
+ net_dev = hr_dev->iboe.netdevs[port];
+ if (!net_dev) {
+ spin_unlock_irqrestore(&hr_dev->iboe.lock, flags);
+ dev_err(dev, "find netdev %d failed!\r\n", port);
+ return -EINVAL;
+ }
+
+ mtu = iboe_get_mtu(net_dev->mtu);
+ props->active_mtu = mtu ? min(props->max_mtu, mtu) : IB_MTU_256;
+ props->state = (netif_running(net_dev) && netif_carrier_ok(net_dev)) ?
+ IB_PORT_ACTIVE : IB_PORT_DOWN;
+ props->phys_state = (props->state == IB_PORT_ACTIVE) ? 5 : 3;
+
+ spin_unlock_irqrestore(&hr_dev->iboe.lock, flags);
+
+ return 0;
+}
+
+static enum rdma_link_layer hns_roce_get_link_layer(struct ib_device *device,
+ u8 port_num)
+{
+ return IB_LINK_LAYER_ETHERNET;
+}
+
+static int hns_roce_query_gid(struct ib_device *ib_dev, u8 port_num, int index,
+ union ib_gid *gid)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(ib_dev);
+ struct device *dev = &hr_dev->pdev->dev;
+ u8 gid_idx = 0;
+ u8 port;
+
+ if (port_num < 1 || port_num > hr_dev->caps.num_ports ||
+ index >= hr_dev->caps.gid_table_len[port_num - 1]) {
+ dev_err(dev,
+ "port_num %d index %d illegal! correct range: port_num 1~%d index 0~%d!\n",
+ port_num, index, hr_dev->caps.num_ports,
+ hr_dev->caps.gid_table_len[port_num - 1] - 1);
+ return -EINVAL;
+ }
+
+ port = port_num - 1;
+ gid_idx = hns_get_gid_index(hr_dev, port, index);
+ if (gid_idx >= HNS_ROCE_MAX_GID_NUM) {
+ dev_err(dev, "port_num %d index %d illegal! total gid num %d!\n",
+ port_num, index, HNS_ROCE_MAX_GID_NUM);
+ return -EINVAL;
+ }
+
+ memcpy(gid->raw, hr_dev->iboe.gid_table[gid_idx].raw,
+ HNS_ROCE_GID_SIZE);
+
+ return 0;
+}
+
+static int hns_roce_query_pkey(struct ib_device *ib_dev, u8 port, u16 index,
+ u16 *pkey)
+{
+ *pkey = PKEY_ID;
+
+ return 0;
+}
+
+static int hns_roce_modify_device(struct ib_device *ib_dev, int mask,
+ struct ib_device_modify *props)
+{
+ unsigned long flags;
+
+ if (mask & ~IB_DEVICE_MODIFY_NODE_DESC)
+ return -EOPNOTSUPP;
+
+ if (mask & IB_DEVICE_MODIFY_NODE_DESC) {
+ spin_lock_irqsave(&to_hr_dev(ib_dev)->sm_lock, flags);
+ memcpy(ib_dev->node_desc, props->node_desc, NODE_DESC_SIZE);
+ spin_unlock_irqrestore(&to_hr_dev(ib_dev)->sm_lock, flags);
+ }
+
+ return 0;
+}
+
+static int hns_roce_modify_port(struct ib_device *ib_dev, u8 port_num, int mask,
+ struct ib_port_modify *props)
+{
+ return 0;
+}
+
+static struct ib_ucontext *hns_roce_alloc_ucontext(struct ib_device *ib_dev,
+ struct ib_udata *udata)
+{
+ int ret = 0;
+ struct hns_roce_ucontext *context;
+ struct hns_roce_ib_alloc_ucontext_resp resp;
+ struct hns_roce_dev *hr_dev = to_hr_dev(ib_dev);
+
+ resp.qp_tab_size = hr_dev->caps.num_qps;
+
+ context = kmalloc(sizeof(*context), GFP_KERNEL);
+ if (!context)
+ return ERR_PTR(-ENOMEM);
+
+ ret = hns_roce_uar_alloc(hr_dev, &context->uar);
+ if (ret)
+ goto _error_fail_uar_alloc;
+
+ ret = ib_copy_to_udata(udata, &resp, sizeof(resp));
+ if (ret)
+ goto _error_fail_copy_to_udata;
+
+ return &context->ibucontext;
+
+_error_fail_copy_to_udata:
+hns_roce_uar_free(hr_dev, &context->uar);
+
+_error_fail_uar_alloc:
+ kfree(context);
+
+ return ERR_PTR(ret);
+}
+
+static int hns_roce_dealloc_ucontext(struct ib_ucontext *ibcontext)
+{
+ struct hns_roce_ucontext *context = to_hr_ucontext(ibcontext);
+
+ hns_roce_uar_free(to_hr_dev(ibcontext->device), &context->uar);
+ kfree(context);
+
+ return 0;
+}
+
+static int hns_roce_mmap(struct ib_ucontext *context,
+ struct vm_area_struct *vma)
+{
+ if (((vma->vm_end - vma->vm_start) % PAGE_SIZE) != 0)
+ return -EINVAL;
+
+ if (vma->vm_pgoff == 0) {
+ vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
+ if (io_remap_pfn_range(vma, vma->vm_start,
+ to_hr_ucontext(context)->uar.pfn,
+ PAGE_SIZE, vma->vm_page_prot))
+ return -EAGAIN;
+
+ } else {
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
void hns_roce_unregister_device(struct hns_roce_dev *hr_dev)
{
struct hns_roce_ib_iboe *iboe = &hr_dev->iboe;
@@ -370,6 +582,22 @@ int hns_roce_register_device(struct hns_roce_dev *hr_dev)
ib_dev->local_dma_lkey = hr_dev->caps.reserved_lkey;
ib_dev->num_comp_vectors = hr_dev->caps.num_comp_vectors;
ib_dev->uverbs_abi_ver = 1;
+ ib_dev->uverbs_cmd_mask =
+ (1ULL << IB_USER_VERBS_CMD_GET_CONTEXT) |
+ (1ULL << IB_USER_VERBS_CMD_QUERY_DEVICE) |
+ (1ULL << IB_USER_VERBS_CMD_QUERY_PORT);
+
+ /* HCA||device||port */
+ ib_dev->modify_device = hns_roce_modify_device;
+ ib_dev->query_device = hns_roce_query_device;
+ ib_dev->query_port = hns_roce_query_port;
+ ib_dev->modify_port = hns_roce_modify_port;
+ ib_dev->get_link_layer = hns_roce_get_link_layer;
+ ib_dev->query_gid = hns_roce_query_gid;
+ ib_dev->query_pkey = hns_roce_query_pkey;
+ ib_dev->alloc_ucontext = hns_roce_alloc_ucontext;
+ ib_dev->dealloc_ucontext = hns_roce_dealloc_ucontext;
+ ib_dev->mmap = hns_roce_mmap;
ret = ib_register_device(ib_dev, NULL);
if (ret) {
diff --git a/drivers/infiniband/hw/hns/hns_roce_user.h b/drivers/infiniband/hw/hns/hns_roce_user.h
new file mode 100644
index 0000000..457e77a
--- /dev/null
+++ b/drivers/infiniband/hw/hns/hns_roce_user.h
@@ -0,0 +1,17 @@
+/*
+ * Copyright (c) 2016 Hisilicon Limited.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#ifndef _HNS_ROCE_USER_H
+#define _HNS_ROCE_USER_H
+
+struct hns_roce_ib_alloc_ucontext_resp {
+ __u32 qp_tab_size;
+};
+
+#endif /*_HNS_ROCE_USER_H */
--
1.9.1
^ permalink raw reply related
* [PATCH v6 13/21] IB/hns: Add interface of the protocol stack registration
From: Lijun Ou @ 2016-04-28 12:09 UTC (permalink / raw)
To: dledford, sean.hefty, hal.rosenstock, davem, jeffrey.t.kirsher,
jiri, ogerlitz
Cc: linux-rdma, linux-kernel, netdev, gongyangming, xiaokun,
tangchaofei, oulijun, haifeng.wei, yisen.zhuang, yankejian,
charles.chenxin, linuxarm
In-Reply-To: <1461845396-61306-1-git-send-email-oulijun@huawei.com>
This patch mainly added the function module which netif notify
registered the protocol stack. It includes interface functions
as follows:
1. The executive called interface of RoCE when the netlink
event that registered protocol stack was generated
2. The executive called interface of RoCE when ip address
that registered protocol stack was changed.
In addition that, it will free the relative resource when RoCE
was removed.
Signed-off-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Nenglong Zhao <zhaonenglong@hisilicon.com>
Signed-off-by: Lijun Ou <oulijun@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_device.h | 3 +
drivers/infiniband/hw/hns/hns_roce_main.c | 211 +++++++++++++++++++++++++++-
2 files changed, 213 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index 06ca4fe..240a801 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -241,7 +241,10 @@ struct hns_roce_qp {
};
struct hns_roce_ib_iboe {
+ spinlock_t lock;
struct net_device *netdevs[HNS_ROCE_MAX_PORTS];
+ struct notifier_block nb;
+ struct notifier_block nb_inet;
/* 16 GID is shared by 6 port in v1 engine. */
union ib_gid gid_table[HNS_ROCE_MAX_GID_NUM];
u8 phy_port[HNS_ROCE_MAX_PORTS];
diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
index 0485e02..c115cb3 100644
--- a/drivers/infiniband/hw/hns/hns_roce_main.c
+++ b/drivers/infiniband/hw/hns/hns_roce_main.c
@@ -44,6 +44,46 @@
#include "hns_roce_icm.h"
/**
+ * hns_roce_addrconf_ifid_eui48 - Get default gid.
+ * @eui: eui.
+ * @vlan_id: gid
+ * @dev: net device
+ * Description:
+ * MAC convert to GID
+ * gid[0..7] = fe80 0000 0000 0000
+ * gid[8] = mac[0] ^ 2
+ * gid[9] = mac[1]
+ * gid[10] = mac[2]
+ * gid[11] = ff (VLAN ID high byte (4 MS bits))
+ * gid[12] = fe (VLAN ID low byte)
+ * gid[13] = mac[3]
+ * gid[14] = mac[4]
+ * gid[15] = mac[5]
+ */
+static void hns_roce_addrconf_ifid_eui48(u8 *eui, u16 vlan_id,
+ struct net_device *dev)
+{
+ memcpy(eui, dev->dev_addr, 3);
+ memcpy(eui + 5, dev->dev_addr + 3, 3);
+ if (vlan_id < 0x1000) {
+ eui[3] = vlan_id >> 8;
+ eui[4] = vlan_id & 0xff;
+ } else {
+ eui[3] = 0xff;
+ eui[4] = 0xfe;
+ }
+ eui[0] ^= 2;
+}
+
+void hns_roce_make_default_gid(struct net_device *dev, union ib_gid *gid)
+{
+ memset(gid, 0, sizeof(*gid));
+ gid->raw[0] = 0xFE;
+ gid->raw[1] = 0x80;
+ hns_roce_addrconf_ifid_eui48(&gid->raw[8], 0xffff, dev);
+}
+
+/**
* hns_get_gid_index - Get gid index.
* @hr_dev: pointer to structure hns_roce_dev.
* @port: port, value range: 0 ~ MAX
@@ -121,6 +161,152 @@ void hns_roce_update_gids(struct hns_roce_dev *hr_dev, int port)
ib_dispatch_event(&event);
}
+static int handle_en_event(struct hns_roce_dev *hr_dev, u8 port,
+ unsigned long event)
+{
+ struct device *dev = &hr_dev->pdev->dev;
+ struct net_device *netdev;
+ unsigned long flags;
+ union ib_gid gid;
+ int ret = 0;
+
+ netdev = hr_dev->iboe.netdevs[port];
+ if (!netdev) {
+ dev_err(dev, "port(%d) can't find netdev\n", port);
+ return -ENODEV;
+ }
+
+ spin_lock_irqsave(&hr_dev->iboe.lock, flags);
+
+ switch (event) {
+ case NETDEV_UP:
+ case NETDEV_CHANGE:
+ case NETDEV_REGISTER:
+ case NETDEV_CHANGEADDR:
+ hns_roce_set_mac(hr_dev, port, netdev->dev_addr);
+ hns_roce_make_default_gid(netdev, &gid);
+ ret = hns_roce_set_gid(hr_dev, port, 0, &gid);
+ if (!ret)
+ hns_roce_update_gids(hr_dev, port);
+ break;
+ case NETDEV_DOWN:
+ /*
+ * In v1 engine, only support all ports closed together.
+ */
+ break;
+ default:
+ dev_dbg(dev, "NETDEV event = 0x%x!\n", (u32)(event));
+ break;
+ }
+
+ spin_unlock_irqrestore(&hr_dev->iboe.lock, flags);
+ return ret;
+}
+
+static int hns_roce_netdev_event(struct notifier_block *self,
+ unsigned long event, void *ptr)
+{
+ struct net_device *dev = netdev_notifier_info_to_dev(ptr);
+ struct hns_roce_ib_iboe *iboe = NULL;
+ struct hns_roce_dev *hr_dev = NULL;
+ u8 port = 0;
+ int ret = 0;
+
+ hr_dev = container_of(self, struct hns_roce_dev, iboe.nb);
+ iboe = &hr_dev->iboe;
+
+ for (port = 0; port < hr_dev->caps.num_ports; port++) {
+ if (dev == iboe->netdevs[port]) {
+ ret = handle_en_event(hr_dev, port, event);
+ if (ret)
+ return NOTIFY_DONE;
+ break;
+ }
+ }
+
+ return NOTIFY_DONE;
+}
+
+static void hns_roce_addr_event(int event, struct net_device *event_netdev,
+ struct hns_roce_dev *hr_dev, union ib_gid *gid)
+{
+ struct hns_roce_ib_iboe *iboe = NULL;
+ int gid_table_len = 0;
+ unsigned long flags;
+ union ib_gid zgid;
+ u8 gid_idx = 0;
+ u8 port = 0;
+ int i = 0;
+ int free;
+ struct net_device *real_dev = rdma_vlan_dev_real_dev(event_netdev) ?
+ rdma_vlan_dev_real_dev(event_netdev) :
+ event_netdev;
+
+ if (event != NETDEV_UP && event != NETDEV_DOWN)
+ return;
+
+ iboe = &hr_dev->iboe;
+ while (port < hr_dev->caps.num_ports) {
+ if (real_dev == iboe->netdevs[port])
+ break;
+ port++;
+ }
+
+ if (port >= hr_dev->caps.num_ports) {
+ dev_dbg(&hr_dev->pdev->dev, "can't find netdev\n");
+ return;
+ }
+
+ memset(zgid.raw, 0, sizeof(zgid.raw));
+ free = -1;
+ gid_table_len = hr_dev->caps.gid_table_len[port];
+
+ spin_lock_irqsave(&hr_dev->iboe.lock, flags);
+
+ for (i = 0; i < gid_table_len; i++) {
+ gid_idx = hns_get_gid_index(hr_dev, port, i);
+ if (!memcmp(gid->raw, iboe->gid_table[gid_idx].raw,
+ sizeof(gid->raw)))
+ break;
+ if (free < 0 && !memcmp(zgid.raw,
+ iboe->gid_table[gid_idx].raw, sizeof(zgid.raw)))
+ free = i;
+ }
+
+ if (i >= gid_table_len) {
+ if (free < 0) {
+ spin_unlock_irqrestore(&hr_dev->iboe.lock, flags);
+ dev_dbg(&hr_dev->pdev->dev,
+ "gid_index overflow, port(%d)\n", port);
+ return;
+ }
+ if (!hns_roce_set_gid(hr_dev, port, free, gid))
+ hns_roce_update_gids(hr_dev, port);
+ } else if (event == NETDEV_DOWN) {
+ if (!hns_roce_set_gid(hr_dev, port, i, &zgid))
+ hns_roce_update_gids(hr_dev, port);
+ }
+
+ spin_unlock_irqrestore(&hr_dev->iboe.lock, flags);
+}
+
+static int hns_roce_inet_event(struct notifier_block *self, unsigned long event,
+ void *ptr)
+{
+ struct in_ifaddr *ifa = ptr;
+ struct hns_roce_dev *hr_dev;
+ struct net_device *dev = ifa->ifa_dev->dev;
+ union ib_gid gid;
+
+ ipv6_addr_set_v4mapped(ifa->ifa_address, (struct in6_addr *)&gid);
+
+ hr_dev = container_of(self, struct hns_roce_dev, iboe.nb_inet);
+
+ hns_roce_addr_event(event, dev, hr_dev, &gid);
+
+ return NOTIFY_DONE;
+}
+
int hns_roce_setup_mtu_gids(struct hns_roce_dev *hr_dev)
{
struct in_ifaddr *ifa_list = NULL;
@@ -157,6 +343,10 @@ int hns_roce_setup_mtu_gids(struct hns_roce_dev *hr_dev)
void hns_roce_unregister_device(struct hns_roce_dev *hr_dev)
{
+ struct hns_roce_ib_iboe *iboe = &hr_dev->iboe;
+
+ unregister_inetaddr_notifier(&iboe->nb_inet);
+ unregister_netdevice_notifier(&iboe->nb);
ib_unregister_device(&hr_dev->ib_dev);
}
@@ -193,15 +383,34 @@ int hns_roce_register_device(struct hns_roce_dev *hr_dev)
goto _error_failed_setup_mtu_gids;
}
+ spin_lock_init(&iboe->lock);
+
+ iboe->nb.notifier_call = hns_roce_netdev_event;
+ ret = register_netdevice_notifier(&iboe->nb);
+ if (ret) {
+ dev_err(dev, "register_netdevice_notifier failed!\n");
+ goto _error_failed_register_netdevice_notifier;
+ }
+
+ iboe->nb_inet.notifier_call = hns_roce_inet_event;
+ ret = register_inetaddr_notifier(&iboe->nb_inet);
+ if (ret) {
+ dev_err(dev, "register inet addr notifier failed!\n");
+ goto _error_failed_register_inetaddr_notifier;
+ }
+
return 0;
+_error_failed_register_inetaddr_notifier:
+ unregister_netdevice_notifier(&iboe->nb);
+
+_error_failed_register_netdevice_notifier:
_error_failed_setup_mtu_gids:
ib_unregister_device(ib_dev);
return ret;
}
-
int hns_roce_get_cfg(struct hns_roce_dev *hr_dev)
{
int i;
--
1.9.1
^ permalink raw reply related
* [PATCH v6 12/21] IB/hns: Set mtu and gid support
From: Lijun Ou @ 2016-04-28 12:09 UTC (permalink / raw)
To: dledford, sean.hefty, hal.rosenstock, davem, jeffrey.t.kirsher,
jiri, ogerlitz
Cc: linux-rdma, linux-kernel, netdev, gongyangming, xiaokun,
tangchaofei, oulijun, haifeng.wei, yisen.zhuang, yankejian,
charles.chenxin, linuxarm
In-Reply-To: <1461845396-61306-1-git-send-email-oulijun@huawei.com>
This patch mainly set mtu and gid resource. These resource
will be used to set up network transmission in nodes.
Signed-off-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Nenglong Zhao <zhaonenglong@hisilicon.com>
Signed-off-by: Lijun Ou <oulijun@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_common.h | 16 ++++
drivers/infiniband/hw/hns/hns_roce_device.h | 14 ++++
drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 64 +++++++++++++++
drivers/infiniband/hw/hns/hns_roce_hw_v1.h | 1 +
drivers/infiniband/hw/hns/hns_roce_main.c | 123 ++++++++++++++++++++++++++++
5 files changed, 218 insertions(+)
diff --git a/drivers/infiniband/hw/hns/hns_roce_common.h b/drivers/infiniband/hw/hns/hns_roce_common.h
index d9c5fda..45b5804 100644
--- a/drivers/infiniband/hw/hns/hns_roce_common.h
+++ b/drivers/infiniband/hw/hns/hns_roce_common.h
@@ -133,6 +133,14 @@
#define ROCEE_BT_CMD_H_ROCEE_BT_CMD_HW_SYNS_S 31
+#define ROCEE_SMAC_H_ROCEE_SMAC_H_S 0
+#define ROCEE_SMAC_H_ROCEE_SMAC_H_M \
+ (((1UL << 16) - 1) << ROCEE_SMAC_H_ROCEE_SMAC_H_S)
+
+#define ROCEE_SMAC_H_ROCEE_PORT_MTU_S 16
+#define ROCEE_SMAC_H_ROCEE_PORT_MTU_M \
+ (((1UL << 4) - 1) << ROCEE_SMAC_H_ROCEE_PORT_MTU_S)
+
#define ROCEE_CAEP_AEQC_AEQE_SHIFT_CAEP_AEQC_STATE_S 0
#define ROCEE_CAEP_AEQC_AEQE_SHIFT_CAEP_AEQC_STATE_M \
(((1UL << 2) - 1) << ROCEE_CAEP_AEQC_AEQE_SHIFT_CAEP_AEQC_STATE_S)
@@ -173,8 +181,16 @@
#define ROCEE_SYS_IMAGE_GUID_L_REG 0xC
#define ROCEE_SYS_IMAGE_GUID_H_REG 0x10
+#define ROCEE_PORT_GID_L_0_REG 0x50
+#define ROCEE_PORT_GID_ML_0_REG 0x54
+#define ROCEE_PORT_GID_MH_0_REG 0x58
+#define ROCEE_PORT_GID_H_0_REG 0x5C
+
#define ROCEE_BT_CMD_H_REG 0x204
+#define ROCEE_SMAC_L_0_REG 0x240
+#define ROCEE_SMAC_H_0_REG 0x244
+
#define ROCEE_CAEP_AEQE_CONS_IDX_REG 0x3AC
#define ROCEE_CAEP_CEQC_CONS_IDX_0_REG 0x3BC
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index 79503bb..06ca4fe 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -21,6 +21,8 @@
#define DRV_NAME "hns_roce"
+#define MAC_ADDR_OCTET_NUM 6
+
#define HNS_ROCE_BA_SIZE (32 * 4096)
#define HNS_ROCE_MAX_IRQ_NUM 34
@@ -31,6 +33,9 @@
#define HNS_ROCE_AEQE_VEC_NUM 1
#define HNS_ROCE_AEQE_OF_VEC_NUM 1
+#define HNS_ROCE_MAX_PORTS 6
+#define HNS_ROCE_MAX_GID_NUM 16
+
#define ADDR_SHIFT_12 12
#define ADDR_SHIFT_32 32
#define ADDR_SHIFT_44 44
@@ -237,6 +242,8 @@ struct hns_roce_qp {
struct hns_roce_ib_iboe {
struct net_device *netdevs[HNS_ROCE_MAX_PORTS];
+ /* 16 GID is shared by 6 port in v1 engine. */
+ union ib_gid gid_table[HNS_ROCE_MAX_GID_NUM];
u8 phy_port[HNS_ROCE_MAX_PORTS];
};
@@ -311,6 +318,11 @@ struct hns_roce_hw {
void (*hw_profile)(struct hns_roce_dev *hr_dev);
int (*hw_init)(struct hns_roce_dev *hr_dev);
void (*hw_uninit)(struct hns_roce_dev *hr_dev);
+ void (*set_gid)(struct hns_roce_dev *hr_dev, u8 port, int gid_index,
+ union ib_gid *gid);
+ void (*set_mac)(struct hns_roce_dev *hr_dev, u8 phy_port, u8 *addr);
+ void (*set_mtu)(struct hns_roce_dev *hr_dev, u8 phy_port,
+ enum ib_mtu mtu);
void *priv;
};
@@ -328,6 +340,7 @@ struct hns_roce_dev {
struct hns_roce_caps caps;
struct radix_tree_root qp_table_tree;
+ unsigned char dev_addr[HNS_ROCE_MAX_PORTS][MAC_ADDR_OCTET_NUM];
u64 fw_ver;
u64 sys_image_guid;
u32 vendor_id;
@@ -397,6 +410,7 @@ void hns_roce_bitmap_free_range(struct hns_roce_bitmap *bitmap,
void hns_roce_cq_completion(struct hns_roce_dev *hr_dev, u32 cqn);
void hns_roce_cq_event(struct hns_roce_dev *hr_dev, u32 cqn, int event_type);
void hns_roce_qp_event(struct hns_roce_dev *hr_dev, u32 qpn, int event_type);
+int hns_get_gid_index(struct hns_roce_dev *hr_dev, u8 port, int gid_index);
extern struct hns_roce_hw hns_roce_hw_v1;
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
index 0e5f816..7a5b211 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
@@ -601,9 +601,73 @@ void hns_roce_v1_uninit(struct hns_roce_dev *hr_dev)
hns_roce_db_free(hr_dev);
}
+void hns_roce_v1_set_gid(struct hns_roce_dev *hr_dev, u8 port, int gid_index,
+ union ib_gid *gid)
+{
+ u32 *p = NULL;
+ u8 gid_idx = 0;
+
+ gid_idx = hns_get_gid_index(hr_dev, port, gid_index);
+
+ p = (u32 *)&gid->raw[0];
+ roce_raw_write(*p, hr_dev->reg_base + ROCEE_PORT_GID_L_0_REG +
+ (HNS_ROCE_V1_GID_NUM * gid_idx));
+
+ p = (u32 *)&gid->raw[4];
+ roce_raw_write(*p, hr_dev->reg_base + ROCEE_PORT_GID_ML_0_REG +
+ (HNS_ROCE_V1_GID_NUM * gid_idx));
+
+ p = (u32 *)&gid->raw[8];
+ roce_raw_write(*p, hr_dev->reg_base + ROCEE_PORT_GID_MH_0_REG +
+ (HNS_ROCE_V1_GID_NUM * gid_idx));
+
+ p = (u32 *)&gid->raw[0xc];
+ roce_raw_write(*p, hr_dev->reg_base + ROCEE_PORT_GID_H_0_REG +
+ (HNS_ROCE_V1_GID_NUM * gid_idx));
+}
+
+void hns_roce_v1_set_mac(struct hns_roce_dev *hr_dev, u8 phy_port, u8 *addr)
+{
+ u32 reg_smac_l;
+ u16 reg_smac_h;
+ u16 *p_h;
+ u32 *p;
+ u32 val;
+
+ p = (u32 *)(&addr[0]);
+ reg_smac_l = *p;
+ roce_raw_write(reg_smac_l, hr_dev->reg_base + ROCEE_SMAC_L_0_REG +
+ PHY_PORT_OFFSET * phy_port);
+
+ val = roce_readl(hr_dev->reg_base + ROCEE_SMAC_H_0_REG +
+ phy_port * PHY_PORT_OFFSET);
+ p_h = (u16 *)(&addr[4]);
+ reg_smac_h = *p_h;
+ roce_set_field(val, ROCEE_SMAC_H_ROCEE_SMAC_H_M,
+ ROCEE_SMAC_H_ROCEE_SMAC_H_S, reg_smac_h);
+ roce_writel(val, hr_dev->reg_base + ROCEE_SMAC_H_0_REG +
+ phy_port * PHY_PORT_OFFSET);
+}
+
+void hns_roce_v1_set_mtu(struct hns_roce_dev *hr_dev, u8 phy_port,
+ enum ib_mtu mtu)
+{
+ u32 val;
+
+ val = roce_readl(hr_dev->reg_base + ROCEE_SMAC_H_0_REG +
+ phy_port * PHY_PORT_OFFSET);
+ roce_set_field(val, ROCEE_SMAC_H_ROCEE_PORT_MTU_M,
+ ROCEE_SMAC_H_ROCEE_PORT_MTU_S, mtu);
+ roce_writel(val, hr_dev->reg_base + ROCEE_SMAC_H_0_REG +
+ phy_port * PHY_PORT_OFFSET);
+}
+
struct hns_roce_hw hns_roce_hw_v1 = {
.reset = hns_roce_v1_reset,
.hw_profile = hns_roce_v1_profile,
.hw_init = hns_roce_v1_init,
.hw_uninit = hns_roce_v1_uninit,
+ .set_gid = hns_roce_v1_set_gid,
+ .set_mac = hns_roce_v1_set_mac,
+ .set_mtu = hns_roce_v1_set_mtu,
};
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.h b/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
index 768862b..143ef0b 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
@@ -84,6 +84,7 @@
#define HNS_ROCE_ODB_EXTEND_MODE 1
+#define PHY_PORT_OFFSET 0x8
#define ALL_PORT_VAL_OPEN 0x3f
#define POL_TIME_INTERVAL_VAL 0x80
#define SLEEP_TIME_INTERVAL 20
diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
index b26509b..0485e02 100644
--- a/drivers/infiniband/hw/hns/hns_roce_main.c
+++ b/drivers/infiniband/hw/hns/hns_roce_main.c
@@ -43,6 +43,118 @@
#include "hns_roce_device.h"
#include "hns_roce_icm.h"
+/**
+ * hns_get_gid_index - Get gid index.
+ * @hr_dev: pointer to structure hns_roce_dev.
+ * @port: port, value range: 0 ~ MAX
+ * @gid_index: gid_index, value range: 0 ~ MAX
+ * Description:
+ * N ports shared gids, allocation method as follow:
+ * GID[0][0], GID[1][0],.....GID[N - 1][0],
+ * GID[0][0], GID[1][0],.....GID[N - 1][0],
+ * And so on
+ */
+int hns_get_gid_index(struct hns_roce_dev *hr_dev, u8 port, int gid_index)
+{
+ return gid_index * hr_dev->caps.num_ports + port;
+}
+
+int hns_roce_set_gid(struct hns_roce_dev *hr_dev, u8 port, int gid_index,
+ union ib_gid *gid)
+{
+ struct device *dev = &hr_dev->pdev->dev;
+ u8 gid_idx = 0;
+
+ if (gid_index >= hr_dev->caps.gid_table_len[port]) {
+ dev_err(dev, "gid_index %d illegal, port %d gid range: 0~%d\n",
+ gid_index, port, hr_dev->caps.gid_table_len[port] - 1);
+ return -EINVAL;
+ }
+
+ gid_idx = hns_get_gid_index(hr_dev, port, gid_index);
+
+ if (!memcmp(gid, &hr_dev->iboe.gid_table[gid_idx], sizeof(*gid)))
+ return -EINVAL;
+
+ memcpy(&hr_dev->iboe.gid_table[gid_idx], gid, sizeof(*gid));
+
+ hr_dev->hw->set_gid(hr_dev, port, gid_index, gid);
+
+ return 0;
+}
+
+void hns_roce_set_mac(struct hns_roce_dev *hr_dev, u8 port, u8 *addr)
+{
+ u8 phy_port;
+ u32 i = 0;
+
+ if (!memcmp(hr_dev->dev_addr[port], addr, MAC_ADDR_OCTET_NUM))
+ return;
+
+ for (i = 0; i < MAC_ADDR_OCTET_NUM; i++)
+ hr_dev->dev_addr[port][i] = addr[i];
+
+ phy_port = hr_dev->iboe.phy_port[port];
+ hr_dev->hw->set_mac(hr_dev, phy_port, addr);
+}
+
+void hns_roce_set_mtu(struct hns_roce_dev *hr_dev, u8 port, int mtu)
+{
+ u8 phy_port = hr_dev->iboe.phy_port[port];
+ enum ib_mtu tmp;
+
+ tmp = iboe_get_mtu(mtu);
+ if (!tmp)
+ tmp = IB_MTU_256;
+
+ hr_dev->hw->set_mtu(hr_dev, phy_port, tmp);
+}
+
+void hns_roce_update_gids(struct hns_roce_dev *hr_dev, int port)
+{
+ struct ib_event event;
+
+ /* Refresh gid in ib_cache */
+ event.device = &hr_dev->ib_dev;
+ event.element.port_num = port + 1;
+ event.event = IB_EVENT_GID_CHANGE;
+ ib_dispatch_event(&event);
+}
+
+int hns_roce_setup_mtu_gids(struct hns_roce_dev *hr_dev)
+{
+ struct in_ifaddr *ifa_list = NULL;
+ union ib_gid gid = {{0} };
+ u32 ipaddr = 0;
+ int index = 0;
+ int ret = 0;
+ u8 i = 0;
+
+ for (i = 0; i < hr_dev->caps.num_ports; i++) {
+ hns_roce_set_mtu(hr_dev, i,
+ ib_mtu_enum_to_int(hr_dev->caps.max_mtu));
+ hns_roce_set_mac(hr_dev, i, hr_dev->iboe.netdevs[i]->dev_addr);
+
+ if (hr_dev->iboe.netdevs[i]->ip_ptr) {
+ ifa_list = hr_dev->iboe.netdevs[i]->ip_ptr->ifa_list;
+ index = 1;
+ while (ifa_list) {
+ ipaddr = ifa_list->ifa_address;
+ ipv6_addr_set_v4mapped(ipaddr,
+ (struct in6_addr *)&gid);
+ ret = hns_roce_set_gid(hr_dev, i, index, &gid);
+ if (ret)
+ break;
+ index++;
+ ifa_list = ifa_list->ifa_next;
+ }
+ hns_roce_update_gids(hr_dev, i);
+ }
+ }
+
+ return ret;
+}
+
void hns_roce_unregister_device(struct hns_roce_dev *hr_dev)
{
ib_unregister_device(&hr_dev->ib_dev);
@@ -75,7 +187,18 @@ int hns_roce_register_device(struct hns_roce_dev *hr_dev)
return ret;
}
+ ret = hns_roce_setup_mtu_gids(hr_dev);
+ if (ret) {
+ dev_err(dev, "roce_setup_mtu_gids failed!\n");
+ goto _error_failed_setup_mtu_gids;
+ }
+
return 0;
+
+_error_failed_setup_mtu_gids:
+ ib_unregister_device(ib_dev);
+
+ return ret;
}
--
1.9.1
^ permalink raw reply related
* [PATCH v6 11/21] IB/hns: Add IB device registration
From: Lijun Ou @ 2016-04-28 12:09 UTC (permalink / raw)
To: dledford, sean.hefty, hal.rosenstock, davem, jeffrey.t.kirsher,
jiri, ogerlitz
Cc: linux-rdma, linux-kernel, netdev, gongyangming, xiaokun,
tangchaofei, oulijun, haifeng.wei, yisen.zhuang, yankejian,
charles.chenxin, linuxarm
In-Reply-To: <1461845396-61306-1-git-send-email-oulijun@huawei.com>
This patch registered IB device and unregistered IB device
when removed.
Signed-off-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Nenglong Zhao <zhaonenglong@hisilicon.com>
Signed-off-by: Lijun Ou <oulijun@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_main.c | 48 +++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
index 64c6b89..b26509b 100644
--- a/drivers/infiniband/hw/hns/hns_roce_main.c
+++ b/drivers/infiniband/hw/hns/hns_roce_main.c
@@ -43,6 +43,42 @@
#include "hns_roce_device.h"
#include "hns_roce_icm.h"
+void hns_roce_unregister_device(struct hns_roce_dev *hr_dev)
+{
+ ib_unregister_device(&hr_dev->ib_dev);
+}
+
+int hns_roce_register_device(struct hns_roce_dev *hr_dev)
+{
+ int ret;
+ struct hns_roce_ib_iboe *iboe = NULL;
+ struct ib_device *ib_dev = NULL;
+ struct device *dev = &hr_dev->pdev->dev;
+
+ iboe = &hr_dev->iboe;
+
+ ib_dev = &hr_dev->ib_dev;
+ strlcpy(ib_dev->name, "hisi_%d", IB_DEVICE_NAME_MAX);
+
+ ib_dev->owner = THIS_MODULE;
+ ib_dev->node_type = RDMA_NODE_IB_CA;
+ ib_dev->dma_device = dev;
+
+ ib_dev->phys_port_cnt = hr_dev->caps.num_ports;
+ ib_dev->local_dma_lkey = hr_dev->caps.reserved_lkey;
+ ib_dev->num_comp_vectors = hr_dev->caps.num_comp_vectors;
+ ib_dev->uverbs_abi_ver = 1;
+
+ ret = ib_register_device(ib_dev, NULL);
+ if (ret) {
+ dev_err(dev, "ib_register_device failed!\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+
int hns_roce_get_cfg(struct hns_roce_dev *hr_dev)
{
int i;
@@ -348,6 +384,17 @@ static int hns_roce_probe(struct platform_device *pdev)
goto error_failed_engine_init;
}
+ ret = hns_roce_register_device(hr_dev);
+ if (ret) {
+ dev_err(dev, "register_device failed!\n");
+ goto error_failed_register_device;
+ }
+
+ return 0;
+
+error_failed_register_device:
+ hns_roce_engine_uninit(hr_dev);
+
error_failed_engine_init:
hns_roce_cleanup_bitmap(hr_dev);
@@ -383,6 +430,7 @@ static int hns_roce_remove(struct platform_device *pdev)
{
struct hns_roce_dev *hr_dev = platform_get_drvdata(pdev);
+ hns_roce_unregister_device(hr_dev);
hns_roce_engine_uninit(hr_dev);
hns_roce_cleanup_bitmap(hr_dev);
hns_roce_cleanup_icm(hr_dev);
--
1.9.1
^ permalink raw reply related
* [PATCH v6 10/21] IB/hns: Add process flow to init RoCE engine
From: Lijun Ou @ 2016-04-28 12:09 UTC (permalink / raw)
To: dledford, sean.hefty, hal.rosenstock, davem, jeffrey.t.kirsher,
jiri, ogerlitz
Cc: linux-rdma, linux-kernel, netdev, gongyangming, xiaokun,
tangchaofei, oulijun, haifeng.wei, yisen.zhuang, yankejian,
charles.chenxin, linuxarm
In-Reply-To: <1461845396-61306-1-git-send-email-oulijun@huawei.com>
This patch mainly initialized the RoCE engine. It is absolutely
necessary to run RoCE. It mainly includes that configure DMAE
user, initialize doorbell and raq operations, enable port.
Signed-off-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Nenglong Zhao <zhaonenglong@hisilicon.com>
Signed-off-by: Lijun Ou <oulijun@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_common.h | 107 +++++++
drivers/infiniband/hw/hns/hns_roce_device.h | 15 +
drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 477 ++++++++++++++++++++++++++++
drivers/infiniband/hw/hns/hns_roce_hw_v1.h | 68 +++-
drivers/infiniband/hw/hns/hns_roce_main.c | 20 ++
5 files changed, 686 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_common.h b/drivers/infiniband/hw/hns/hns_roce_common.h
index fcac5c4..d9c5fda 100644
--- a/drivers/infiniband/hw/hns/hns_roce_common.h
+++ b/drivers/infiniband/hw/hns/hns_roce_common.h
@@ -30,6 +30,93 @@
#define roce_set_bit(origin, shift, val) \
roce_set_field((origin), (1ul << (shift)), (shift), (val))
+#define ROCEE_GLB_CFG_ROCEE_DB_SQ_MODE_S 3
+#define ROCEE_GLB_CFG_ROCEE_DB_OTH_MODE_S 4
+
+#define ROCEE_GLB_CFG_SQ_EXT_DB_MODE_S 5
+
+#define ROCEE_GLB_CFG_OTH_EXT_DB_MODE_S 6
+
+#define ROCEE_GLB_CFG_ROCEE_PORT_ST_S 10
+#define ROCEE_GLB_CFG_ROCEE_PORT_ST_M \
+ (((1UL << 6) - 1) << ROCEE_GLB_CFG_ROCEE_PORT_ST_S)
+
+#define ROCEE_GLB_CFG_TRP_RAQ_DROP_EN_S 16
+
+#define ROCEE_DMAE_USER_CFG1_ROCEE_STREAM_ID_TB_CFG_S 0
+#define ROCEE_DMAE_USER_CFG1_ROCEE_STREAM_ID_TB_CFG_M \
+ (((1UL << 24) - 1) << ROCEE_DMAE_USER_CFG1_ROCEE_STREAM_ID_TB_CFG_S)
+
+#define ROCEE_DMAE_USER_CFG1_ROCEE_CACHE_TB_CFG_S 24
+#define ROCEE_DMAE_USER_CFG1_ROCEE_CACHE_TB_CFG_M \
+ (((1UL << 4) - 1) << ROCEE_DMAE_USER_CFG1_ROCEE_CACHE_TB_CFG_S)
+
+#define ROCEE_DMAE_USER_CFG2_ROCEE_STREAM_ID_PKT_CFG_S 0
+#define ROCEE_DMAE_USER_CFG2_ROCEE_STREAM_ID_PKT_CFG_M \
+ (((1UL << 24) - 1) << ROCEE_DMAE_USER_CFG2_ROCEE_STREAM_ID_PKT_CFG_S)
+
+#define ROCEE_DMAE_USER_CFG2_ROCEE_CACHE_PKT_CFG_S 24
+#define ROCEE_DMAE_USER_CFG2_ROCEE_CACHE_PKT_CFG_M \
+ (((1UL << 4) - 1) << ROCEE_DMAE_USER_CFG2_ROCEE_CACHE_PKT_CFG_S)
+
+#define ROCEE_DB_SQ_WL_ROCEE_DB_SQ_WL_S 0
+#define ROCEE_DB_SQ_WL_ROCEE_DB_SQ_WL_M \
+ (((1UL << 16) - 1) << ROCEE_DB_SQ_WL_ROCEE_DB_SQ_WL_S)
+
+#define ROCEE_DB_SQ_WL_ROCEE_DB_SQ_WL_EMPTY_S 16
+#define ROCEE_DB_SQ_WL_ROCEE_DB_SQ_WL_EMPTY_M \
+ (((1UL << 16) - 1) << ROCEE_DB_SQ_WL_ROCEE_DB_SQ_WL_EMPTY_S)
+
+#define ROCEE_DB_OTHERS_WL_ROCEE_DB_OTH_WL_S 0
+#define ROCEE_DB_OTHERS_WL_ROCEE_DB_OTH_WL_M \
+ (((1UL << 16) - 1) << ROCEE_DB_OTHERS_WL_ROCEE_DB_OTH_WL_S)
+
+#define ROCEE_DB_OTHERS_WL_ROCEE_DB_OTH_WL_EMPTY_S 16
+#define ROCEE_DB_OTHERS_WL_ROCEE_DB_OTH_WL_EMPTY_M \
+ (((1UL << 16) - 1) << ROCEE_DB_OTHERS_WL_ROCEE_DB_OTH_WL_EMPTY_S)
+
+#define ROCEE_RAQ_WL_ROCEE_RAQ_WL_S 0
+#define ROCEE_RAQ_WL_ROCEE_RAQ_WL_M \
+ (((1UL << 8) - 1) << ROCEE_RAQ_WL_ROCEE_RAQ_WL_S)
+
+#define ROCEE_WRMS_POL_TIME_INTERVAL_WRMS_POL_TIME_INTERVAL_S 0
+#define ROCEE_WRMS_POL_TIME_INTERVAL_WRMS_POL_TIME_INTERVAL_M \
+ (((1UL << 15) - 1) << \
+ ROCEE_WRMS_POL_TIME_INTERVAL_WRMS_POL_TIME_INTERVAL_S)
+
+#define ROCEE_WRMS_POL_TIME_INTERVAL_WRMS_RAQ_TIMEOUT_CHK_CFG_S 16
+#define ROCEE_WRMS_POL_TIME_INTERVAL_WRMS_RAQ_TIMEOUT_CHK_CFG_M \
+ (((1UL << 4) - 1) << \
+ ROCEE_WRMS_POL_TIME_INTERVAL_WRMS_RAQ_TIMEOUT_CHK_CFG_S)
+
+#define ROCEE_WRMS_POL_TIME_INTERVAL_WRMS_RAQ_TIMEOUT_CHK_EN_S 20
+
+#define ROCEE_WRMS_POL_TIME_INTERVAL_WRMS_EXT_RAQ_MODE 21
+
+#define ROCEE_EXT_DB_SQ_H_EXT_DB_SQ_SHIFT_S 0
+#define ROCEE_EXT_DB_SQ_H_EXT_DB_SQ_SHIFT_M \
+ (((1UL << 5) - 1) << ROCEE_EXT_DB_SQ_H_EXT_DB_SQ_SHIFT_S)
+
+#define ROCEE_EXT_DB_SQ_H_EXT_DB_SQ_BA_H_S 5
+#define ROCEE_EXT_DB_SQ_H_EXT_DB_SQ_BA_H_M \
+ (((1UL << 5) - 1) << ROCEE_EXT_DB_SQ_H_EXT_DB_SQ_BA_H_S)
+
+#define ROCEE_EXT_DB_OTH_H_EXT_DB_OTH_SHIFT_S 0
+#define ROCEE_EXT_DB_OTH_H_EXT_DB_OTH_SHIFT_M \
+ (((1UL << 5) - 1) << ROCEE_EXT_DB_OTH_H_EXT_DB_OTH_SHIFT_S)
+
+#define ROCEE_EXT_DB_SQ_H_EXT_DB_OTH_BA_H_S 5
+#define ROCEE_EXT_DB_SQ_H_EXT_DB_OTH_BA_H_M \
+ (((1UL << 5) - 1) << ROCEE_EXT_DB_SQ_H_EXT_DB_OTH_BA_H_S)
+
+#define ROCEE_EXT_RAQ_H_EXT_RAQ_SHIFT_S 0
+#define ROCEE_EXT_RAQ_H_EXT_RAQ_SHIFT_M \
+ (((1UL << 5) - 1) << ROCEE_EXT_RAQ_H_EXT_RAQ_SHIFT_S)
+
+#define ROCEE_EXT_RAQ_H_EXT_RAQ_BA_H_S 8
+#define ROCEE_EXT_RAQ_H_EXT_RAQ_BA_H_M \
+ (((1UL << 5) - 1) << ROCEE_EXT_RAQ_H_EXT_RAQ_BA_H_S)
+
#define ROCEE_BT_CMD_H_ROCEE_BT_CMD_IN_MDF_S 0
#define ROCEE_BT_CMD_H_ROCEE_BT_CMD_IN_MDF_M \
(((1UL << 19) - 1) << ROCEE_BT_CMD_H_ROCEE_BT_CMD_IN_MDF_S)
@@ -97,6 +184,26 @@
#define ROCEE_ECC_CERR_ALM2_REG 0xB48
#define ROCEE_ACK_DELAY_REG 0x14
+#define ROCEE_GLB_CFG_REG 0x18
+
+#define ROCEE_DMAE_USER_CFG1_REG 0x40
+#define ROCEE_DMAE_USER_CFG2_REG 0x44
+
+#define ROCEE_DB_SQ_WL_REG 0x154
+#define ROCEE_DB_OTHERS_WL_REG 0x158
+#define ROCEE_RAQ_WL_REG 0x15C
+#define ROCEE_WRMS_POL_TIME_INTERVAL_REG 0x160
+#define ROCEE_EXT_DB_SQ_REG 0x164
+#define ROCEE_EXT_DB_SQ_H_REG 0x168
+#define ROCEE_EXT_DB_OTH_REG 0x16C
+
+#define ROCEE_EXT_DB_OTH_H_REG 0x170
+#define ROCEE_EXT_DB_SQ_WL_EMPTY_REG 0x174
+#define ROCEE_EXT_DB_SQ_WL_REG 0x178
+#define ROCEE_EXT_DB_OTHERS_WL_EMPTY_REG 0x17C
+#define ROCEE_EXT_DB_OTHERS_WL_REG 0x180
+#define ROCEE_EXT_RAQ_REG 0x184
+#define ROCEE_EXT_RAQ_H_REG 0x188
#define ROCEE_CAEP_CE_INTERVAL_CFG_REG 0x190
#define ROCEE_CAEP_CE_BURST_NUM_CFG_REG 0x194
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index a6e9efc..79503bb 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -35,6 +35,8 @@
#define ADDR_SHIFT_32 32
#define ADDR_SHIFT_44 44
+#define PAGES_SHIFT_16 16
+
enum hns_roce_event {
HNS_ROCE_EVENT_TYPE_PATH_MIG = 0x01,
HNS_ROCE_EVENT_TYPE_PATH_MIG_FAILED = 0x02,
@@ -91,6 +93,9 @@ enum {
HNS_ROCE_CMD_SUCCESS = 1,
};
+#define HNS_ROCE_PORT_DOWN 0
+#define HNS_ROCE_PORT_UP 1
+
struct hns_roce_uar {
u64 pfn;
unsigned long index;
@@ -176,6 +181,13 @@ struct hns_roce_cq_table {
struct hns_roce_icm_table table;
};
+struct hns_roce_raq_table {
+ void __iomem *e_raq_addr;
+ void __iomem *e_raq_wl_addr;
+ void __iomem *e_raq_shift_addr;
+ struct hns_roce_buf_list *e_raq_buf;
+};
+
struct hns_roce_cmd_context {
struct completion done;
int result;
@@ -297,6 +309,9 @@ struct hns_roce_caps {
struct hns_roce_hw {
int (*reset)(struct hns_roce_dev *hr_dev, u32 val);
void (*hw_profile)(struct hns_roce_dev *hr_dev);
+ int (*hw_init)(struct hns_roce_dev *hr_dev);
+ void (*hw_uninit)(struct hns_roce_dev *hr_dev);
+ void *priv;
};
struct hns_roce_dev {
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
index 6ce160e..0e5f816 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
@@ -21,6 +21,431 @@
#include "hns_roce_device.h"
#include "hns_roce_hw_v1.h"
+void hns_roce_set_db_event_mode(struct hns_roce_dev *hr_dev, int sdb_mode,
+ int odb_mode)
+{
+ u32 val;
+
+ val = roce_readl(hr_dev->reg_base + ROCEE_GLB_CFG_REG);
+ roce_set_bit(val, ROCEE_GLB_CFG_ROCEE_DB_SQ_MODE_S, sdb_mode);
+ roce_set_bit(val, ROCEE_GLB_CFG_ROCEE_DB_OTH_MODE_S, odb_mode);
+ roce_writel(val, hr_dev->reg_base + ROCEE_GLB_CFG_REG);
+}
+
+void hns_roce_set_db_ext_mode(struct hns_roce_dev *hr_dev,
+ u32 sdb_mode, u32 odb_mode)
+{
+ u32 val;
+
+ /* Configure SDB/ODB extend mode */
+ val = roce_readl(hr_dev->reg_base + ROCEE_GLB_CFG_REG);
+ roce_set_bit(val, ROCEE_GLB_CFG_SQ_EXT_DB_MODE_S, sdb_mode);
+ roce_set_bit(val, ROCEE_GLB_CFG_OTH_EXT_DB_MODE_S, odb_mode);
+ roce_writel(val, hr_dev->reg_base + ROCEE_GLB_CFG_REG);
+}
+
+void hns_roce_set_sdb(struct hns_roce_dev *hr_dev, u32 sdb_alept, u32 sdb_alful)
+{
+ struct hns_roce_v1_priv *priv;
+ struct hns_roce_db_table *db;
+ u32 val;
+
+ priv = (struct hns_roce_v1_priv *)hr_dev->hw->priv;
+ db = &priv->db_table;
+
+ db->sdb_almept = sdb_alept;
+ db->sdb_almful = sdb_alful;
+
+ /* Configure SDB */
+ val = roce_readl(hr_dev->reg_base + ROCEE_DB_SQ_WL_REG);
+ roce_set_field(val, ROCEE_DB_SQ_WL_ROCEE_DB_SQ_WL_M,
+ ROCEE_DB_SQ_WL_ROCEE_DB_SQ_WL_S, db->sdb_almful);
+ roce_set_field(val, ROCEE_DB_SQ_WL_ROCEE_DB_SQ_WL_EMPTY_M,
+ ROCEE_DB_SQ_WL_ROCEE_DB_SQ_WL_EMPTY_S, db->sdb_almept);
+ roce_writel(val, hr_dev->reg_base + ROCEE_DB_SQ_WL_REG);
+}
+
+void hns_roce_set_odb(struct hns_roce_dev *hr_dev, u32 odb_alept, u32 odb_alful)
+{
+ struct hns_roce_v1_priv *priv;
+ struct hns_roce_db_table *db;
+ u32 val;
+
+ priv = (struct hns_roce_v1_priv *)hr_dev->hw->priv;
+ db = &priv->db_table;
+
+ db->odb_almept = odb_alept;
+ db->odb_almful = odb_alful;
+
+ /* Configure ODB */
+ val = roce_readl(hr_dev->reg_base + ROCEE_DB_OTHERS_WL_REG);
+ roce_set_field(val, ROCEE_DB_OTHERS_WL_ROCEE_DB_OTH_WL_M,
+ ROCEE_DB_OTHERS_WL_ROCEE_DB_OTH_WL_S, db->odb_almful);
+ roce_set_field(val, ROCEE_DB_OTHERS_WL_ROCEE_DB_OTH_WL_EMPTY_M,
+ ROCEE_DB_OTHERS_WL_ROCEE_DB_OTH_WL_EMPTY_S,
+ db->odb_almept);
+ roce_writel(val, hr_dev->reg_base + ROCEE_DB_OTHERS_WL_REG);
+}
+
+void hns_roce_set_sdb_ext(struct hns_roce_dev *hr_dev, u32 ext_sdb_alept,
+ u32 ext_sdb_alful)
+{
+ struct device *dev = &hr_dev->pdev->dev;
+ struct hns_roce_v1_priv *priv;
+ struct hns_roce_db_table *db;
+ dma_addr_t sdb_dma_addr;
+ u32 val;
+
+ priv = (struct hns_roce_v1_priv *)hr_dev->hw->priv;
+ db = &priv->db_table;
+
+ db->ext_db->esdb_almept = ext_sdb_alept;
+ db->ext_db->esdb_alful = ext_sdb_alful;
+
+ /* Configure extend SDB threshold */
+ roce_writel(db->ext_db->esdb_almept,
+ hr_dev->reg_base + ROCEE_EXT_DB_SQ_WL_EMPTY_REG);
+ roce_writel(db->ext_db->esdb_alful,
+ hr_dev->reg_base + ROCEE_EXT_DB_SQ_WL_REG);
+
+ /* Configure extend SDB base addr */
+ sdb_dma_addr = db->ext_db->sdb_buf_list->map;
+ roce_writel((u32)(sdb_dma_addr >> ADDR_SHIFT_12),
+ hr_dev->reg_base + ROCEE_EXT_DB_SQ_REG);
+
+ /* Configure extend SDB depth */
+ val = roce_readl(hr_dev->reg_base + ROCEE_EXT_DB_SQ_H_REG);
+ roce_set_field(val, ROCEE_EXT_DB_SQ_H_EXT_DB_SQ_SHIFT_M,
+ ROCEE_EXT_DB_SQ_H_EXT_DB_SQ_SHIFT_S,
+ db->ext_db->esdb_dep);
+ roce_set_field(val, ROCEE_EXT_DB_SQ_H_EXT_DB_SQ_BA_H_M,
+ ROCEE_EXT_DB_SQ_H_EXT_DB_SQ_BA_H_S,
+ sdb_dma_addr >> ADDR_SHIFT_44);
+ roce_writel(val, hr_dev->reg_base + ROCEE_EXT_DB_SQ_H_REG);
+
+ dev_dbg(dev, "ext SDB depth: 0x%x\n", db->ext_db->esdb_dep);
+ dev_dbg(dev, "ext SDB threshold: epmty: 0x%x, ful: 0x%x\n",
+ db->ext_db->esdb_almept, db->ext_db->esdb_alful);
+}
+
+void hns_roce_set_odb_ext(struct hns_roce_dev *hr_dev, u32 ext_odb_alept,
+ u32 ext_odb_alful)
+{
+ struct device *dev = &hr_dev->pdev->dev;
+ struct hns_roce_v1_priv *priv;
+ struct hns_roce_db_table *db;
+ dma_addr_t odb_dma_addr;
+ u32 val;
+
+ priv = (struct hns_roce_v1_priv *)hr_dev->hw->priv;
+ db = &priv->db_table;
+
+ db->ext_db->eodb_almept = ext_odb_alept;
+ db->ext_db->eodb_alful = ext_odb_alful;
+
+ /* Configure extend ODB threshold */
+ roce_writel(db->ext_db->eodb_almept,
+ hr_dev->reg_base + ROCEE_EXT_DB_OTHERS_WL_EMPTY_REG);
+ roce_writel(db->ext_db->eodb_alful,
+ hr_dev->reg_base + ROCEE_EXT_DB_OTHERS_WL_REG);
+
+ /* Configure extend ODB base addr */
+ odb_dma_addr = db->ext_db->odb_buf_list->map;
+ roce_writel((u32)(odb_dma_addr >> ADDR_SHIFT_12),
+ hr_dev->reg_base + ROCEE_EXT_DB_OTH_REG);
+
+ /* Configure extend ODB depth */
+ val = roce_readl(hr_dev->reg_base + ROCEE_EXT_DB_OTH_H_REG);
+ roce_set_field(val, ROCEE_EXT_DB_OTH_H_EXT_DB_OTH_SHIFT_M,
+ ROCEE_EXT_DB_OTH_H_EXT_DB_OTH_SHIFT_S,
+ db->ext_db->eodb_dep);
+ roce_set_field(val, ROCEE_EXT_DB_SQ_H_EXT_DB_OTH_BA_H_M,
+ ROCEE_EXT_DB_SQ_H_EXT_DB_OTH_BA_H_S,
+ db->ext_db->eodb_dep);
+ roce_writel(val, hr_dev->reg_base + ROCEE_EXT_DB_OTH_H_REG);
+
+ dev_dbg(dev, "ext ODB depth: 0x%x\n", db->ext_db->eodb_dep);
+ dev_dbg(dev, "ext ODB threshold: empty: 0x%x, ful: 0x%x\n",
+ db->ext_db->eodb_almept, db->ext_db->eodb_alful);
+}
+
+int hns_roce_db_ext_init(struct hns_roce_dev *hr_dev, u32 sdb_ext_mod,
+ u32 odb_ext_mod)
+{
+ struct device *dev = &hr_dev->pdev->dev;
+ struct hns_roce_v1_priv *priv;
+ struct hns_roce_db_table *db;
+ dma_addr_t sdb_dma_addr;
+ dma_addr_t odb_dma_addr;
+ int ret = 0;
+
+ priv = (struct hns_roce_v1_priv *)hr_dev->hw->priv;
+ db = &priv->db_table;
+
+ db->ext_db = kmalloc(sizeof(*db->ext_db), GFP_KERNEL);
+ if (!db->ext_db) {
+ ret = -ENOMEM;
+ dev_err(dev, "extend db buf alloc fail\n");
+ return ret;
+ }
+
+ if (sdb_ext_mod) {
+ db->ext_db->sdb_buf_list = kmalloc(
+ sizeof(*db->ext_db->sdb_buf_list), GFP_KERNEL);
+ if (!db->ext_db->sdb_buf_list) {
+ ret = -ENOMEM;
+ dev_err(dev, "sdb buf alloc failed\n");
+ goto ext_sdb_buf_fail_out;
+ }
+
+ db->ext_db->sdb_buf_list->buf = dma_alloc_coherent(dev,
+ HNS_ROCE_V1_EXT_SDB_SIZE,
+ &sdb_dma_addr, GFP_KERNEL);
+ if (!db->ext_db->sdb_buf_list->buf) {
+ ret = -ENOMEM;
+ dev_err(dev, "Send queue db buf alloc fail\n");
+ goto alloc_sq_db_buf_fail;
+ }
+ db->ext_db->sdb_buf_list->map = sdb_dma_addr;
+
+ db->ext_db->esdb_dep = ilog2(HNS_ROCE_V1_EXT_SDB_DEPTH);
+ hns_roce_set_sdb_ext(hr_dev, HNS_ROCE_V1_EXT_SDB_ALEPT,
+ HNS_ROCE_V1_EXT_SDB_ALFUL);
+ } else
+ hns_roce_set_sdb(hr_dev, HNS_ROCE_V1_SDB_ALEPT,
+ HNS_ROCE_V1_SDB_ALFUL);
+
+ if (odb_ext_mod) {
+ db->ext_db->odb_buf_list = kmalloc(
+ sizeof(*db->ext_db->odb_buf_list), GFP_KERNEL);
+ if (!db->ext_db->odb_buf_list) {
+ ret = -ENOMEM;
+ dev_err(dev, "odb buf alloc failed\n");
+ goto ext_odb_buf_fail_out;
+ }
+
+ db->ext_db->odb_buf_list->buf = dma_alloc_coherent(dev,
+ HNS_ROCE_V1_EXT_ODB_SIZE,
+ &odb_dma_addr, GFP_KERNEL);
+ if (!db->ext_db->odb_buf_list->buf) {
+ ret = -ENOMEM;
+ dev_err(dev, "Other queue db buf alloc fail\n");
+ goto alloc_otr_db_buf_fail;
+ }
+ db->ext_db->odb_buf_list->map = odb_dma_addr;
+
+ db->ext_db->eodb_dep = ilog2(HNS_ROCE_V1_EXT_ODB_DEPTH);
+ hns_roce_set_odb_ext(hr_dev, HNS_ROCE_V1_EXT_ODB_ALEPT,
+ HNS_ROCE_V1_EXT_ODB_ALFUL);
+ } else
+ hns_roce_set_odb(hr_dev, HNS_ROCE_V1_ODB_ALEPT,
+ HNS_ROCE_V1_ODB_ALFUL);
+
+ hns_roce_set_db_ext_mode(hr_dev, sdb_ext_mod, odb_ext_mod);
+
+ return 0;
+
+alloc_otr_db_buf_fail:
+ kfree(db->ext_db->odb_buf_list);
+
+ext_odb_buf_fail_out:
+ if (sdb_ext_mod) {
+ dma_free_coherent(dev, HNS_ROCE_V1_EXT_SDB_SIZE,
+ db->ext_db->sdb_buf_list->buf,
+ db->ext_db->sdb_buf_list->map);
+ }
+
+alloc_sq_db_buf_fail:
+ if (sdb_ext_mod)
+ kfree(db->ext_db->sdb_buf_list);
+
+ext_sdb_buf_fail_out:
+ kfree(db->ext_db);
+ return ret;
+}
+
+int hns_roce_db_init(struct hns_roce_dev *hr_dev)
+{
+ struct device *dev = &hr_dev->pdev->dev;
+ struct hns_roce_v1_priv *priv;
+ struct hns_roce_db_table *db;
+ u32 sdb_ext_mod;
+ u32 odb_ext_mod;
+ u32 sdb_evt_mod;
+ u32 odb_evt_mod;
+ int ret = 0;
+
+ priv = (struct hns_roce_v1_priv *)hr_dev->hw->priv;
+ db = &priv->db_table;
+
+ memset(db, 0, sizeof(*db));
+
+ /* Default DB mode */
+ sdb_ext_mod = HNS_ROCE_SDB_EXTEND_MODE;
+ odb_ext_mod = HNS_ROCE_ODB_EXTEND_MODE;
+ sdb_evt_mod = HNS_ROCE_SDB_NORMAL_MODE;
+ odb_evt_mod = HNS_ROCE_ODB_POLL_MODE;
+
+ db->sdb_ext_mod = sdb_ext_mod;
+ db->odb_ext_mod = odb_ext_mod;
+
+ /* Init extend DB */
+ ret = hns_roce_db_ext_init(hr_dev, sdb_ext_mod, odb_ext_mod);
+ if (ret) {
+ dev_err(dev, "Failed in extend DB configuration.\n");
+ return ret;
+ }
+
+ hns_roce_set_db_event_mode(hr_dev, sdb_evt_mod, odb_evt_mod);
+
+ return 0;
+}
+
+void hns_roce_db_free(struct hns_roce_dev *hr_dev)
+{
+ struct device *dev = &hr_dev->pdev->dev;
+ struct hns_roce_v1_priv *priv;
+ struct hns_roce_db_table *db;
+
+ priv = (struct hns_roce_v1_priv *)hr_dev->hw->priv;
+ db = &priv->db_table;
+
+ if (db->sdb_ext_mod) {
+ dma_free_coherent(dev, HNS_ROCE_V1_EXT_SDB_SIZE,
+ db->ext_db->sdb_buf_list->buf,
+ db->ext_db->sdb_buf_list->map);
+ kfree(db->ext_db->sdb_buf_list);
+ }
+
+ if (db->odb_ext_mod) {
+ dma_free_coherent(dev, HNS_ROCE_V1_EXT_ODB_SIZE,
+ db->ext_db->odb_buf_list->buf,
+ db->ext_db->odb_buf_list->map);
+ kfree(db->ext_db->odb_buf_list);
+ }
+
+ kfree(db->ext_db);
+}
+
+int hns_roce_raq_init(struct hns_roce_dev *hr_dev)
+{
+ int ret;
+ int raq_shift = 0;
+ dma_addr_t addr;
+ u32 val;
+ struct hns_roce_v1_priv *priv;
+ struct hns_roce_raq_table *raq;
+ struct device *dev = &hr_dev->pdev->dev;
+
+ priv = (struct hns_roce_v1_priv *)hr_dev->hw->priv;
+ raq = &priv->raq_table;
+
+ raq->e_raq_addr = hr_dev->reg_base + ROCEE_EXT_RAQ_REG;
+ raq->e_raq_wl_addr = hr_dev->reg_base + ROCEE_RAQ_WL_REG;
+ raq->e_raq_shift_addr = hr_dev->reg_base + ROCEE_EXT_RAQ_H_REG;
+
+ raq->e_raq_buf = kzalloc(sizeof(*(raq->e_raq_buf)), GFP_KERNEL);
+ if (!raq->e_raq_buf) {
+ ret = -ENOMEM;
+ dev_err(dev, "Failed to alloc raq buf, Aborting.\n");
+ return ret;
+ }
+
+ raq->e_raq_buf->buf = dma_alloc_coherent(dev, HNS_ROCE_V1_RAQ_SIZE,
+ &addr, GFP_KERNEL);
+ if (!raq->e_raq_buf->buf) {
+ ret = -ENOMEM;
+ dev_err(dev, "Failed to dma_alloc ext raq buf.\n");
+ goto _err_dma_alloc_raq;
+ }
+ raq->e_raq_buf->map = addr;
+
+ /* Configure raq extended address. 48bit 4K align*/
+ roce_writel(raq->e_raq_buf->map >> ADDR_SHIFT_12, raq->e_raq_addr);
+
+ /* Configure raq_shift */
+ raq_shift = ilog2(HNS_ROCE_V1_RAQ_SIZE / HNS_ROCE_V1_RAQ_ENTRY);
+ val = roce_readl(raq->e_raq_shift_addr);
+ roce_set_field(val, ROCEE_EXT_RAQ_H_EXT_RAQ_SHIFT_M,
+ ROCEE_EXT_RAQ_H_EXT_RAQ_SHIFT_S,
+ raq_shift);
+ roce_set_field(val, ROCEE_EXT_RAQ_H_EXT_RAQ_BA_H_M,
+ ROCEE_EXT_RAQ_H_EXT_RAQ_BA_H_S,
+ raq->e_raq_buf->map >> ADDR_SHIFT_44);
+ roce_writel(val, raq->e_raq_shift_addr);
+ dev_dbg(dev, "Configure raq_shift 0x%x.\n", val);
+
+ /* Configure raq threshold */
+ val = roce_readl(raq->e_raq_wl_addr);
+ roce_set_field(val, ROCEE_RAQ_WL_ROCEE_RAQ_WL_M,
+ ROCEE_RAQ_WL_ROCEE_RAQ_WL_S,
+ HNS_ROCE_V1_EXT_RAQ_WF);
+ roce_writel(val, raq->e_raq_wl_addr);
+ dev_dbg(dev, "Configure raq_wl 0x%x.\n", val);
+
+ /* Enable extend raq */
+ val = roce_readl(hr_dev->reg_base + ROCEE_WRMS_POL_TIME_INTERVAL_REG);
+ roce_set_field(val,
+ ROCEE_WRMS_POL_TIME_INTERVAL_WRMS_POL_TIME_INTERVAL_M,
+ ROCEE_WRMS_POL_TIME_INTERVAL_WRMS_POL_TIME_INTERVAL_S,
+ POL_TIME_INTERVAL_VAL);
+ roce_set_bit(val, ROCEE_WRMS_POL_TIME_INTERVAL_WRMS_EXT_RAQ_MODE, 1);
+ roce_set_field(val,
+ ROCEE_WRMS_POL_TIME_INTERVAL_WRMS_RAQ_TIMEOUT_CHK_CFG_M,
+ ROCEE_WRMS_POL_TIME_INTERVAL_WRMS_RAQ_TIMEOUT_CHK_CFG_S,
+ 2);
+ roce_set_bit(val,
+ ROCEE_WRMS_POL_TIME_INTERVAL_WRMS_RAQ_TIMEOUT_CHK_EN_S, 1);
+ roce_writel(val, hr_dev->reg_base + ROCEE_WRMS_POL_TIME_INTERVAL_REG);
+ dev_dbg(dev, "Configure WrmsPolTimeInterval 0x%x.\n", val);
+
+ /* Enable raq drop */
+ val = roce_readl(hr_dev->reg_base + ROCEE_GLB_CFG_REG);
+ roce_set_bit(val, ROCEE_GLB_CFG_TRP_RAQ_DROP_EN_S, 1);
+ roce_writel(val, hr_dev->reg_base + ROCEE_GLB_CFG_REG);
+ dev_dbg(dev, "Configure GlbCfg = 0x%x.\n", val);
+
+ return 0;
+
+_err_dma_alloc_raq:
+ kfree(raq->e_raq_buf);
+ return ret;
+}
+
+void hns_roce_raq_free(struct hns_roce_dev *hr_dev)
+{
+ struct device *dev = &hr_dev->pdev->dev;
+ struct hns_roce_v1_priv *priv;
+ struct hns_roce_raq_table *raq;
+
+ priv = (struct hns_roce_v1_priv *)hr_dev->hw->priv;
+ raq = &priv->raq_table;
+
+ dma_free_coherent(dev, HNS_ROCE_V1_RAQ_SIZE, raq->e_raq_buf->buf,
+ raq->e_raq_buf->map);
+ kfree(raq->e_raq_buf);
+}
+
+void hns_roce_port_enable(struct hns_roce_dev *hr_dev, int enable_flag)
+{
+ u32 val;
+
+ if (enable_flag) {
+ val = roce_readl(hr_dev->reg_base + ROCEE_GLB_CFG_REG);
+ /* Open all ports */
+ roce_set_field(val, ROCEE_GLB_CFG_ROCEE_PORT_ST_M,
+ ROCEE_GLB_CFG_ROCEE_PORT_ST_S,
+ ALL_PORT_VAL_OPEN);
+ roce_writel(val, hr_dev->reg_base + ROCEE_GLB_CFG_REG);
+ } else {
+ val = roce_readl(hr_dev->reg_base + ROCEE_GLB_CFG_REG);
+ /* Close all ports */
+ roce_set_field(val, ROCEE_GLB_CFG_ROCEE_PORT_ST_M,
+ ROCEE_GLB_CFG_ROCEE_PORT_ST_S, 0x0);
+ roce_writel(val, hr_dev->reg_base + ROCEE_GLB_CFG_REG);
+ }
+}
+
/**
* hns_roce_v1_reset - reset roce
* @hr_dev: roce device struct pointer
@@ -126,7 +551,59 @@ void hns_roce_v1_profile(struct hns_roce_dev *hr_dev)
caps->max_mtu = IB_MTU_2048;
}
+int hns_roce_v1_init(struct hns_roce_dev *hr_dev)
+{
+ int ret;
+ u32 val;
+ struct device *dev = &hr_dev->pdev->dev;
+
+ /* DMAE user config */
+ val = roce_readl(hr_dev->reg_base + ROCEE_DMAE_USER_CFG1_REG);
+ roce_set_field(val, ROCEE_DMAE_USER_CFG1_ROCEE_CACHE_TB_CFG_M,
+ ROCEE_DMAE_USER_CFG1_ROCEE_CACHE_TB_CFG_S, 0xf);
+ roce_set_field(val, ROCEE_DMAE_USER_CFG1_ROCEE_STREAM_ID_TB_CFG_M,
+ ROCEE_DMAE_USER_CFG1_ROCEE_STREAM_ID_TB_CFG_S,
+ 1 << PAGES_SHIFT_16);
+ roce_writel(val, hr_dev->reg_base + ROCEE_DMAE_USER_CFG1_REG);
+
+ val = roce_readl(hr_dev->reg_base + ROCEE_DMAE_USER_CFG2_REG);
+ roce_set_field(val, ROCEE_DMAE_USER_CFG2_ROCEE_CACHE_PKT_CFG_M,
+ ROCEE_DMAE_USER_CFG2_ROCEE_CACHE_PKT_CFG_S, 0xf);
+ roce_set_field(val, ROCEE_DMAE_USER_CFG2_ROCEE_STREAM_ID_PKT_CFG_M,
+ ROCEE_DMAE_USER_CFG2_ROCEE_STREAM_ID_PKT_CFG_S,
+ 1 << PAGES_SHIFT_16);
+
+ ret = hns_roce_db_init(hr_dev);
+ if (ret) {
+ dev_err(dev, "doorbell init failed!\n");
+ return ret;
+ }
+
+ ret = hns_roce_raq_init(hr_dev);
+ if (ret) {
+ dev_err(dev, "raq init failed!\n");
+ goto _error_failed_raq_init;
+ }
+
+ hns_roce_port_enable(hr_dev, HNS_ROCE_PORT_UP);
+
+ return 0;
+
+_error_failed_raq_init:
+ hns_roce_db_free(hr_dev);
+ return ret;
+}
+
+void hns_roce_v1_uninit(struct hns_roce_dev *hr_dev)
+{
+ hns_roce_port_enable(hr_dev, HNS_ROCE_PORT_DOWN);
+ hns_roce_raq_free(hr_dev);
+ hns_roce_db_free(hr_dev);
+}
+
struct hns_roce_hw hns_roce_hw_v1 = {
.reset = hns_roce_v1_reset,
.hw_profile = hns_roce_v1_profile,
+ .hw_init = hns_roce_v1_init,
+ .hw_uninit = hns_roce_v1_uninit,
};
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.h b/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
index 7f4859d..768862b 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.h
@@ -46,7 +46,73 @@
#define HNS_ROCE_V1_CQE_ENTRY_SIZE 32
#define HNS_ROCE_V1_PAGE_SIZE_SUPPORT 0xFFFFF000
-#define SLEEP_TIME_INTERVAL 20
+#define HNS_ROCE_V1_EXT_RAQ_WF 8
+#define HNS_ROCE_V1_RAQ_ENTRY 64
+#define HNS_ROCE_V1_RAQ_DEPTH 32768
+#define HNS_ROCE_V1_RAQ_SIZE (HNS_ROCE_V1_RAQ_ENTRY * HNS_ROCE_V1_RAQ_DEPTH)
+
+#define HNS_ROCE_V1_SDB_DEPTH 0x400
+#define HNS_ROCE_V1_ODB_DEPTH 0x400
+
+#define HNS_ROCE_V1_DB_RSVD 0x80
+
+#define HNS_ROCE_V1_SDB_ALEPT HNS_ROCE_V1_DB_RSVD
+#define HNS_ROCE_V1_SDB_ALFUL (HNS_ROCE_V1_SDB_DEPTH - HNS_ROCE_V1_DB_RSVD)
+#define HNS_ROCE_V1_ODB_ALEPT HNS_ROCE_V1_DB_RSVD
+#define HNS_ROCE_V1_ODB_ALFUL (HNS_ROCE_V1_ODB_DEPTH - HNS_ROCE_V1_DB_RSVD)
+
+#define HNS_ROCE_V1_EXT_SDB_DEPTH 0x4000
+#define HNS_ROCE_V1_EXT_ODB_DEPTH 0x4000
+#define HNS_ROCE_V1_EXT_SDB_ENTRY 16
+#define HNS_ROCE_V1_EXT_ODB_ENTRY 16
+#define HNS_ROCE_V1_EXT_SDB_SIZE \
+ (HNS_ROCE_V1_EXT_SDB_DEPTH * HNS_ROCE_V1_EXT_SDB_ENTRY)
+#define HNS_ROCE_V1_EXT_ODB_SIZE \
+ (HNS_ROCE_V1_EXT_ODB_DEPTH * HNS_ROCE_V1_EXT_ODB_ENTRY)
+
+#define HNS_ROCE_V1_EXT_SDB_ALEPT HNS_ROCE_V1_DB_RSVD
+#define HNS_ROCE_V1_EXT_SDB_ALFUL \
+ (HNS_ROCE_V1_EXT_SDB_DEPTH - HNS_ROCE_V1_DB_RSVD)
+#define HNS_ROCE_V1_EXT_ODB_ALEPT HNS_ROCE_V1_DB_RSVD
+#define HNS_ROCE_V1_EXT_ODB_ALFUL \
+ (HNS_ROCE_V1_EXT_ODB_DEPTH - HNS_ROCE_V1_DB_RSVD)
+
+#define HNS_ROCE_ODB_POLL_MODE 0
+
+#define HNS_ROCE_SDB_NORMAL_MODE 0
+#define HNS_ROCE_SDB_EXTEND_MODE 1
+
+#define HNS_ROCE_ODB_EXTEND_MODE 1
+
+#define ALL_PORT_VAL_OPEN 0x3f
+#define POL_TIME_INTERVAL_VAL 0x80
+#define SLEEP_TIME_INTERVAL 20
+
+struct hns_roce_ext_db {
+ int esdb_almept;
+ int esdb_alful;
+ int eodb_almept;
+ int eodb_alful;
+ int esdb_dep;
+ int eodb_dep;
+ struct hns_roce_buf_list *sdb_buf_list;
+ struct hns_roce_buf_list *odb_buf_list;
+};
+
+struct hns_roce_db_table {
+ int sdb_ext_mod;
+ int odb_ext_mod;
+ int sdb_almept;
+ int sdb_almful;
+ int odb_almept;
+ int odb_almful;
+ struct hns_roce_ext_db *ext_db;
+};
+
+struct hns_roce_v1_priv {
+ struct hns_roce_db_table db_table;
+ struct hns_roce_raq_table raq_table;
+};
extern int hns_dsaf_roce_reset(struct fwnode_handle *dsaf_fwnode, u32 val);
diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
index bde7aa0..64c6b89 100644
--- a/drivers/infiniband/hw/hns/hns_roce_main.c
+++ b/drivers/infiniband/hw/hns/hns_roce_main.c
@@ -183,6 +183,16 @@ err_unmap_mtt:
return ret;
}
+int hns_roce_engine_init(struct hns_roce_dev *hr_dev)
+{
+ return hr_dev->hw->hw_init(hr_dev);
+}
+
+void hns_roce_engine_uninit(struct hns_roce_dev *hr_dev)
+{
+ hr_dev->hw->hw_uninit(hr_dev);
+}
+
/**
* hns_roce_setup_hca - setup host channel adapter
* @hr_dev: pointer to hns roce device
@@ -332,6 +342,15 @@ static int hns_roce_probe(struct platform_device *pdev)
goto error_failed_setup_hca;
}
+ ret = hns_roce_engine_init(hr_dev);
+ if (ret) {
+ dev_err(dev, "hw_init failed!\n");
+ goto error_failed_engine_init;
+ }
+
+error_failed_engine_init:
+ hns_roce_cleanup_bitmap(hr_dev);
+
error_failed_setup_hca:
hns_roce_cleanup_icm(hr_dev);
@@ -364,6 +383,7 @@ static int hns_roce_remove(struct platform_device *pdev)
{
struct hns_roce_dev *hr_dev = platform_get_drvdata(pdev);
+ hns_roce_engine_uninit(hr_dev);
hns_roce_cleanup_bitmap(hr_dev);
hns_roce_cleanup_icm(hr_dev);
--
1.9.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox