* [Patch net 3/3] hsr: fix a NULL pointer deref in hsr_dev_xmit()
From: Cong Wang @ 2019-07-04 0:21 UTC (permalink / raw)
To: netdev; +Cc: Cong Wang, Arvid Brodin
In-Reply-To: <20190704002114.29004-1-xiyou.wangcong@gmail.com>
hsr_port_get_hsr() could return NULL and kernel
could crash:
BUG: kernel NULL pointer dereference, address: 0000000000000010
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 8000000074b84067 P4D 8000000074b84067 PUD 7057d067 PMD 0
Oops: 0000 [#1] SMP PTI
CPU: 0 PID: 754 Comm: a.out Not tainted 5.2.0-rc6+ #718
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-2.fc30 04/01/2014
RIP: 0010:hsr_dev_xmit+0x20/0x31
Code: 48 8b 1b eb e0 5b 5d 41 5c c3 66 66 66 66 90 55 48 89 fd 48 8d be 40 0b 00 00 be 04 00 00 00 e8 ee f2 ff ff 48 89 ef 48 89 c6 <48> 8b 40 10 48 89 45 10 e8 6c 1b 00 00 31 c0 5d c3 66 66 66 66 90
RSP: 0018:ffffb5b400003c48 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff9821b4509a88 RCX: 0000000000000000
RDX: ffff9821b4509a88 RSI: 0000000000000000 RDI: ffff9821bc3fc7c0
RBP: ffff9821bc3fc7c0 R08: 0000000000000000 R09: 00000000000c2019
R10: 0000000000000000 R11: 0000000000000002 R12: ffff9821bc3fc7c0
R13: ffff9821b4509a88 R14: 0000000000000000 R15: 000000000000006e
FS: 00007fee112a1800(0000) GS:ffff9821bd800000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000010 CR3: 000000006e9ce000 CR4: 00000000000406f0
Call Trace:
<IRQ>
netdev_start_xmit+0x1b/0x38
dev_hard_start_xmit+0x121/0x21e
? validate_xmit_skb.isra.0+0x19/0x1e3
__dev_queue_xmit+0x74c/0x823
? lockdep_hardirqs_on+0x12b/0x17d
ip6_finish_output2+0x3d3/0x42c
? ip6_mtu+0x55/0x5c
? mld_sendpack+0x191/0x229
mld_sendpack+0x191/0x229
mld_ifc_timer_expire+0x1f7/0x230
? mld_dad_timer_expire+0x58/0x58
call_timer_fn+0x12e/0x273
__run_timers.part.0+0x174/0x1b5
? mld_dad_timer_expire+0x58/0x58
? sched_clock_cpu+0x10/0xad
? mark_lock+0x26/0x1f2
? __lock_is_held+0x40/0x71
run_timer_softirq+0x26/0x48
__do_softirq+0x1af/0x392
irq_exit+0x53/0xa2
smp_apic_timer_interrupt+0x1c4/0x1d9
apic_timer_interrupt+0xf/0x20
</IRQ>
Cc: Arvid Brodin <arvid.brodin@alten.se>
Signed-off-by: Cong Wang <xiyou.wangcong@gmail.com>
---
net/hsr/hsr_device.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/net/hsr/hsr_device.c b/net/hsr/hsr_device.c
index 4ea7d54a8262..f0f9b493c47b 100644
--- a/net/hsr/hsr_device.c
+++ b/net/hsr/hsr_device.c
@@ -227,9 +227,13 @@ static int hsr_dev_xmit(struct sk_buff *skb, struct net_device *dev)
struct hsr_port *master;
master = hsr_port_get_hsr(hsr, HSR_PT_MASTER);
- skb->dev = master->dev;
- hsr_forward_skb(skb, master);
-
+ if (master) {
+ skb->dev = master->dev;
+ hsr_forward_skb(skb, master);
+ } else {
+ atomic_long_inc(&dev->tx_dropped);
+ dev_kfree_skb_any(skb);
+ }
return NETDEV_TX_OK;
}
--
2.21.0
^ permalink raw reply related
* [PATCH v2 net-next 0/3] tc-testing: Add JSON verification and simple traffic generation
From: Lucas Bates @ 2019-07-04 0:44 UTC (permalink / raw)
To: davem
Cc: netdev, jhs, xiyou.wangcong, jiri, mleitner, vladbu, dcaratti,
kernel, Lucas Bates
This patchset introduces JSON as a verification method in tdc and adds a new
plugin, scapyPlugin, as a way to send traffic to test tc filters and actions.
This version includes the patch signoffs missing in the previous submission.
The first patch adds the JSON verification to the core tdc script.
The second patch makes a change to the TdcPlugin module that will allow tdc
plugins to examine the test case currently being executed, such that plugins
can play a more active role in testing. This feature is needed for the
new plugin.
The third patch adds the scapyPlugin itself, and an example test case file to
demonstrate how the scapy block works.
Lucas Bates (3):
tc-testing: Add JSON verification to tdc
tc-testing: Allow tdc plugins to see test case data
tc-testing: introduce scapyPlugin for basic traffic
tools/testing/selftests/tc-testing/TdcPlugin.py | 5 +-
.../creating-testcases/scapy-example.json | 98 ++++++++++++++
.../selftests/tc-testing/plugin-lib/scapyPlugin.py | 51 +++++++
tools/testing/selftests/tc-testing/tdc.py | 146 ++++++++++++++++++---
4 files changed, 279 insertions(+), 21 deletions(-)
create mode 100644 tools/testing/selftests/tc-testing/creating-testcases/scapy-example.json
create mode 100644 tools/testing/selftests/tc-testing/plugin-lib/scapyPlugin.py
--
2.7.4
^ permalink raw reply
* [PATCH v2 net-next 1/3] tc-testing: Add JSON verification to tdc
From: Lucas Bates @ 2019-07-04 0:45 UTC (permalink / raw)
To: davem
Cc: netdev, jhs, xiyou.wangcong, jiri, mleitner, vladbu, dcaratti,
kernel, Lucas Bates
In-Reply-To: <1562201102-4332-1-git-send-email-lucasb@mojatatu.com>
This patch allows tdc to process JSON output to perform secondary
verification of the command under test. If the verifyCmd generates
JSON, one can provide the 'matchJSON' key to process it
instead of a regex.
matchJSON has two elements: 'path' and 'value'. The 'path' key is a
list of integers and strings that provide the key values for tdc to
navigate the JSON information. The value is an integer or string
that tdc will compare against what it finds in the provided path.
If the numerical position of an element can vary, it's possible to
substitute an asterisk as a wildcard. tdc will search all possible
entries in the array.
Multiple matches are possible, but everything specified must
match for the test to pass.
If both matchPattern and matchJSON are present, tdc will only
operate on matchPattern. If neither are present, verification
is skipped.
Example:
"cmdUnderTest": "$TC actions add action pass index 8",
"verifyCmd": "$TC actions list action gact",
"matchJSON": [
{
"path": [
0,
"actions",
0,
"control action",
"type"
],
"value": "gact"
},
{
"path": [
0,
"actions",
0,
"index"
],
"value": 8
}
]
Signed-off-by: Lucas Bates <lucasb@mojatatu.com>
---
tools/testing/selftests/tc-testing/tdc.py | 136 +++++++++++++++++++++++++++---
1 file changed, 123 insertions(+), 13 deletions(-)
diff --git a/tools/testing/selftests/tc-testing/tdc.py b/tools/testing/selftests/tc-testing/tdc.py
index 678182a..1afa803 100755
--- a/tools/testing/selftests/tc-testing/tdc.py
+++ b/tools/testing/selftests/tc-testing/tdc.py
@@ -35,6 +35,10 @@ class PluginMgrTestFail(Exception):
self.output = output
self.message = message
+class ElemNotFound(Exception):
+ def __init__(self, path_element):
+ self.path_element = path_element
+
class PluginMgr:
def __init__(self, argparser):
super().__init__()
@@ -167,6 +171,40 @@ class PluginMgr:
self.argparser = argparse.ArgumentParser(
description='Linux TC unit tests')
+def find_in_json(jsonobj, path):
+ if type(jsonobj) == list:
+ if type(path[0]) == int:
+ if len(jsonobj) > path[0]:
+ return find_in_json(jsonobj[path[0]], path[1:])
+ else:
+ raise ElemNotFound(path[0])
+ elif path[0] == '*':
+ res = []
+ for index in jsonobj:
+ try:
+ res.append(find_in_json(index, path[1:]))
+ except ElemNotFound:
+ continue
+ if len(res) == 0:
+ raise ElemNotFound(path[0])
+ else:
+ return res
+ elif type(jsonobj) == dict:
+ if path[0] in jsonobj:
+ if len(path) > 1:
+ return find_in_json(jsonobj[path[0]], path[1:])
+ return jsonobj[path[0]]
+ else:
+ raise ElemNotFound(path[0])
+ else:
+ # Assume we have found the correct depth in the object
+ if len(path) >= 1:
+ print('The remainder of the specified path cannot be found!')
+ print('Path values: {}'.format(path))
+ raise ElemNotFound(path[0])
+ return jsonobj
+
+
def replace_keywords(cmd):
"""
For a given executable command, substitute any known
@@ -246,6 +284,86 @@ def prepare_env(args, pm, stage, prefix, cmdlist, output = None):
stage, output,
'"{}" did not complete successfully'.format(prefix))
+def verify_by_regex(res, tidx, args, pm):
+ if 'matchCount' not in tidx:
+ res.set_result(ResultState.skip)
+ fmsg = 'matchCount was not provided in the test case. '
+ fmsg += 'Unable to complete pattern match.'
+ res.set_failmsg(fmsg)
+ print(fmsg)
+ return res
+ (p, procout) = exec_cmd(args, pm, 'verify', tidx["verifyCmd"])
+ match_pattern = re.compile(
+ str(tidx["matchPattern"]), re.DOTALL | re.MULTILINE)
+ if procout:
+ match_index = re.findall(match_pattern, procout)
+ if len(match_index) != int(tidx["matchCount"]):
+ res.set_result(ResultState.fail)
+ fmsg = 'Verify stage failed because the output did not match '
+ fmsg += 'the pattern in the test case.\nMatch pattern is:\n'
+ fmsg += '\t{}\n'.format(tidx["matchPattern"])
+ fmsg += 'Output generated by the verify command:\n'
+ fmsg += '{}\n'.format(procout)
+ res.set_failmsg(fmsg)
+ else:
+ res.set_result(ResultState.success)
+ elif int(tidx["matchCount"]) != 0:
+ res.set_result(ResultState.fail)
+ res.set_failmsg('No output generated by verify command.')
+ else:
+ res.set_result(ResultState.success)
+ return res
+
+def verify_by_json(res, tidx, args, pm):
+ # Validate the matchJSON struct
+ for match in tidx['matchJSON']:
+ if 'path' in match and 'value' in match:
+ pass
+ else:
+ res.set_result(ResultState.skip)
+ res.set_failmsg('matchJSON missing required keys for this case.')
+ return res
+ (p, procout) = exec_cmd(args, pm, 'verify', tidx["verifyCmd"])
+ # Run procout through the JSON decoder
+ try:
+ jsonobj = json.loads(procout)
+ except json.JSONDecodeError:
+ if len(tidx['matchJSON']) > 0:
+ res.set_result(ResultState.fail)
+ res.set_failmsg('Cannot decode verify command\'s output. Is it JSON?')
+ return res
+ # Then recurse through the object
+ valuesmatch = True
+ for match in tidx['matchJSON']:
+ try:
+ value = find_in_json(jsonobj, match['path'])
+ except ElemNotFound as ENF:
+ fmsg = 'Could not find the element {} specified in the path.'.format(ENF.path_element)
+ valuesmatch = False
+ break
+ if type(value) == list:
+ if match['value'] not in value:
+ valuesmatch = False
+ fmsg = 'Verify stage failed because the value specified in the path\n'
+ fmsg += '{}\n'.format(match['path'])
+ fmsg += 'Expected value: {}\nReceived value: {}'.format(
+ match['value'], value)
+ break
+ elif match['value'] != value:
+ valuesmatch = False
+ fmsg = 'Verify stage failed because the value specified in the path\n'
+ fmsg += '{}\n'.format(match['path'])
+ fmsg += 'Expected value: {}\nReceived value: {}'.format(
+ match['value'], value)
+ break
+ if valuesmatch:
+ res.set_result(ResultState.success)
+ else:
+ res.set_result(ResultState.fail)
+ res.set_failmsg(fmsg)
+ print(fmsg)
+ return res
+
def run_one_test(pm, args, index, tidx):
global NAMES
result = True
@@ -292,21 +410,13 @@ def run_one_test(pm, args, index, tidx):
else:
if args.verbose > 0:
print('-----> verify stage')
- match_pattern = re.compile(
- str(tidx["matchPattern"]), re.DOTALL | re.MULTILINE)
- (p, procout) = exec_cmd(args, pm, 'verify', tidx["verifyCmd"])
- if procout:
- match_index = re.findall(match_pattern, procout)
- if len(match_index) != int(tidx["matchCount"]):
- res.set_result(ResultState.fail)
- res.set_failmsg('Could not match regex pattern. Verify command output:\n{}'.format(procout))
- else:
- res.set_result(ResultState.success)
- elif int(tidx["matchCount"]) != 0:
- res.set_result(ResultState.fail)
- res.set_failmsg('No output generated by verify command.')
+ if 'matchPattern' in tidx:
+ res = verify_by_regex(res, tidx, args, pm)
+ elif 'matchJSON' in tidx:
+ res = verify_by_json(res, tidx, args, pm)
else:
res.set_result(ResultState.success)
+ print('No match method defined in current test case, skipping verify')
prepare_env(args, pm, 'teardown', '-----> teardown stage', tidx['teardown'], procout)
pm.call_post_case()
--
2.7.4
^ permalink raw reply related
* [PATCH v2 net-next 2/3] tc-testing: Allow tdc plugins to see test case data
From: Lucas Bates @ 2019-07-04 0:45 UTC (permalink / raw)
To: davem
Cc: netdev, jhs, xiyou.wangcong, jiri, mleitner, vladbu, dcaratti,
kernel, Lucas Bates
In-Reply-To: <1562201102-4332-1-git-send-email-lucasb@mojatatu.com>
Instead of only passing the test case name and ID, pass the
entire current test case down to the plugins. This change
allows plugins to start accepting commands and directives
from the test cases themselves, for greater flexibility
in testing.
Signed-off-by: Lucas Bates <lucasb@mojatatu.com>
---
tools/testing/selftests/tc-testing/TdcPlugin.py | 5 ++---
tools/testing/selftests/tc-testing/tdc.py | 10 +++++-----
2 files changed, 7 insertions(+), 8 deletions(-)
diff --git a/tools/testing/selftests/tc-testing/TdcPlugin.py b/tools/testing/selftests/tc-testing/TdcPlugin.py
index b980a56..79f3ca8 100644
--- a/tools/testing/selftests/tc-testing/TdcPlugin.py
+++ b/tools/testing/selftests/tc-testing/TdcPlugin.py
@@ -18,12 +18,11 @@ class TdcPlugin:
if self.args.verbose > 1:
print(' -- {}.post_suite'.format(self.sub_class))
- def pre_case(self, testid, test_name, test_skip):
+ def pre_case(self, caseinfo, test_skip):
'''run commands before test_runner does one test'''
if self.args.verbose > 1:
print(' -- {}.pre_case'.format(self.sub_class))
- self.args.testid = testid
- self.args.test_name = test_name
+ self.args.caseinfo = caseinfo
self.args.test_skip = test_skip
def post_case(self):
diff --git a/tools/testing/selftests/tc-testing/tdc.py b/tools/testing/selftests/tc-testing/tdc.py
index 1afa803..de7da9a 100755
--- a/tools/testing/selftests/tc-testing/tdc.py
+++ b/tools/testing/selftests/tc-testing/tdc.py
@@ -126,15 +126,15 @@ class PluginMgr:
for pgn_inst in reversed(self.plugin_instances):
pgn_inst.post_suite(index)
- def call_pre_case(self, testid, test_name, *, test_skip=False):
+ def call_pre_case(self, caseinfo, *, test_skip=False):
for pgn_inst in self.plugin_instances:
try:
- pgn_inst.pre_case(testid, test_name, test_skip)
+ pgn_inst.pre_case(caseinfo, test_skip)
except Exception as ee:
print('exception {} in call to pre_case for {} plugin'.
format(ee, pgn_inst.__class__))
print('test_ordinal is {}'.format(test_ordinal))
- print('testid is {}'.format(testid))
+ print('testid is {}'.format(caseinfo['id']))
raise
def call_post_case(self):
@@ -379,14 +379,14 @@ def run_one_test(pm, args, index, tidx):
res = TestResult(tidx['id'], tidx['name'])
res.set_result(ResultState.skip)
res.set_errormsg('Test case designated as skipped.')
- pm.call_pre_case(tidx['id'], tidx['name'], test_skip=True)
+ pm.call_pre_case(tidx, test_skip=True)
pm.call_post_execute()
return res
# populate NAMES with TESTID for this test
NAMES['TESTID'] = tidx['id']
- pm.call_pre_case(tidx['id'], tidx['name'])
+ pm.call_pre_case(tidx)
prepare_env(args, pm, 'setup', "-----> prepare stage", tidx["setup"])
if (args.verbose > 0):
--
2.7.4
^ permalink raw reply related
* [PATCH v2 net-next 3/3] tc-testing: introduce scapyPlugin for basic traffic
From: Lucas Bates @ 2019-07-04 0:45 UTC (permalink / raw)
To: davem
Cc: netdev, jhs, xiyou.wangcong, jiri, mleitner, vladbu, dcaratti,
kernel, Lucas Bates
In-Reply-To: <1562201102-4332-1-git-send-email-lucasb@mojatatu.com>
The scapyPlugin allows for simple traffic generation in tdc to
test various tc features. It was tested with scapy v2.4.2, but
should work with any successive version.
In order to use the plugin's functionality, scapy must be
installed. This can be done with:
pip3 install scapy
or to install 2.4.2:
pip3 install scapy==2.4.2
If the plugin is unable to import the scapy module, it will
terminate the tdc run.
The plugin makes use of a new key in the test case data, 'scapy'.
This block contains three other elements: 'iface', 'count', and
'packet':
"scapy": {
"iface": "$DEV0",
"count": 1,
"packet": "Ether(type=0x800)/IP(src='16.61.16.61')/ICMP()"
},
* iface is the name of the device on the host machine from which
the packet(s) will be sent. Values contained within tdc_config.py's
NAMES dict can be used here - this is useful if paired with
nsPlugin
* count is the number of copies of this packet to be sent
* packet is a string detailing the different layers of the packet
to be sent. If a property isn't explicitly set, scapy will set
default values for you.
Layers in the packet info are separated by slashes. For info about
common TCP and IP properties, see:
https://blogs.sans.org/pen-testing/files/2016/04/ScapyCheatSheet_v0.2.pdf
Caution is advised when running tests using the scapy functionality,
since the plugin blindly sends the packet as defined in the test case
data.
See creating-testcases/scapy-example.json for sample test cases;
the first test is intended to pass while the second is intended to
fail. Consider using the matchJSON functionality for verification
when using scapy.
Signed-off-by: Lucas Bates <lucasb@mojatatu.com>
---
.../creating-testcases/scapy-example.json | 98 ++++++++++++++++++++++
.../selftests/tc-testing/plugin-lib/scapyPlugin.py | 51 +++++++++++
2 files changed, 149 insertions(+)
create mode 100644 tools/testing/selftests/tc-testing/creating-testcases/scapy-example.json
create mode 100644 tools/testing/selftests/tc-testing/plugin-lib/scapyPlugin.py
diff --git a/tools/testing/selftests/tc-testing/creating-testcases/scapy-example.json b/tools/testing/selftests/tc-testing/creating-testcases/scapy-example.json
new file mode 100644
index 0000000..5a9377b
--- /dev/null
+++ b/tools/testing/selftests/tc-testing/creating-testcases/scapy-example.json
@@ -0,0 +1,98 @@
+[
+ {
+ "id": "b1e9",
+ "name": "Test matching of source IP",
+ "category": [
+ "actions",
+ "scapy"
+ ],
+ "plugins": {
+ "requires": [
+ "nsPlugin",
+ "scapyPlugin"
+ ]
+ },
+ "setup": [
+ [
+ "$TC qdisc del dev $DEV1 ingress",
+ 0,
+ 1,
+ 2,
+ 255
+ ],
+ "$TC qdisc add dev $DEV1 ingress"
+ ],
+ "cmdUnderTest": "$TC filter add dev $DEV1 parent ffff: prio 3 protocol ip flower src_ip 16.61.16.61 flowid 1:1 action ok",
+ "scapy": {
+ "iface": "$DEV0",
+ "count": 1,
+ "packet": "Ether(type=0x800)/IP(src='16.61.16.61')/ICMP()"
+ },
+ "expExitCode": "0",
+ "verifyCmd": "$TC -s -j filter ls dev $DEV1 ingress prio 3",
+ "matchJSON": [
+ {
+ "path": [
+ 1,
+ "options",
+ "actions",
+ 0,
+ "stats",
+ "packets"
+ ],
+ "value": 1
+ }
+ ],
+ "teardown": [
+ "$TC qdisc del dev $DEV1 ingress"
+ ]
+ },
+ {
+ "id": "e9c4",
+ "name": "Test matching of source IP with wrong count",
+ "category": [
+ "actions",
+ "scapy"
+ ],
+ "plugins": {
+ "requires": [
+ "nsPlugin",
+ "scapyPlugin"
+ ]
+ },
+ "setup": [
+ [
+ "$TC qdisc del dev $DEV1 ingress",
+ 0,
+ 1,
+ 2,
+ 255
+ ],
+ "$TC qdisc add dev $DEV1 ingress"
+ ],
+ "cmdUnderTest": "$TC filter add dev $DEV1 parent ffff: prio 3 protocol ip flower src_ip 16.61.16.61 flowid 1:1 action ok",
+ "scapy": {
+ "iface": "$DEV0",
+ "count": 3,
+ "packet": "Ether(type=0x800)/IP(src='16.61.16.61')/ICMP()"
+ },
+ "expExitCode": "0",
+ "verifyCmd": "$TC -s -j filter ls dev $DEV1 parent ffff:",
+ "matchJSON": [
+ {
+ "path": [
+ 1,
+ "options",
+ "actions",
+ 0,
+ "stats",
+ "packets"
+ ],
+ "value": 1
+ }
+ ],
+ "teardown": [
+ "$TC qdisc del dev $DEV1 ingress"
+ ]
+ }
+]
diff --git a/tools/testing/selftests/tc-testing/plugin-lib/scapyPlugin.py b/tools/testing/selftests/tc-testing/plugin-lib/scapyPlugin.py
new file mode 100644
index 0000000..db57916
--- /dev/null
+++ b/tools/testing/selftests/tc-testing/plugin-lib/scapyPlugin.py
@@ -0,0 +1,51 @@
+#!/usr/bin/env python3
+
+import os
+import signal
+from string import Template
+import subprocess
+import time
+from TdcPlugin import TdcPlugin
+
+from tdc_config import *
+
+try:
+ from scapy.all import *
+except ImportError:
+ print("Unable to import the scapy python module.")
+ print("\nIf not already installed, you may do so with:")
+ print("\t\tpip3 install scapy==2.4.2")
+ exit(1)
+
+class SubPlugin(TdcPlugin):
+ def __init__(self):
+ self.sub_class = 'scapy/SubPlugin'
+ super().__init__()
+
+ def post_execute(self):
+ if 'scapy' not in self.args.caseinfo:
+ if self.args.verbose:
+ print('{}.post_execute: no scapy info in test case'.format(self.sub_class))
+ return
+
+ # Check for required fields
+ scapyinfo = self.args.caseinfo['scapy']
+ scapy_keys = ['iface', 'count', 'packet']
+ missing_keys = []
+ keyfail = False
+ for k in scapy_keys:
+ if k not in scapyinfo:
+ keyfail = True
+ missing_keys.add(k)
+ if keyfail:
+ print('{}: Scapy block present in the test, but is missing info:'
+ .format(self.sub_class))
+ print('{}'.format(missing_keys))
+
+ pkt = eval(scapyinfo['packet'])
+ if '$' in scapyinfo['iface']:
+ tpl = Template(scapyinfo['iface'])
+ scapyinfo['iface'] = tpl.safe_substitute(NAMES)
+ for count in range(scapyinfo['count']):
+ sendp(pkt, iface=scapyinfo['iface'])
+
--
2.7.4
^ permalink raw reply related
* [PATCH v4 bpf-next 0/4] capture integers in BTF type info for map defs
From: Andrii Nakryiko @ 2019-07-04 0:49 UTC (permalink / raw)
To: andrii.nakryiko, kernel-team, bpf, netdev, ast, daniel; +Cc: Andrii Nakryiko
This patch set implements an update to how BTF-defined maps are specified. The
change is in how integer attributes, e.g., type, max_entries, map_flags, are
specified: now they are captured as part of map definition struct's BTF type
information (using array dimension), eliminating the need for compile-time
data initialization and keeping all the metadata in one place.
All existing selftests that were using BTF-defined maps are updated, along
with some other selftests, that were switched to new syntax.
v3->v4:
- add acks;
- fix int -> uint type in commit message;
v2->v3:
- rename __int into __uint (Yonghong);
v1->v2:
- split bpf_helpers.h change from libbpf change (Song).
Andrii Nakryiko (4):
libbpf: capture value in BTF type info for BTF-defined map defs
selftests/bpf: add __uint and __type macro for BTF-defined maps
selftests/bpf: convert selftests using BTF-defined maps to new syntax
selftests/bpf: convert legacy BPF maps to BTF-defined ones
tools/lib/bpf/libbpf.c | 58 +++++----
tools/testing/selftests/bpf/bpf_helpers.h | 3 +
tools/testing/selftests/bpf/progs/bpf_flow.c | 28 ++---
.../selftests/bpf/progs/get_cgroup_id_kern.c | 26 ++---
.../testing/selftests/bpf/progs/netcnt_prog.c | 20 ++--
tools/testing/selftests/bpf/progs/pyperf.h | 90 +++++++-------
.../selftests/bpf/progs/sample_map_ret0.c | 24 ++--
.../selftests/bpf/progs/socket_cookie_prog.c | 13 +--
.../bpf/progs/sockmap_verdict_prog.c | 48 ++++----
.../testing/selftests/bpf/progs/strobemeta.h | 68 +++++------
.../selftests/bpf/progs/test_btf_newkv.c | 13 +--
.../bpf/progs/test_get_stack_rawtp.c | 39 +++----
.../selftests/bpf/progs/test_global_data.c | 37 +++---
tools/testing/selftests/bpf/progs/test_l4lb.c | 65 ++++-------
.../selftests/bpf/progs/test_l4lb_noinline.c | 65 ++++-------
.../selftests/bpf/progs/test_map_in_map.c | 30 ++---
.../selftests/bpf/progs/test_map_lock.c | 26 ++---
.../testing/selftests/bpf/progs/test_obj_id.c | 12 +-
.../bpf/progs/test_select_reuseport_kern.c | 67 ++++-------
.../bpf/progs/test_send_signal_kern.c | 26 ++---
.../bpf/progs/test_sock_fields_kern.c | 78 +++++--------
.../selftests/bpf/progs/test_spin_lock.c | 36 +++---
.../bpf/progs/test_stacktrace_build_id.c | 55 ++++-----
.../selftests/bpf/progs/test_stacktrace_map.c | 52 +++------
.../selftests/bpf/progs/test_tcp_estats.c | 13 +--
.../selftests/bpf/progs/test_tcpbpf_kern.c | 26 ++---
.../selftests/bpf/progs/test_tcpnotify_kern.c | 28 ++---
tools/testing/selftests/bpf/progs/test_xdp.c | 26 ++---
.../selftests/bpf/progs/test_xdp_loop.c | 26 ++---
.../selftests/bpf/progs/test_xdp_noinline.c | 81 +++++--------
.../selftests/bpf/progs/xdp_redirect_map.c | 12 +-
.../testing/selftests/bpf/progs/xdping_kern.c | 12 +-
.../selftests/bpf/test_queue_stack_map.h | 30 ++---
.../testing/selftests/bpf/test_sockmap_kern.h | 110 +++++++++---------
34 files changed, 571 insertions(+), 772 deletions(-)
--
2.17.1
^ permalink raw reply
* [PATCH v4 bpf-next 1/4] libbpf: capture value in BTF type info for BTF-defined map defs
From: Andrii Nakryiko @ 2019-07-04 0:49 UTC (permalink / raw)
To: andrii.nakryiko, kernel-team, bpf, netdev, ast, daniel; +Cc: Andrii Nakryiko
In-Reply-To: <20190704004911.978460-1-andriin@fb.com>
Change BTF-defined map definitions to capture compile-time integer
values as part of BTF type definition, to avoid split of key/value type
information and actual type/size/flags initialization for maps.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Song Liu <songliubraving@fb.com>
Acked-by: Yonghong Song <yhs@fb.com>
---
tools/lib/bpf/libbpf.c | 58 ++++++++++++++++++++----------------------
1 file changed, 28 insertions(+), 30 deletions(-)
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 4907997289e9..fad8901ee774 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -1028,40 +1028,40 @@ static const struct btf_type *skip_mods_and_typedefs(const struct btf *btf,
}
}
-static bool get_map_field_int(const char *map_name,
- const struct btf *btf,
+/*
+ * Fetch integer attribute of BTF map definition. Such attributes are
+ * represented using a pointer to an array, in which dimensionality of array
+ * encodes specified integer value. E.g., int (*type)[BPF_MAP_TYPE_ARRAY];
+ * encodes `type => BPF_MAP_TYPE_ARRAY` key/value pair completely using BTF
+ * type definition, while using only sizeof(void *) space in ELF data section.
+ */
+static bool get_map_field_int(const char *map_name, const struct btf *btf,
const struct btf_type *def,
- const struct btf_member *m,
- const void *data, __u32 *res) {
+ const struct btf_member *m, __u32 *res) {
const struct btf_type *t = skip_mods_and_typedefs(btf, m->type);
const char *name = btf__name_by_offset(btf, m->name_off);
- __u32 int_info = *(const __u32 *)(const void *)(t + 1);
+ const struct btf_array *arr_info;
+ const struct btf_type *arr_t;
- if (BTF_INFO_KIND(t->info) != BTF_KIND_INT) {
- pr_warning("map '%s': attr '%s': expected INT, got %u.\n",
+ if (BTF_INFO_KIND(t->info) != BTF_KIND_PTR) {
+ pr_warning("map '%s': attr '%s': expected PTR, got %u.\n",
map_name, name, BTF_INFO_KIND(t->info));
return false;
}
- if (t->size != 4 || BTF_INT_BITS(int_info) != 32 ||
- BTF_INT_OFFSET(int_info)) {
- pr_warning("map '%s': attr '%s': expected 32-bit non-bitfield integer, "
- "got %u-byte (%d-bit) one with bit offset %d.\n",
- map_name, name, t->size, BTF_INT_BITS(int_info),
- BTF_INT_OFFSET(int_info));
- return false;
- }
- if (BTF_INFO_KFLAG(def->info) && BTF_MEMBER_BITFIELD_SIZE(m->offset)) {
- pr_warning("map '%s': attr '%s': bitfield is not supported.\n",
- map_name, name);
+
+ arr_t = btf__type_by_id(btf, t->type);
+ if (!arr_t) {
+ pr_warning("map '%s': attr '%s': type [%u] not found.\n",
+ map_name, name, t->type);
return false;
}
- if (m->offset % 32) {
- pr_warning("map '%s': attr '%s': unaligned fields are not supported.\n",
- map_name, name);
+ if (BTF_INFO_KIND(arr_t->info) != BTF_KIND_ARRAY) {
+ pr_warning("map '%s': attr '%s': expected ARRAY, got %u.\n",
+ map_name, name, BTF_INFO_KIND(arr_t->info));
return false;
}
-
- *res = *(const __u32 *)(data + m->offset / 8);
+ arr_info = (const void *)(arr_t + 1);
+ *res = arr_info->nelems;
return true;
}
@@ -1074,7 +1074,6 @@ static int bpf_object__init_user_btf_map(struct bpf_object *obj,
const struct btf_var_secinfo *vi;
const struct btf_var *var_extra;
const struct btf_member *m;
- const void *def_data;
const char *map_name;
struct bpf_map *map;
int vlen, i;
@@ -1131,7 +1130,6 @@ static int bpf_object__init_user_btf_map(struct bpf_object *obj,
pr_debug("map '%s': at sec_idx %d, offset %zu.\n",
map_name, map->sec_idx, map->sec_offset);
- def_data = data->d_buf + vi->offset;
vlen = BTF_INFO_VLEN(def->info);
m = (const void *)(def + 1);
for (i = 0; i < vlen; i++, m++) {
@@ -1144,19 +1142,19 @@ static int bpf_object__init_user_btf_map(struct bpf_object *obj,
}
if (strcmp(name, "type") == 0) {
if (!get_map_field_int(map_name, obj->btf, def, m,
- def_data, &map->def.type))
+ &map->def.type))
return -EINVAL;
pr_debug("map '%s': found type = %u.\n",
map_name, map->def.type);
} else if (strcmp(name, "max_entries") == 0) {
if (!get_map_field_int(map_name, obj->btf, def, m,
- def_data, &map->def.max_entries))
+ &map->def.max_entries))
return -EINVAL;
pr_debug("map '%s': found max_entries = %u.\n",
map_name, map->def.max_entries);
} else if (strcmp(name, "map_flags") == 0) {
if (!get_map_field_int(map_name, obj->btf, def, m,
- def_data, &map->def.map_flags))
+ &map->def.map_flags))
return -EINVAL;
pr_debug("map '%s': found map_flags = %u.\n",
map_name, map->def.map_flags);
@@ -1164,7 +1162,7 @@ static int bpf_object__init_user_btf_map(struct bpf_object *obj,
__u32 sz;
if (!get_map_field_int(map_name, obj->btf, def, m,
- def_data, &sz))
+ &sz))
return -EINVAL;
pr_debug("map '%s': found key_size = %u.\n",
map_name, sz);
@@ -1207,7 +1205,7 @@ static int bpf_object__init_user_btf_map(struct bpf_object *obj,
__u32 sz;
if (!get_map_field_int(map_name, obj->btf, def, m,
- def_data, &sz))
+ &sz))
return -EINVAL;
pr_debug("map '%s': found value_size = %u.\n",
map_name, sz);
--
2.17.1
^ permalink raw reply related
* [PATCH v4 bpf-next 2/4] selftests/bpf: add __uint and __type macro for BTF-defined maps
From: Andrii Nakryiko @ 2019-07-04 0:49 UTC (permalink / raw)
To: andrii.nakryiko, kernel-team, bpf, netdev, ast, daniel; +Cc: Andrii Nakryiko
In-Reply-To: <20190704004911.978460-1-andriin@fb.com>
Add simple __uint and __type macro that hide details of how type and
integer values are captured in BTF-defined maps.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Song Liu <songliubraving@fb.com>
Acked-by: Yonghong Song <yhs@fb.com>
---
tools/testing/selftests/bpf/bpf_helpers.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/tools/testing/selftests/bpf/bpf_helpers.h b/tools/testing/selftests/bpf/bpf_helpers.h
index 1a5b1accf091..5a3d92c8bec8 100644
--- a/tools/testing/selftests/bpf/bpf_helpers.h
+++ b/tools/testing/selftests/bpf/bpf_helpers.h
@@ -8,6 +8,9 @@
*/
#define SEC(NAME) __attribute__((section(NAME), used))
+#define __uint(name, val) int (*name)[val]
+#define __type(name, val) val *name
+
/* helper macro to print out debug messages */
#define bpf_printk(fmt, ...) \
({ \
--
2.17.1
^ permalink raw reply related
* [PATCH v4 bpf-next 4/4] selftests/bpf: convert legacy BPF maps to BTF-defined ones
From: Andrii Nakryiko @ 2019-07-04 0:49 UTC (permalink / raw)
To: andrii.nakryiko, kernel-team, bpf, netdev, ast, daniel; +Cc: Andrii Nakryiko
In-Reply-To: <20190704004911.978460-1-andriin@fb.com>
Convert selftests that were originally left out and new ones added
recently to consistently use BTF-defined maps.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Song Liu <songliubraving@fb.com>
Acked-by: Yonghong Song <yhs@fb.com>
---
.../selftests/bpf/progs/get_cgroup_id_kern.c | 26 ++---
tools/testing/selftests/bpf/progs/pyperf.h | 90 +++++++-------
.../selftests/bpf/progs/sample_map_ret0.c | 24 ++--
.../bpf/progs/sockmap_verdict_prog.c | 48 ++++----
.../testing/selftests/bpf/progs/strobemeta.h | 68 +++++------
.../selftests/bpf/progs/test_map_in_map.c | 30 ++---
.../testing/selftests/bpf/progs/test_obj_id.c | 12 +-
.../selftests/bpf/progs/test_xdp_loop.c | 26 ++---
.../selftests/bpf/progs/xdp_redirect_map.c | 12 +-
.../testing/selftests/bpf/progs/xdping_kern.c | 12 +-
.../selftests/bpf/test_queue_stack_map.h | 30 ++---
.../testing/selftests/bpf/test_sockmap_kern.h | 110 +++++++++---------
12 files changed, 240 insertions(+), 248 deletions(-)
diff --git a/tools/testing/selftests/bpf/progs/get_cgroup_id_kern.c b/tools/testing/selftests/bpf/progs/get_cgroup_id_kern.c
index 014dba10b8a5..16c54ade6888 100644
--- a/tools/testing/selftests/bpf/progs/get_cgroup_id_kern.c
+++ b/tools/testing/selftests/bpf/progs/get_cgroup_id_kern.c
@@ -4,19 +4,19 @@
#include <linux/bpf.h>
#include "bpf_helpers.h"
-struct bpf_map_def SEC("maps") cg_ids = {
- .type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u64),
- .max_entries = 1,
-};
-
-struct bpf_map_def SEC("maps") pidmap = {
- .type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u32),
- .max_entries = 1,
-};
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, __u32);
+ __type(value, __u64);
+} cg_ids SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, __u32);
+ __type(value, __u32);
+} pidmap SEC(".maps");
SEC("tracepoint/syscalls/sys_enter_nanosleep")
int trace(void *ctx)
diff --git a/tools/testing/selftests/bpf/progs/pyperf.h b/tools/testing/selftests/bpf/progs/pyperf.h
index abf6224649be..003fe106fc70 100644
--- a/tools/testing/selftests/bpf/progs/pyperf.h
+++ b/tools/testing/selftests/bpf/progs/pyperf.h
@@ -58,14 +58,6 @@ typedef struct {
} Event;
-struct bpf_elf_map {
- __u32 type;
- __u32 size_key;
- __u32 size_value;
- __u32 max_elem;
- __u32 flags;
-};
-
typedef int pid_t;
typedef struct {
@@ -118,47 +110,47 @@ static __always_inline bool get_frame_data(void *frame_ptr, PidData *pidData,
return true;
}
-struct bpf_elf_map SEC("maps") pidmap = {
- .type = BPF_MAP_TYPE_HASH,
- .size_key = sizeof(int),
- .size_value = sizeof(PidData),
- .max_elem = 1,
-};
-
-struct bpf_elf_map SEC("maps") eventmap = {
- .type = BPF_MAP_TYPE_HASH,
- .size_key = sizeof(int),
- .size_value = sizeof(Event),
- .max_elem = 1,
-};
-
-struct bpf_elf_map SEC("maps") symbolmap = {
- .type = BPF_MAP_TYPE_HASH,
- .size_key = sizeof(Symbol),
- .size_value = sizeof(int),
- .max_elem = 1,
-};
-
-struct bpf_elf_map SEC("maps") statsmap = {
- .type = BPF_MAP_TYPE_ARRAY,
- .size_key = sizeof(Stats),
- .size_value = sizeof(int),
- .max_elem = 1,
-};
-
-struct bpf_elf_map SEC("maps") perfmap = {
- .type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
- .size_key = sizeof(int),
- .size_value = sizeof(int),
- .max_elem = 32,
-};
-
-struct bpf_elf_map SEC("maps") stackmap = {
- .type = BPF_MAP_TYPE_STACK_TRACE,
- .size_key = sizeof(int),
- .size_value = sizeof(long long) * 127,
- .max_elem = 1000,
-};
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, 1);
+ __type(key, int);
+ __type(value, PidData);
+} pidmap SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, 1);
+ __type(key, int);
+ __type(value, Event);
+} eventmap SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, 1);
+ __type(key, Symbol);
+ __type(value, int);
+} symbolmap SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, int);
+ __type(value, Stats);
+} statsmap SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY);
+ __uint(max_entries, 32);
+ __uint(key_size, sizeof(int));
+ __uint(value_size, sizeof(int));
+} perfmap SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_STACK_TRACE);
+ __uint(max_entries, 1000);
+ __uint(key_size, sizeof(int));
+ __uint(value_size, sizeof(long long) * 127);
+} stackmap SEC(".maps");
static __always_inline int __on_event(struct pt_regs *ctx)
{
diff --git a/tools/testing/selftests/bpf/progs/sample_map_ret0.c b/tools/testing/selftests/bpf/progs/sample_map_ret0.c
index 0756303676ac..69464f747355 100644
--- a/tools/testing/selftests/bpf/progs/sample_map_ret0.c
+++ b/tools/testing/selftests/bpf/progs/sample_map_ret0.c
@@ -2,19 +2,19 @@
#include <linux/bpf.h>
#include "bpf_helpers.h"
-struct bpf_map_def SEC("maps") htab = {
- .type = BPF_MAP_TYPE_HASH,
- .key_size = sizeof(__u32),
- .value_size = sizeof(long),
- .max_entries = 2,
-};
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, 2);
+ __type(key, __u32);
+ __type(value, long);
+} htab SEC(".maps");
-struct bpf_map_def SEC("maps") array = {
- .type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(long),
- .max_entries = 2,
-};
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 2);
+ __type(key, __u32);
+ __type(value, long);
+} array SEC(".maps");
/* Sample program which should always load for testing control paths. */
SEC(".text") int func()
diff --git a/tools/testing/selftests/bpf/progs/sockmap_verdict_prog.c b/tools/testing/selftests/bpf/progs/sockmap_verdict_prog.c
index d85c874ef25e..433e23918a62 100644
--- a/tools/testing/selftests/bpf/progs/sockmap_verdict_prog.c
+++ b/tools/testing/selftests/bpf/progs/sockmap_verdict_prog.c
@@ -4,33 +4,33 @@
int _version SEC("version") = 1;
-struct bpf_map_def SEC("maps") sock_map_rx = {
- .type = BPF_MAP_TYPE_SOCKMAP,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- .max_entries = 20,
-};
+struct {
+ __uint(type, BPF_MAP_TYPE_SOCKMAP);
+ __uint(max_entries, 20);
+ __uint(key_size, sizeof(int));
+ __uint(value_size, sizeof(int));
+} sock_map_rx SEC(".maps");
-struct bpf_map_def SEC("maps") sock_map_tx = {
- .type = BPF_MAP_TYPE_SOCKMAP,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- .max_entries = 20,
-};
+struct {
+ __uint(type, BPF_MAP_TYPE_SOCKMAP);
+ __uint(max_entries, 20);
+ __uint(key_size, sizeof(int));
+ __uint(value_size, sizeof(int));
+} sock_map_tx SEC(".maps");
-struct bpf_map_def SEC("maps") sock_map_msg = {
- .type = BPF_MAP_TYPE_SOCKMAP,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- .max_entries = 20,
-};
+struct {
+ __uint(type, BPF_MAP_TYPE_SOCKMAP);
+ __uint(max_entries, 20);
+ __uint(key_size, sizeof(int));
+ __uint(value_size, sizeof(int));
+} sock_map_msg SEC(".maps");
-struct bpf_map_def SEC("maps") sock_map_break = {
- .type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- .max_entries = 20,
-};
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 20);
+ __type(key, int);
+ __type(value, int);
+} sock_map_break SEC(".maps");
SEC("sk_skb2")
int bpf_prog2(struct __sk_buff *skb)
diff --git a/tools/testing/selftests/bpf/progs/strobemeta.h b/tools/testing/selftests/bpf/progs/strobemeta.h
index 553bc3b62e89..8a399bdfd920 100644
--- a/tools/testing/selftests/bpf/progs/strobemeta.h
+++ b/tools/testing/selftests/bpf/progs/strobemeta.h
@@ -204,40 +204,40 @@ struct strobelight_bpf_sample {
char dummy_safeguard;
};
-struct bpf_map_def SEC("maps") samples = {
- .type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- .max_entries = 32,
-};
-
-struct bpf_map_def SEC("maps") stacks_0 = {
- .type = BPF_MAP_TYPE_STACK_TRACE,
- .key_size = sizeof(uint32_t),
- .value_size = sizeof(uint64_t) * PERF_MAX_STACK_DEPTH,
- .max_entries = 16,
-};
-
-struct bpf_map_def SEC("maps") stacks_1 = {
- .type = BPF_MAP_TYPE_STACK_TRACE,
- .key_size = sizeof(uint32_t),
- .value_size = sizeof(uint64_t) * PERF_MAX_STACK_DEPTH,
- .max_entries = 16,
-};
-
-struct bpf_map_def SEC("maps") sample_heap = {
- .type = BPF_MAP_TYPE_PERCPU_ARRAY,
- .key_size = sizeof(uint32_t),
- .value_size = sizeof(struct strobelight_bpf_sample),
- .max_entries = 1,
-};
-
-struct bpf_map_def SEC("maps") strobemeta_cfgs = {
- .type = BPF_MAP_TYPE_PERCPU_ARRAY,
- .key_size = sizeof(pid_t),
- .value_size = sizeof(struct strobemeta_cfg),
- .max_entries = STROBE_MAX_CFGS,
-};
+struct {
+ __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY);
+ __uint(max_entries, 32);
+ __uint(key_size, sizeof(int));
+ __uint(value_size, sizeof(int));
+} samples SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_STACK_TRACE);
+ __uint(max_entries, 16);
+ __uint(key_size, sizeof(uint32_t));
+ __uint(value_size, sizeof(uint64_t) * PERF_MAX_STACK_DEPTH);
+} stacks_0 SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_STACK_TRACE);
+ __uint(max_entries, 16);
+ __uint(key_size, sizeof(uint32_t));
+ __uint(value_size, sizeof(uint64_t) * PERF_MAX_STACK_DEPTH);
+} stacks_1 SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, uint32_t);
+ __type(value, struct strobelight_bpf_sample);
+} sample_heap SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+ __uint(max_entries, STROBE_MAX_CFGS);
+ __type(key, pid_t);
+ __type(value, struct strobemeta_cfg);
+} strobemeta_cfgs SEC(".maps");
/* Type for the dtv. */
/* https://github.com/lattera/glibc/blob/master/nptl/sysdeps/x86_64/tls.h#L34 */
diff --git a/tools/testing/selftests/bpf/progs/test_map_in_map.c b/tools/testing/selftests/bpf/progs/test_map_in_map.c
index 2985f262846e..113226115365 100644
--- a/tools/testing/selftests/bpf/progs/test_map_in_map.c
+++ b/tools/testing/selftests/bpf/progs/test_map_in_map.c
@@ -5,23 +5,23 @@
#include <linux/types.h>
#include "bpf_helpers.h"
-struct bpf_map_def SEC("maps") mim_array = {
- .type = BPF_MAP_TYPE_ARRAY_OF_MAPS,
- .key_size = sizeof(int),
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY_OF_MAPS);
+ __uint(max_entries, 1);
+ __uint(map_flags, 0);
+ __uint(key_size, sizeof(__u32));
/* must be sizeof(__u32) for map in map */
- .value_size = sizeof(__u32),
- .max_entries = 1,
- .map_flags = 0,
-};
-
-struct bpf_map_def SEC("maps") mim_hash = {
- .type = BPF_MAP_TYPE_HASH_OF_MAPS,
- .key_size = sizeof(int),
+ __uint(value_size, sizeof(__u32));
+} mim_array SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH_OF_MAPS);
+ __uint(max_entries, 1);
+ __uint(map_flags, 0);
+ __uint(key_size, sizeof(int));
/* must be sizeof(__u32) for map in map */
- .value_size = sizeof(__u32),
- .max_entries = 1,
- .map_flags = 0,
-};
+ __uint(value_size, sizeof(__u32));
+} mim_hash SEC(".maps");
SEC("xdp_mimtest")
int xdp_mimtest0(struct xdp_md *ctx)
diff --git a/tools/testing/selftests/bpf/progs/test_obj_id.c b/tools/testing/selftests/bpf/progs/test_obj_id.c
index 726340fa6fe0..3d30c02bdae9 100644
--- a/tools/testing/selftests/bpf/progs/test_obj_id.c
+++ b/tools/testing/selftests/bpf/progs/test_obj_id.c
@@ -13,12 +13,12 @@
int _version SEC("version") = 1;
-struct bpf_map_def SEC("maps") test_map_id = {
- .type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u64),
- .max_entries = 1,
-};
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, __u32);
+ __type(value, __u64);
+} test_map_id SEC(".maps");
SEC("test_obj_id_dummy")
int test_obj_id(struct __sk_buff *skb)
diff --git a/tools/testing/selftests/bpf/progs/test_xdp_loop.c b/tools/testing/selftests/bpf/progs/test_xdp_loop.c
index 7fa4677df22e..97175f73c3fe 100644
--- a/tools/testing/selftests/bpf/progs/test_xdp_loop.c
+++ b/tools/testing/selftests/bpf/progs/test_xdp_loop.c
@@ -18,19 +18,19 @@
int _version SEC("version") = 1;
-struct bpf_map_def SEC("maps") rxcnt = {
- .type = BPF_MAP_TYPE_PERCPU_ARRAY,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u64),
- .max_entries = 256,
-};
-
-struct bpf_map_def SEC("maps") vip2tnl = {
- .type = BPF_MAP_TYPE_HASH,
- .key_size = sizeof(struct vip),
- .value_size = sizeof(struct iptnl_info),
- .max_entries = MAX_IPTNL_ENTRIES,
-};
+struct {
+ __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+ __uint(max_entries, 256);
+ __type(key, __u32);
+ __type(value, __u64);
+} rxcnt SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, MAX_IPTNL_ENTRIES);
+ __type(key, struct vip);
+ __type(value, struct iptnl_info);
+} vip2tnl SEC(".maps");
static __always_inline void count_tx(__u32 protocol)
{
diff --git a/tools/testing/selftests/bpf/progs/xdp_redirect_map.c b/tools/testing/selftests/bpf/progs/xdp_redirect_map.c
index e87a985b9df9..1c5f298d7196 100644
--- a/tools/testing/selftests/bpf/progs/xdp_redirect_map.c
+++ b/tools/testing/selftests/bpf/progs/xdp_redirect_map.c
@@ -3,12 +3,12 @@
#include <linux/bpf.h>
#include "bpf_helpers.h"
-struct bpf_map_def SEC("maps") tx_port = {
- .type = BPF_MAP_TYPE_DEVMAP,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- .max_entries = 8,
-};
+struct {
+ __uint(type, BPF_MAP_TYPE_DEVMAP);
+ __uint(max_entries, 8);
+ __uint(key_size, sizeof(int));
+ __uint(value_size, sizeof(int));
+} tx_port SEC(".maps");
SEC("redirect_map_0")
int xdp_redirect_map_0(struct xdp_md *xdp)
diff --git a/tools/testing/selftests/bpf/progs/xdping_kern.c b/tools/testing/selftests/bpf/progs/xdping_kern.c
index 87393e7c667c..112a2857f4e2 100644
--- a/tools/testing/selftests/bpf/progs/xdping_kern.c
+++ b/tools/testing/selftests/bpf/progs/xdping_kern.c
@@ -17,12 +17,12 @@
#include "xdping.h"
-struct bpf_map_def SEC("maps") ping_map = {
- .type = BPF_MAP_TYPE_HASH,
- .key_size = sizeof(__u32),
- .value_size = sizeof(struct pinginfo),
- .max_entries = 256,
-};
+struct {
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, 256);
+ __type(key, __u32);
+ __type(value, struct pinginfo);
+} ping_map SEC(".maps");
static __always_inline void swap_src_dst_mac(void *data)
{
diff --git a/tools/testing/selftests/bpf/test_queue_stack_map.h b/tools/testing/selftests/bpf/test_queue_stack_map.h
index 295b9b3bc5c7..0e014d3b2b36 100644
--- a/tools/testing/selftests/bpf/test_queue_stack_map.h
+++ b/tools/testing/selftests/bpf/test_queue_stack_map.h
@@ -10,21 +10,21 @@
int _version SEC("version") = 1;
-struct bpf_map_def __attribute__ ((section("maps"), used)) map_in = {
- .type = MAP_TYPE,
- .key_size = 0,
- .value_size = sizeof(__u32),
- .max_entries = 32,
- .map_flags = 0,
-};
-
-struct bpf_map_def __attribute__ ((section("maps"), used)) map_out = {
- .type = MAP_TYPE,
- .key_size = 0,
- .value_size = sizeof(__u32),
- .max_entries = 32,
- .map_flags = 0,
-};
+struct {
+ __uint(type, MAP_TYPE);
+ __uint(max_entries, 32);
+ __uint(map_flags, 0);
+ __uint(key_size, 0);
+ __uint(value_size, sizeof(__u32));
+} map_in SEC(".maps");
+
+struct {
+ __uint(type, MAP_TYPE);
+ __uint(max_entries, 32);
+ __uint(map_flags, 0);
+ __uint(key_size, 0);
+ __uint(value_size, sizeof(__u32));
+} map_out SEC(".maps");
SEC("test")
int _test(struct __sk_buff *skb)
diff --git a/tools/testing/selftests/bpf/test_sockmap_kern.h b/tools/testing/selftests/bpf/test_sockmap_kern.h
index 4e7d3da21357..d008b41b7d8d 100644
--- a/tools/testing/selftests/bpf/test_sockmap_kern.h
+++ b/tools/testing/selftests/bpf/test_sockmap_kern.h
@@ -28,61 +28,61 @@
* are established and verdicts are decided.
*/
-struct bpf_map_def SEC("maps") sock_map = {
- .type = TEST_MAP_TYPE,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- .max_entries = 20,
-};
-
-struct bpf_map_def SEC("maps") sock_map_txmsg = {
- .type = TEST_MAP_TYPE,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- .max_entries = 20,
-};
-
-struct bpf_map_def SEC("maps") sock_map_redir = {
- .type = TEST_MAP_TYPE,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- .max_entries = 20,
-};
-
-struct bpf_map_def SEC("maps") sock_apply_bytes = {
- .type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- .max_entries = 1
-};
-
-struct bpf_map_def SEC("maps") sock_cork_bytes = {
- .type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- .max_entries = 1
-};
-
-struct bpf_map_def SEC("maps") sock_bytes = {
- .type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- .max_entries = 6
-};
-
-struct bpf_map_def SEC("maps") sock_redir_flags = {
- .type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- .max_entries = 1
-};
-
-struct bpf_map_def SEC("maps") sock_skb_opts = {
- .type = BPF_MAP_TYPE_ARRAY,
- .key_size = sizeof(int),
- .value_size = sizeof(int),
- .max_entries = 1
-};
+struct {
+ __uint(type, TEST_MAP_TYPE);
+ __uint(max_entries, 20);
+ __uint(key_size, sizeof(int));
+ __uint(value_size, sizeof(int));
+} sock_map SEC(".maps");
+
+struct {
+ __uint(type, TEST_MAP_TYPE);
+ __uint(max_entries, 20);
+ __uint(key_size, sizeof(int));
+ __uint(value_size, sizeof(int));
+} sock_map_txmsg SEC(".maps");
+
+struct {
+ __uint(type, TEST_MAP_TYPE);
+ __uint(max_entries, 20);
+ __uint(key_size, sizeof(int));
+ __uint(value_size, sizeof(int));
+} sock_map_redir SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, int);
+ __type(value, int);
+} sock_apply_bytes SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, int);
+ __type(value, int);
+} sock_cork_bytes SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 6);
+ __type(key, int);
+ __type(value, int);
+} sock_bytes SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, int);
+ __type(value, int);
+} sock_redir_flags SEC(".maps");
+
+struct {
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, int);
+ __type(value, int);
+} sock_skb_opts SEC(".maps");
SEC("sk_skb1")
int bpf_prog1(struct __sk_buff *skb)
--
2.17.1
^ permalink raw reply related
* [PATCH v4 bpf-next 3/4] selftests/bpf: convert selftests using BTF-defined maps to new syntax
From: Andrii Nakryiko @ 2019-07-04 0:49 UTC (permalink / raw)
To: andrii.nakryiko, kernel-team, bpf, netdev, ast, daniel; +Cc: Andrii Nakryiko
In-Reply-To: <20190704004911.978460-1-andriin@fb.com>
Convert all the existing selftests that are already using BTF-defined
maps to use new syntax (with no static data initialization).
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Song Liu <songliubraving@fb.com>
Acked-by: Yonghong Song <yhs@fb.com>
---
tools/testing/selftests/bpf/progs/bpf_flow.c | 28 +++----
.../testing/selftests/bpf/progs/netcnt_prog.c | 20 ++---
.../selftests/bpf/progs/socket_cookie_prog.c | 13 ++-
.../selftests/bpf/progs/test_btf_newkv.c | 13 ++-
.../bpf/progs/test_get_stack_rawtp.c | 39 ++++-----
.../selftests/bpf/progs/test_global_data.c | 37 ++++-----
tools/testing/selftests/bpf/progs/test_l4lb.c | 65 ++++++---------
.../selftests/bpf/progs/test_l4lb_noinline.c | 65 ++++++---------
.../selftests/bpf/progs/test_map_lock.c | 26 +++---
.../bpf/progs/test_select_reuseport_kern.c | 67 ++++++---------
.../bpf/progs/test_send_signal_kern.c | 26 +++---
.../bpf/progs/test_sock_fields_kern.c | 78 +++++++-----------
.../selftests/bpf/progs/test_spin_lock.c | 36 ++++-----
.../bpf/progs/test_stacktrace_build_id.c | 55 +++++--------
.../selftests/bpf/progs/test_stacktrace_map.c | 52 +++++-------
.../selftests/bpf/progs/test_tcp_estats.c | 13 ++-
.../selftests/bpf/progs/test_tcpbpf_kern.c | 26 +++---
.../selftests/bpf/progs/test_tcpnotify_kern.c | 28 +++----
tools/testing/selftests/bpf/progs/test_xdp.c | 26 +++---
.../selftests/bpf/progs/test_xdp_noinline.c | 81 +++++++------------
20 files changed, 300 insertions(+), 494 deletions(-)
diff --git a/tools/testing/selftests/bpf/progs/bpf_flow.c b/tools/testing/selftests/bpf/progs/bpf_flow.c
index 849f42e548b5..5ae485a6af3f 100644
--- a/tools/testing/selftests/bpf/progs/bpf_flow.c
+++ b/tools/testing/selftests/bpf/progs/bpf_flow.c
@@ -58,26 +58,18 @@ struct frag_hdr {
};
struct {
- __u32 type;
- __u32 max_entries;
- __u32 key_size;
- __u32 value_size;
-} jmp_table SEC(".maps") = {
- .type = BPF_MAP_TYPE_PROG_ARRAY,
- .max_entries = 8,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u32),
-};
+ __uint(type, BPF_MAP_TYPE_PROG_ARRAY);
+ __uint(max_entries, 8);
+ __uint(key_size, sizeof(__u32));
+ __uint(value_size, sizeof(__u32));
+} jmp_table SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct bpf_flow_keys *value;
-} last_dissection SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 1,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, __u32);
+ __type(value, struct bpf_flow_keys);
+} last_dissection SEC(".maps");
static __always_inline int export_flow_keys(struct bpf_flow_keys *keys,
int ret)
diff --git a/tools/testing/selftests/bpf/progs/netcnt_prog.c b/tools/testing/selftests/bpf/progs/netcnt_prog.c
index a25c82a5b7c8..38a997852cad 100644
--- a/tools/testing/selftests/bpf/progs/netcnt_prog.c
+++ b/tools/testing/selftests/bpf/progs/netcnt_prog.c
@@ -11,20 +11,16 @@
#define NS_PER_SEC 1000000000
struct {
- __u32 type;
- struct bpf_cgroup_storage_key *key;
- struct percpu_net_cnt *value;
-} percpu_netcnt SEC(".maps") = {
- .type = BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE,
-};
+ __uint(type, BPF_MAP_TYPE_PERCPU_CGROUP_STORAGE);
+ __type(key, struct bpf_cgroup_storage_key);
+ __type(value, struct percpu_net_cnt);
+} percpu_netcnt SEC(".maps");
struct {
- __u32 type;
- struct bpf_cgroup_storage_key *key;
- struct net_cnt *value;
-} netcnt SEC(".maps") = {
- .type = BPF_MAP_TYPE_CGROUP_STORAGE,
-};
+ __uint(type, BPF_MAP_TYPE_CGROUP_STORAGE);
+ __type(key, struct bpf_cgroup_storage_key);
+ __type(value, struct net_cnt);
+} netcnt SEC(".maps");
SEC("cgroup/skb")
int bpf_nextcnt(struct __sk_buff *skb)
diff --git a/tools/testing/selftests/bpf/progs/socket_cookie_prog.c b/tools/testing/selftests/bpf/progs/socket_cookie_prog.c
index 6aabb681fb9a..e4440fdd94cb 100644
--- a/tools/testing/selftests/bpf/progs/socket_cookie_prog.c
+++ b/tools/testing/selftests/bpf/progs/socket_cookie_prog.c
@@ -13,14 +13,11 @@ struct socket_cookie {
};
struct {
- __u32 type;
- __u32 map_flags;
- int *key;
- struct socket_cookie *value;
-} socket_cookies SEC(".maps") = {
- .type = BPF_MAP_TYPE_SK_STORAGE,
- .map_flags = BPF_F_NO_PREALLOC,
-};
+ __uint(type, BPF_MAP_TYPE_SK_STORAGE);
+ __uint(map_flags, BPF_F_NO_PREALLOC);
+ __type(key, int);
+ __type(value, struct socket_cookie);
+} socket_cookies SEC(".maps");
SEC("cgroup/connect6")
int set_cookie(struct bpf_sock_addr *ctx)
diff --git a/tools/testing/selftests/bpf/progs/test_btf_newkv.c b/tools/testing/selftests/bpf/progs/test_btf_newkv.c
index 28c16bb583b6..5ee3622ddebb 100644
--- a/tools/testing/selftests/bpf/progs/test_btf_newkv.c
+++ b/tools/testing/selftests/bpf/progs/test_btf_newkv.c
@@ -21,14 +21,11 @@ struct bpf_map_def SEC("maps") btf_map_legacy = {
BPF_ANNOTATE_KV_PAIR(btf_map_legacy, int, struct ipv_counts);
struct {
- int *key;
- struct ipv_counts *value;
- unsigned int type;
- unsigned int max_entries;
-} btf_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 4,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 4);
+ __type(key, int);
+ __type(value, struct ipv_counts);
+} btf_map SEC(".maps");
struct dummy_tracepoint_args {
unsigned long long pad;
diff --git a/tools/testing/selftests/bpf/progs/test_get_stack_rawtp.c b/tools/testing/selftests/bpf/progs/test_get_stack_rawtp.c
index aaa6ec250e15..d06b47a09097 100644
--- a/tools/testing/selftests/bpf/progs/test_get_stack_rawtp.c
+++ b/tools/testing/selftests/bpf/progs/test_get_stack_rawtp.c
@@ -16,26 +16,18 @@ struct stack_trace_t {
};
struct {
- __u32 type;
- __u32 max_entries;
- __u32 key_size;
- __u32 value_size;
-} perfmap SEC(".maps") = {
- .type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
- .max_entries = 2,
- .key_size = sizeof(int),
- .value_size = sizeof(__u32),
-};
+ __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY);
+ __uint(max_entries, 2);
+ __uint(key_size, sizeof(int));
+ __uint(value_size, sizeof(__u32));
+} perfmap SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct stack_trace_t *value;
-} stackdata_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_PERCPU_ARRAY,
- .max_entries = 1,
-};
+ __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, __u32);
+ __type(value, struct stack_trace_t);
+} stackdata_map SEC(".maps");
/* Allocate per-cpu space twice the needed. For the code below
* usize = bpf_get_stack(ctx, raw_data, max_len, BPF_F_USER_STACK);
@@ -56,14 +48,11 @@ struct {
* This is an acceptable workaround since there is one entry here.
*/
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
+ __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, __u32);
__u64 (*value)[2 * MAX_STACK_RAWTP];
-} rawdata_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_PERCPU_ARRAY,
- .max_entries = 1,
-};
+} rawdata_map SEC(".maps");
SEC("tracepoint/raw_syscalls/sys_enter")
int bpf_prog1(void *ctx)
diff --git a/tools/testing/selftests/bpf/progs/test_global_data.c b/tools/testing/selftests/bpf/progs/test_global_data.c
index 866cc7ddbe43..32a6073acb99 100644
--- a/tools/testing/selftests/bpf/progs/test_global_data.c
+++ b/tools/testing/selftests/bpf/progs/test_global_data.c
@@ -8,24 +8,18 @@
#include "bpf_helpers.h"
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- __u64 *value;
-} result_number SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 11,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 11);
+ __type(key, __u32);
+ __type(value, __u64);
+} result_number SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 5);
+ __type(key, __u32);
const char (*value)[32];
-} result_string SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 5,
-};
+} result_string SEC(".maps");
struct foo {
__u8 a;
@@ -34,14 +28,11 @@ struct foo {
};
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct foo *value;
-} result_struct SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 5,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 5);
+ __type(key, __u32);
+ __type(value, struct foo);
+} result_struct SEC(".maps");
/* Relocation tests for __u64s. */
static __u64 num0;
diff --git a/tools/testing/selftests/bpf/progs/test_l4lb.c b/tools/testing/selftests/bpf/progs/test_l4lb.c
index 848cbb90f581..1d652ee8e73d 100644
--- a/tools/testing/selftests/bpf/progs/test_l4lb.c
+++ b/tools/testing/selftests/bpf/progs/test_l4lb.c
@@ -170,54 +170,39 @@ struct eth_hdr {
};
struct {
- __u32 type;
- __u32 max_entries;
- struct vip *key;
- struct vip_meta *value;
-} vip_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_HASH,
- .max_entries = MAX_VIPS,
-};
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, MAX_VIPS);
+ __type(key, struct vip);
+ __type(value, struct vip_meta);
+} vip_map SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- __u32 *value;
-} ch_rings SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = CH_RINGS_SIZE,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, CH_RINGS_SIZE);
+ __type(key, __u32);
+ __type(value, __u32);
+} ch_rings SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct real_definition *value;
-} reals SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = MAX_REALS,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, MAX_REALS);
+ __type(key, __u32);
+ __type(value, struct real_definition);
+} reals SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct vip_stats *value;
-} stats SEC(".maps") = {
- .type = BPF_MAP_TYPE_PERCPU_ARRAY,
- .max_entries = MAX_VIPS,
-};
+ __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+ __uint(max_entries, MAX_VIPS);
+ __type(key, __u32);
+ __type(value, struct vip_stats);
+} stats SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct ctl_value *value;
-} ctl_array SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = CTL_MAP_SIZE,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, CTL_MAP_SIZE);
+ __type(key, __u32);
+ __type(value, struct ctl_value);
+} ctl_array SEC(".maps");
static __always_inline __u32 get_packet_hash(struct packet_description *pckt,
bool ipv6)
diff --git a/tools/testing/selftests/bpf/progs/test_l4lb_noinline.c b/tools/testing/selftests/bpf/progs/test_l4lb_noinline.c
index c63ecf3ca573..2e4efe70b1e5 100644
--- a/tools/testing/selftests/bpf/progs/test_l4lb_noinline.c
+++ b/tools/testing/selftests/bpf/progs/test_l4lb_noinline.c
@@ -166,54 +166,39 @@ struct eth_hdr {
};
struct {
- __u32 type;
- __u32 max_entries;
- struct vip *key;
- struct vip_meta *value;
-} vip_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_HASH,
- .max_entries = MAX_VIPS,
-};
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, MAX_VIPS);
+ __type(key, struct vip);
+ __type(value, struct vip_meta);
+} vip_map SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- __u32 *value;
-} ch_rings SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = CH_RINGS_SIZE,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, CH_RINGS_SIZE);
+ __type(key, __u32);
+ __type(value, __u32);
+} ch_rings SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct real_definition *value;
-} reals SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = MAX_REALS,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, MAX_REALS);
+ __type(key, __u32);
+ __type(value, struct real_definition);
+} reals SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct vip_stats *value;
-} stats SEC(".maps") = {
- .type = BPF_MAP_TYPE_PERCPU_ARRAY,
- .max_entries = MAX_VIPS,
-};
+ __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+ __uint(max_entries, MAX_VIPS);
+ __type(key, __u32);
+ __type(value, struct vip_stats);
+} stats SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct ctl_value *value;
-} ctl_array SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = CTL_MAP_SIZE,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, CTL_MAP_SIZE);
+ __type(key, __u32);
+ __type(value, struct ctl_value);
+} ctl_array SEC(".maps");
static __u32 get_packet_hash(struct packet_description *pckt,
bool ipv6)
diff --git a/tools/testing/selftests/bpf/progs/test_map_lock.c b/tools/testing/selftests/bpf/progs/test_map_lock.c
index 40d9c2853393..bb7ce35f691b 100644
--- a/tools/testing/selftests/bpf/progs/test_map_lock.c
+++ b/tools/testing/selftests/bpf/progs/test_map_lock.c
@@ -12,14 +12,11 @@ struct hmap_elem {
};
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct hmap_elem *value;
-} hash_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_HASH,
- .max_entries = 1,
-};
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, 1);
+ __type(key, __u32);
+ __type(value, struct hmap_elem);
+} hash_map SEC(".maps");
struct array_elem {
struct bpf_spin_lock lock;
@@ -27,14 +24,11 @@ struct array_elem {
};
struct {
- __u32 type;
- __u32 max_entries;
- int *key;
- struct array_elem *value;
-} array_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 1,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, int);
+ __type(value, struct array_elem);
+} array_map SEC(".maps");
SEC("map_lock_demo")
int bpf_map_lock_test(struct __sk_buff *skb)
diff --git a/tools/testing/selftests/bpf/progs/test_select_reuseport_kern.c b/tools/testing/selftests/bpf/progs/test_select_reuseport_kern.c
index 435a9527733e..ea7d84f01235 100644
--- a/tools/testing/selftests/bpf/progs/test_select_reuseport_kern.c
+++ b/tools/testing/selftests/bpf/progs/test_select_reuseport_kern.c
@@ -22,56 +22,39 @@ int _version SEC("version") = 1;
#endif
struct {
- __u32 type;
- __u32 max_entries;
- __u32 key_size;
- __u32 value_size;
-} outer_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY_OF_MAPS,
- .max_entries = 1,
- .key_size = sizeof(__u32),
- .value_size = sizeof(__u32),
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY_OF_MAPS);
+ __uint(max_entries, 1);
+ __uint(key_size, sizeof(__u32));
+ __uint(value_size, sizeof(__u32));
+} outer_map SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- __u32 *value;
-} result_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = NR_RESULTS,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, NR_RESULTS);
+ __type(key, __u32);
+ __type(value, __u32);
+} result_map SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- int *value;
-} tmp_index_ovr_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 1,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, __u32);
+ __type(value, int);
+} tmp_index_ovr_map SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- __u32 *value;
-} linum_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 1,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, __u32);
+ __type(value, __u32);
+} linum_map SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct data_check *value;
-} data_check_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 1,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, __u32);
+ __type(value, struct data_check);
+} data_check_map SEC(".maps");
#define GOTO_DONE(_result) ({ \
result = (_result); \
diff --git a/tools/testing/selftests/bpf/progs/test_send_signal_kern.c b/tools/testing/selftests/bpf/progs/test_send_signal_kern.c
index 6ac68be5d68b..0e6be01157e6 100644
--- a/tools/testing/selftests/bpf/progs/test_send_signal_kern.c
+++ b/tools/testing/selftests/bpf/progs/test_send_signal_kern.c
@@ -5,24 +5,18 @@
#include "bpf_helpers.h"
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- __u64 *value;
-} info_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 1,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, __u32);
+ __type(value, __u64);
+} info_map SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- __u64 *value;
-} status_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 1,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, __u32);
+ __type(value, __u64);
+} status_map SEC(".maps");
SEC("send_signal_demo")
int bpf_send_signal_test(void *ctx)
diff --git a/tools/testing/selftests/bpf/progs/test_sock_fields_kern.c b/tools/testing/selftests/bpf/progs/test_sock_fields_kern.c
index c3d383d650cb..a47b003623ef 100644
--- a/tools/testing/selftests/bpf/progs/test_sock_fields_kern.c
+++ b/tools/testing/selftests/bpf/progs/test_sock_fields_kern.c
@@ -28,44 +28,32 @@ enum bpf_linum_array_idx {
};
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct sockaddr_in6 *value;
-} addr_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = __NR_BPF_ADDR_ARRAY_IDX,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, __NR_BPF_ADDR_ARRAY_IDX);
+ __type(key, __u32);
+ __type(value, struct sockaddr_in6);
+} addr_map SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct bpf_sock *value;
-} sock_result_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = __NR_BPF_RESULT_ARRAY_IDX,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, __NR_BPF_RESULT_ARRAY_IDX);
+ __type(key, __u32);
+ __type(value, struct bpf_sock);
+} sock_result_map SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct bpf_tcp_sock *value;
-} tcp_sock_result_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = __NR_BPF_RESULT_ARRAY_IDX,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, __NR_BPF_RESULT_ARRAY_IDX);
+ __type(key, __u32);
+ __type(value, struct bpf_tcp_sock);
+} tcp_sock_result_map SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- __u32 *value;
-} linum_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = __NR_BPF_LINUM_ARRAY_IDX,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, __NR_BPF_LINUM_ARRAY_IDX);
+ __type(key, __u32);
+ __type(value, __u32);
+} linum_map SEC(".maps");
struct bpf_spinlock_cnt {
struct bpf_spin_lock lock;
@@ -73,24 +61,18 @@ struct bpf_spinlock_cnt {
};
struct {
- __u32 type;
- __u32 map_flags;
- int *key;
- struct bpf_spinlock_cnt *value;
-} sk_pkt_out_cnt SEC(".maps") = {
- .type = BPF_MAP_TYPE_SK_STORAGE,
- .map_flags = BPF_F_NO_PREALLOC,
-};
+ __uint(type, BPF_MAP_TYPE_SK_STORAGE);
+ __uint(map_flags, BPF_F_NO_PREALLOC);
+ __type(key, int);
+ __type(value, struct bpf_spinlock_cnt);
+} sk_pkt_out_cnt SEC(".maps");
struct {
- __u32 type;
- __u32 map_flags;
- int *key;
- struct bpf_spinlock_cnt *value;
-} sk_pkt_out_cnt10 SEC(".maps") = {
- .type = BPF_MAP_TYPE_SK_STORAGE,
- .map_flags = BPF_F_NO_PREALLOC,
-};
+ __uint(type, BPF_MAP_TYPE_SK_STORAGE);
+ __uint(map_flags, BPF_F_NO_PREALLOC);
+ __type(key, int);
+ __type(value, struct bpf_spinlock_cnt);
+} sk_pkt_out_cnt10 SEC(".maps");
static bool is_loopback6(__u32 *a6)
{
diff --git a/tools/testing/selftests/bpf/progs/test_spin_lock.c b/tools/testing/selftests/bpf/progs/test_spin_lock.c
index 0a77ae36d981..a43b999c8da2 100644
--- a/tools/testing/selftests/bpf/progs/test_spin_lock.c
+++ b/tools/testing/selftests/bpf/progs/test_spin_lock.c
@@ -11,14 +11,11 @@ struct hmap_elem {
};
struct {
- __u32 type;
- __u32 max_entries;
- int *key;
- struct hmap_elem *value;
-} hmap SEC(".maps") = {
- .type = BPF_MAP_TYPE_HASH,
- .max_entries = 1,
-};
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, 1);
+ __type(key, int);
+ __type(value, struct hmap_elem);
+} hmap SEC(".maps");
struct cls_elem {
struct bpf_spin_lock lock;
@@ -26,12 +23,10 @@ struct cls_elem {
};
struct {
- __u32 type;
- struct bpf_cgroup_storage_key *key;
- struct cls_elem *value;
-} cls_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_CGROUP_STORAGE,
-};
+ __uint(type, BPF_MAP_TYPE_CGROUP_STORAGE);
+ __type(key, struct bpf_cgroup_storage_key);
+ __type(value, struct cls_elem);
+} cls_map SEC(".maps");
struct bpf_vqueue {
struct bpf_spin_lock lock;
@@ -42,14 +37,11 @@ struct bpf_vqueue {
};
struct {
- __u32 type;
- __u32 max_entries;
- int *key;
- struct bpf_vqueue *value;
-} vqueue SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 1,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, int);
+ __type(value, struct bpf_vqueue);
+} vqueue SEC(".maps");
#define CREDIT_PER_NS(delta, rate) (((delta) * rate) >> 20)
diff --git a/tools/testing/selftests/bpf/progs/test_stacktrace_build_id.c b/tools/testing/selftests/bpf/progs/test_stacktrace_build_id.c
index fcf2280bb60c..bbfc8337b6f0 100644
--- a/tools/testing/selftests/bpf/progs/test_stacktrace_build_id.c
+++ b/tools/testing/selftests/bpf/progs/test_stacktrace_build_id.c
@@ -9,51 +9,36 @@
#endif
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- __u32 *value;
-} control_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 1,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, __u32);
+ __type(value, __u32);
+} control_map SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- __u32 *value;
-} stackid_hmap SEC(".maps") = {
- .type = BPF_MAP_TYPE_HASH,
- .max_entries = 16384,
-};
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, 16384);
+ __type(key, __u32);
+ __type(value, __u32);
+} stackid_hmap SEC(".maps");
typedef struct bpf_stack_build_id stack_trace_t[PERF_MAX_STACK_DEPTH];
struct {
- __u32 type;
- __u32 max_entries;
- __u32 map_flags;
- __u32 key_size;
- __u32 value_size;
-} stackmap SEC(".maps") = {
- .type = BPF_MAP_TYPE_STACK_TRACE,
- .max_entries = 128,
- .map_flags = BPF_F_STACK_BUILD_ID,
- .key_size = sizeof(__u32),
- .value_size = sizeof(stack_trace_t),
-};
+ __uint(type, BPF_MAP_TYPE_STACK_TRACE);
+ __uint(max_entries, 128);
+ __uint(map_flags, BPF_F_STACK_BUILD_ID);
+ __uint(key_size, sizeof(__u32));
+ __uint(value_size, sizeof(stack_trace_t));
+} stackmap SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 128);
+ __type(key, __u32);
/* there seems to be a bug in kernel not handling typedef properly */
struct bpf_stack_build_id (*value)[PERF_MAX_STACK_DEPTH];
-} stack_amap SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 128,
-};
+} stack_amap SEC(".maps");
/* taken from /sys/kernel/debug/tracing/events/random/urandom_read/format */
struct random_urandom_args {
diff --git a/tools/testing/selftests/bpf/progs/test_stacktrace_map.c b/tools/testing/selftests/bpf/progs/test_stacktrace_map.c
index 7ad09adbf648..803c15dc109d 100644
--- a/tools/testing/selftests/bpf/progs/test_stacktrace_map.c
+++ b/tools/testing/selftests/bpf/progs/test_stacktrace_map.c
@@ -9,48 +9,34 @@
#endif
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- __u32 *value;
-} control_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 1,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 1);
+ __type(key, __u32);
+ __type(value, __u32);
+} control_map SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- __u32 *value;
-} stackid_hmap SEC(".maps") = {
- .type = BPF_MAP_TYPE_HASH,
- .max_entries = 16384,
-};
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, 16384);
+ __type(key, __u32);
+ __type(value, __u32);
+} stackid_hmap SEC(".maps");
typedef __u64 stack_trace_t[PERF_MAX_STACK_DEPTH];
struct {
- __u32 type;
- __u32 max_entries;
- __u32 key_size;
- __u32 value_size;
-} stackmap SEC(".maps") = {
- .type = BPF_MAP_TYPE_STACK_TRACE,
- .max_entries = 16384,
- .key_size = sizeof(__u32),
- .value_size = sizeof(stack_trace_t),
-};
+ __uint(type, BPF_MAP_TYPE_STACK_TRACE);
+ __uint(max_entries, 16384);
+ __uint(key_size, sizeof(__u32));
+ __uint(value_size, sizeof(stack_trace_t));
+} stackmap SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 16384);
+ __type(key, __u32);
__u64 (*value)[PERF_MAX_STACK_DEPTH];
-} stack_amap SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 16384,
-};
+} stack_amap SEC(".maps");
/* taken from /sys/kernel/debug/tracing/events/sched/sched_switch/format */
struct sched_switch_args {
diff --git a/tools/testing/selftests/bpf/progs/test_tcp_estats.c b/tools/testing/selftests/bpf/progs/test_tcp_estats.c
index df98f7e32832..c8c595da38d4 100644
--- a/tools/testing/selftests/bpf/progs/test_tcp_estats.c
+++ b/tools/testing/selftests/bpf/progs/test_tcp_estats.c
@@ -149,14 +149,11 @@ struct tcp_estats_basic_event {
};
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct tcp_estats_basic_event *value;
-} ev_record_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_HASH,
- .max_entries = 1024,
-};
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, 1024);
+ __type(key, __u32);
+ __type(value, struct tcp_estats_basic_event);
+} ev_record_map SEC(".maps");
struct dummy_tracepoint_args {
unsigned long long pad;
diff --git a/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c b/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c
index 38e10c9fd996..2e233613d1fc 100644
--- a/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c
+++ b/tools/testing/selftests/bpf/progs/test_tcpbpf_kern.c
@@ -15,24 +15,18 @@
#include "test_tcpbpf.h"
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct tcpbpf_globals *value;
-} global_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 4,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 4);
+ __type(key, __u32);
+ __type(value, struct tcpbpf_globals);
+} global_map SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- int *value;
-} sockopt_results SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 2,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 2);
+ __type(key, __u32);
+ __type(value, int);
+} sockopt_results SEC(".maps");
static inline void update_event_map(int event)
{
diff --git a/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c b/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c
index d073d37d4e27..08346e7765d5 100644
--- a/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c
+++ b/tools/testing/selftests/bpf/progs/test_tcpnotify_kern.c
@@ -15,26 +15,18 @@
#include "test_tcpnotify.h"
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct tcpnotify_globals *value;
-} global_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 4,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 4);
+ __type(key, __u32);
+ __type(value, struct tcpnotify_globals);
+} global_map SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 key_size;
- __u32 value_size;
-} perf_event_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
- .max_entries = 2,
- .key_size = sizeof(int),
- .value_size = sizeof(__u32),
-};
+ __uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY);
+ __uint(max_entries, 2);
+ __uint(key_size, sizeof(int));
+ __uint(value_size, sizeof(__u32));
+} perf_event_map SEC(".maps");
int _version SEC("version") = 1;
diff --git a/tools/testing/selftests/bpf/progs/test_xdp.c b/tools/testing/selftests/bpf/progs/test_xdp.c
index ec3d2c1c8cf9..0941c655b07b 100644
--- a/tools/testing/selftests/bpf/progs/test_xdp.c
+++ b/tools/testing/selftests/bpf/progs/test_xdp.c
@@ -23,24 +23,18 @@
int _version SEC("version") = 1;
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- __u64 *value;
-} rxcnt SEC(".maps") = {
- .type = BPF_MAP_TYPE_PERCPU_ARRAY,
- .max_entries = 256,
-};
+ __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+ __uint(max_entries, 256);
+ __type(key, __u32);
+ __type(value, __u64);
+} rxcnt SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- struct vip *key;
- struct iptnl_info *value;
-} vip2tnl SEC(".maps") = {
- .type = BPF_MAP_TYPE_HASH,
- .max_entries = MAX_IPTNL_ENTRIES,
-};
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, MAX_IPTNL_ENTRIES);
+ __type(key, struct vip);
+ __type(value, struct iptnl_info);
+} vip2tnl SEC(".maps");
static __always_inline void count_tx(__u32 protocol)
{
diff --git a/tools/testing/selftests/bpf/progs/test_xdp_noinline.c b/tools/testing/selftests/bpf/progs/test_xdp_noinline.c
index d2eddb5553d1..dad8a7e33eaa 100644
--- a/tools/testing/selftests/bpf/progs/test_xdp_noinline.c
+++ b/tools/testing/selftests/bpf/progs/test_xdp_noinline.c
@@ -164,66 +164,47 @@ struct lb_stats {
};
struct {
- __u32 type;
- __u32 max_entries;
- struct vip_definition *key;
- struct vip_meta *value;
-} vip_map SEC(".maps") = {
- .type = BPF_MAP_TYPE_HASH,
- .max_entries = 512,
-};
+ __uint(type, BPF_MAP_TYPE_HASH);
+ __uint(max_entries, 512);
+ __type(key, struct vip_definition);
+ __type(value, struct vip_meta);
+} vip_map SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 map_flags;
- struct flow_key *key;
- struct real_pos_lru *value;
-} lru_cache SEC(".maps") = {
- .type = BPF_MAP_TYPE_LRU_HASH,
- .max_entries = 300,
- .map_flags = 1U << 1,
-};
+ __uint(type, BPF_MAP_TYPE_LRU_HASH);
+ __uint(max_entries, 300);
+ __uint(map_flags, 1U << 1);
+ __type(key, struct flow_key);
+ __type(value, struct real_pos_lru);
+} lru_cache SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- __u32 *value;
-} ch_rings SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 12 * 655,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 12 * 655);
+ __type(key, __u32);
+ __type(value, __u32);
+} ch_rings SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct real_definition *value;
-} reals SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 40,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 40);
+ __type(key, __u32);
+ __type(value, struct real_definition);
+} reals SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct lb_stats *value;
-} stats SEC(".maps") = {
- .type = BPF_MAP_TYPE_PERCPU_ARRAY,
- .max_entries = 515,
-};
+ __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
+ __uint(max_entries, 515);
+ __type(key, __u32);
+ __type(value, struct lb_stats);
+} stats SEC(".maps");
struct {
- __u32 type;
- __u32 max_entries;
- __u32 *key;
- struct ctl_value *value;
-} ctl_array SEC(".maps") = {
- .type = BPF_MAP_TYPE_ARRAY,
- .max_entries = 16,
-};
+ __uint(type, BPF_MAP_TYPE_ARRAY);
+ __uint(max_entries, 16);
+ __type(key, __u32);
+ __type(value, struct ctl_value);
+} ctl_array SEC(".maps");
struct eth_hdr {
unsigned char eth_dest[6];
--
2.17.1
^ permalink raw reply related
* Re: [PATCH v5 bpf-next 4/9] libbpf: add kprobe/uprobe attach API
From: Andrii Nakryiko @ 2019-07-04 0:57 UTC (permalink / raw)
To: Daniel Borkmann
Cc: Andrii Nakryiko, bpf, Networking, Alexei Starovoitov, Kernel Team,
Yonghong Song
In-Reply-To: <CAEf4BzaHM5432VS-1wDxKJXr7U-9zkM+A_XsU+1p77YCd8VRgg@mail.gmail.com>
On Wed, Jul 3, 2019 at 9:47 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Wed, Jul 3, 2019 at 5:39 AM Daniel Borkmann <daniel@iogearbox.net> wrote:
> >
> > On 07/02/2019 01:58 AM, Andrii Nakryiko wrote:
> > > Add ability to attach to kernel and user probes and retprobes.
> > > Implementation depends on perf event support for kprobes/uprobes.
> > >
> > > Signed-off-by: Andrii Nakryiko <andriin@fb.com>
> > > Reviewed-by: Stanislav Fomichev <sdf@google.com>
> > > ---
<snip>
> > > +}
> >
> > Hm, this only addresses half the feedback I had in prior version [0]. Patch 2/9
>
> Hi Daniel,
>
> Yes, and I explained why in reply to your original email, please see [1].
>
> I've started with exactly separation you wanted, but it turned out to
> be cumbersome and harder to use user API, while also somewhat more
> complicated to implement. Mostly because in that design bpf_link
> exists in two states: created-but-not-attached and attached. Which
> forces user to do additional clean ups if creation succeeded, but
> attachment failed. It also makes it a bit harder to provide good
> contextual error logging if something goes wrong, because not all
> original parameters are preserved, as some of them might be needed
> only for creation, but not attachment (or we'll have to allocate and
> copy extra stuff just for logging purposes).
>
> On the other hand, having separate generic attach_event method doesn't
> help that much, as there is little common functionality to reuse
> across all kinds of possible bpf_link types.
>
>
> [1] https://lore.kernel.org/bpf/20190621045555.4152743-4-andriin@fb.com/T/#m6cfc141e7b57970bc948134bf671a46972b95134
>
> > with bpf_link with destructor looks good to me, but my feedback from back then was
> > that all the kprobe/uprobe/tracepoint/raw_tracepoint should be split API-wise, so
> > you'll end up with something like the below, that is, 1) a set of functions that
> > only /create/ the bpf_link handle /once/, and 2) a helper that allows /attaching/
> > progs to one or multiple bpf_links. The set of APIs would look like:
> >
> > struct bpf_link *bpf_link__create_kprobe(bool retprobe, const char *func_name);
> > struct bpf_link *bpf_link__create_uprobe(bool retprobe, pid_t pid,
> > const char *binary_path,
> > size_t func_offset);
> > int bpf_program__attach_to_link(struct bpf_link *link, struct bpf_program *prog);
> > int bpf_link__destroy(struct bpf_link *link);
> >
> > This seems much more natural to me. Right now you sort of do both in one single API.
>
> It felt that way for me as well, until I implemented it and used it in
> selftests. And then it felt unnecessarily verbose without giving any
> benefit. I still have a local patchset with that change, I can post it
> as RFC, if you don't trust my judgement. Please let me know.
>
> > Detangling the bpf_program__attach_{uprobe,kprobe}() would also avoid that you have
> > to redo all the perf_event_open_probe() work over and over in order to get the pfd
So re-reading this again, I wonder if you meant that with separate
bpf_link (or rather bpf_hook in that case) creation and attachment
operations, one would be able to create single bpf_hook for same
kprobe and then attach multiple BPF programs to that single pfd
representing that specific probe.
If that's how I should have read it, I agree that it probably would be
possible for some types of hooks, but not for every type of hook. But
furthermore, how often in practice same application attaches many
different BPF programs to the same hook? And it's also hard to imagine
that hook creation (i.e., creating such FD for BPF hook), would ever
be a bottleneck.
So I still think it's not a strong reason to go with API that's harder
to use for typical use cases just because of hypothetical benefits in
some extreme cases.
>
> What do you mean by "redo all the perf_event_open_probe work"? In
> terms of code, I just reuse the same function, so there is no
> duplicate code. And in either design you'll have to open that
> perf_event, so that work will have to be done one way or another.
>
> > context where you can later attach something to. Given bpf_program__attach_to_link()
> > API, you also wouldn't need to expose the bpf_program__attach_perf_event() from
>
> I'd expose attach_perf_event either way, it's high-level API I want to
> provide, we have use cases where user is creating some specific
> non-kprobe/non-tracepoint perf events and wants to attach to it. E.g.,
> HW counter overflow events for CPU profilers. So that API is not some
> kind of leaked abstraction, it's something I want to have anyway.
>
>
> > patch 3/9. Thoughts?
>
> I believe this hybrid approach provides better usability without
> compromising anything. The only theoretical benefit of complete
> separation of bpf_link creation and attachment is that user code would
> be able to separate those two steps code organization-wise. But it's
> easily doable through custom application code (just encapsulate all
> the parameters and type of attachment and pass it around until you
> actually need to attach), but I don't think it's necessary in practice
> (so far I never needed anything like that).
>
> Hope I convinced you that while elegant, it's not that practical. Also
> hybrid approach isn't inelegant either and doesn't produce code
> duplication (it actually eliminates some unnecessary allocations,
> e.g., for storing tp_name for raw_tracepoint attach) :)
>
> >
> > [0] https://lore.kernel.org/bpf/a7780057-1d70-9ace-960b-ff65867dc277@iogearbox.net/
> >
> > > enum bpf_perf_event_ret
> > > bpf_perf_event_read_simple(void *mmap_mem, size_t mmap_size, size_t page_size,
> > > void **copy_mem, size_t *copy_size,
> > > diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
> > > index 1bf66c4a9330..bd767cc11967 100644
> > > --- a/tools/lib/bpf/libbpf.h
> > > +++ b/tools/lib/bpf/libbpf.h
> > > @@ -171,6 +171,13 @@ LIBBPF_API int bpf_link__destroy(struct bpf_link *link);
> > >
> > > LIBBPF_API struct bpf_link *
> > > bpf_program__attach_perf_event(struct bpf_program *prog, int pfd);
> > > +LIBBPF_API struct bpf_link *
> > > +bpf_program__attach_kprobe(struct bpf_program *prog, bool retprobe,
> > > + const char *func_name);
> > > +LIBBPF_API struct bpf_link *
> > > +bpf_program__attach_uprobe(struct bpf_program *prog, bool retprobe,
> > > + pid_t pid, const char *binary_path,
> > > + size_t func_offset);
> > >
> > > struct bpf_insn;
> > >
> > > diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
> > > index 756f5aa802e9..57a40fb60718 100644
> > > --- a/tools/lib/bpf/libbpf.map
> > > +++ b/tools/lib/bpf/libbpf.map
> > > @@ -169,7 +169,9 @@ LIBBPF_0.0.4 {
> > > global:
> > > bpf_link__destroy;
> > > bpf_object__load_xattr;
> > > + bpf_program__attach_kprobe;
> > > bpf_program__attach_perf_event;
> > > + bpf_program__attach_uprobe;
> > > btf_dump__dump_type;
> > > btf_dump__free;
> > > btf_dump__new;
> > >
> >
^ permalink raw reply
* RE: [PATCH v1 net-next] net: stmmac: enable clause 45 mdio support
From: Voon, Weifeng @ 2019-07-04 1:33 UTC (permalink / raw)
To: Andrew Lunn
Cc: David S. Miller, Maxime Coquelin, netdev@vger.kernel.org,
linux-kernel@vger.kernel.org, Jose Abreu, Giuseppe Cavallaro,
Florian Fainelli, Alexandre Torgue, biao huang, Ong, Boon Leong,
Kweh, Hock Leong
In-Reply-To: <20190703140520.GA18473@lunn.ch>
> > @@ -155,22 +171,26 @@ static int stmmac_mdio_read(struct mii_bus *bus,
> int phyaddr, int phyreg)
> > struct stmmac_priv *priv = netdev_priv(ndev);
> > unsigned int mii_address = priv->hw->mii.addr;
> > unsigned int mii_data = priv->hw->mii.data;
> > - u32 v;
> > - int data;
> > u32 value = MII_BUSY;
> > + int data = 0;
> > + u32 v;
> >
> > value |= (phyaddr << priv->hw->mii.addr_shift)
> > & priv->hw->mii.addr_mask;
> > value |= (phyreg << priv->hw->mii.reg_shift) & priv->hw-
> >mii.reg_mask;
> > value |= (priv->clk_csr << priv->hw->mii.clk_csr_shift)
> > & priv->hw->mii.clk_csr_mask;
> > - if (priv->plat->has_gmac4)
> > + if (priv->plat->has_gmac4) {
> > value |= MII_GMAC4_READ;
> > + if (phyreg & MII_ADDR_C45)
> > + stmmac_mdio_c45_setup(priv, phyreg, &value, &data);
> > + }
> >
> > if (readl_poll_timeout(priv->ioaddr + mii_address, v, !(v &
> MII_BUSY),
> > 100, 10000))
> > return -EBUSY;
> >
> > + writel(data, priv->ioaddr + mii_data);
>
> That looks odd. Could you explain why it is needed.
>
> Thanks
> Andrew
Hi Andrew,
This mdio c45 support needed to access DWC xPCS which is a Clause-45
MDIO Manageable Device (MMD). This is discuss in:
https://patchwork.ozlabs.org/patch/1109776/
https://patchwork.ozlabs.org/patch/1109777/
Since the patch is still WIP to make the xPCS as PHY driver for SGMII
to RGMII converter. So i decided to upstream this patch first.
Biao Huang also needed this patch and tested pass with this patch on
their platform.
https://patchwork.ozlabs.org/patch/1103861/
Biao Huang's patch on MDIO-c45 access
https://patchwork.ozlabs.org/patch/1092436/
Regards,
Weifeng
^ permalink raw reply
* Re: [RFC v2] vhost: introduce mdev based hardware vhost backend
From: Tiwei Bie @ 2019-07-04 1:36 UTC (permalink / raw)
To: Alex Williamson
Cc: mst, jasowang, maxime.coquelin, linux-kernel, kvm, virtualization,
netdev, dan.daly, cunming.liang, zhihong.wang
In-Reply-To: <20190703123157.2452bf95@x1.home>
On Wed, Jul 03, 2019 at 12:31:57PM -0600, Alex Williamson wrote:
> On Wed, 3 Jul 2019 17:13:39 +0800
> Tiwei Bie <tiwei.bie@intel.com> wrote:
> > diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h
> > index 8f10748dac79..6c5718ab7eeb 100644
> > --- a/include/uapi/linux/vfio.h
> > +++ b/include/uapi/linux/vfio.h
> > @@ -201,6 +201,7 @@ struct vfio_device_info {
> > #define VFIO_DEVICE_FLAGS_AMBA (1 << 3) /* vfio-amba device */
> > #define VFIO_DEVICE_FLAGS_CCW (1 << 4) /* vfio-ccw device */
> > #define VFIO_DEVICE_FLAGS_AP (1 << 5) /* vfio-ap device */
> > +#define VFIO_DEVICE_FLAGS_VHOST (1 << 6) /* vfio-vhost device */
> > __u32 num_regions; /* Max region index + 1 */
> > __u32 num_irqs; /* Max IRQ index + 1 */
> > };
> > @@ -217,6 +218,7 @@ struct vfio_device_info {
> > #define VFIO_DEVICE_API_AMBA_STRING "vfio-amba"
> > #define VFIO_DEVICE_API_CCW_STRING "vfio-ccw"
> > #define VFIO_DEVICE_API_AP_STRING "vfio-ap"
> > +#define VFIO_DEVICE_API_VHOST_STRING "vfio-vhost"
> >
> > /**
> > * VFIO_DEVICE_GET_REGION_INFO - _IOWR(VFIO_TYPE, VFIO_BASE + 8,
> > @@ -573,6 +575,23 @@ enum {
> > VFIO_CCW_NUM_IRQS
> > };
> >
> > +/*
> > + * The vfio-vhost bus driver makes use of the following fixed region and
> > + * IRQ index mapping. Unimplemented regions return a size of zero.
> > + * Unimplemented IRQ types return a count of zero.
> > + */
> > +
> > +enum {
> > + VFIO_VHOST_CONFIG_REGION_INDEX,
> > + VFIO_VHOST_NOTIFY_REGION_INDEX,
> > + VFIO_VHOST_NUM_REGIONS
> > +};
> > +
> > +enum {
> > + VFIO_VHOST_VQ_IRQ_INDEX,
> > + VFIO_VHOST_NUM_IRQS
> > +};
> > +
>
> Note that the vfio API has evolved a bit since vfio-pci started this
> way, with fixed indexes for pre-defined region types. We now support
> device specific regions which can be identified by a capability within
> the REGION_INFO ioctl return data. This allows a bit more flexibility,
> at the cost of complexity, but the infrastructure already exists in
> kernel and QEMU to make it relatively easy. I think we'll have the
> same support for interrupts soon too. If you continue to pursue the
> vfio-vhost direction you might want to consider these before committing
> to fixed indexes. Thanks,
Thanks for the details! Will give it a try!
Thanks,
Tiwei
^ permalink raw reply
* linux-next: manual merge of the net-next tree with the net tree
From: Stephen Rothwell @ 2019-07-04 2:07 UTC (permalink / raw)
To: David Miller, Networking
Cc: Linux Next Mailing List, Linux Kernel Mailing List, Po-Hsu Lin,
Willem de Bruijn, Mahesh Bandewar
[-- Attachment #1: Type: text/plain, Size: 1226 bytes --]
Hi all,
Today's linux-next merge of the net-next tree got a conflict in:
tools/testing/selftests/net/config
between commit:
ff95bf28c234 ("selftests/net: skip psock_tpacket test if KALLSYMS was not enabled")
from the net tree and commit:
af5136f95045 ("selftests/net: SO_TXTIME with ETF and FQ")
509e56b37cc3 ("blackhole_dev: add a selftest")
from the net-next tree.
I fixed it up (see below) and can carry the fix as necessary. This
is now fixed as far as linux-next is concerned, but any non trivial
conflicts should be mentioned to your upstream maintainer when your tree
is submitted for merging. You may also want to consider cooperating
with the maintainer of the conflicting tree to minimise any particularly
complex conflicts.
--
Cheers,
Stephen Rothwell
diff --cc tools/testing/selftests/net/config
index 3dea2cba2325,e4b878d95ba0..000000000000
--- a/tools/testing/selftests/net/config
+++ b/tools/testing/selftests/net/config
@@@ -25,4 -25,6 +25,7 @@@ CONFIG_NF_TABLES_IPV6=
CONFIG_NF_TABLES_IPV4=y
CONFIG_NFT_CHAIN_NAT_IPV6=m
CONFIG_NFT_CHAIN_NAT_IPV4=m
+CONFIG_KALLSYMS=y
+ CONFIG_NET_SCH_FQ=m
+ CONFIG_NET_SCH_ETF=m
+ CONFIG_TEST_BLACKHOLE_DEV=m
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply
* [net-next 0/3][pull request] Intel Wired LAN ver Updates 2019-07-03
From: Jeff Kirsher @ 2019-07-04 2:12 UTC (permalink / raw)
To: davem, dledford, jgg
Cc: Jeff Kirsher, netdev, linux-rdma, nhorman, sassmann,
mustafa.ismail, shiraz.saleem, david.m.ertman
This series contains updates to i40e an ice drivers only and is required
for a series of changes being submitted to the RDMA maintainer/tree.
Vice Versa, the Intel RDMA driver patches could not be applied to
net-next due to dependencies to the changes currently in the for-next
branch of the rdma git tree. The Intel RDMA driver patches are
available in the following git repository:
git://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/rdma for-next
(reference only)
Dave initializes the ice driver to support RDMA by creating and
registering a platform device for the RDMA driver to register to on a
virtual bus by utilizing the platform bus to provide this access.
Followed up by operations for the peer device/driver to communicate with
each other to request resources and manage event notification.
Shiraz does the similar work in the i40e driver, while retaining the
old global register/unregister calls exported for i40iw until that
driver is removed from the kernel.
The following are changes since commit a51df9f8da43e8bf9e508143630849b7d696e053:
gve: fix -ENOMEM null check on a page allocation
and are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/next-queue 100GbE
Shiraz Saleem (1):
i40e: Register RDMA client devices to the virtual platform bus
Tony Nguyen (2):
ice: Initialize and register platform device to provide RDMA
ice: Implement peer communications
drivers/net/ethernet/intel/i40e/i40e_client.c | 116 +-
drivers/net/ethernet/intel/i40e/i40e_client.h | 8 +
drivers/net/ethernet/intel/ice/Makefile | 1 +
drivers/net/ethernet/intel/ice/ice.h | 15 +
.../net/ethernet/intel/ice/ice_adminq_cmd.h | 33 +
drivers/net/ethernet/intel/ice/ice_common.c | 194 +++
drivers/net/ethernet/intel/ice/ice_common.h | 9 +
drivers/net/ethernet/intel/ice/ice_dcb_lib.c | 64 +
drivers/net/ethernet/intel/ice/ice_dcb_lib.h | 2 +
.../net/ethernet/intel/ice/ice_hw_autogen.h | 1 +
drivers/net/ethernet/intel/ice/ice_idc.c | 1354 +++++++++++++++++
drivers/net/ethernet/intel/ice/ice_idc.h | 377 +++++
drivers/net/ethernet/intel/ice/ice_idc_int.h | 82 +
drivers/net/ethernet/intel/ice/ice_lib.c | 35 +-
drivers/net/ethernet/intel/ice/ice_lib.h | 2 +
drivers/net/ethernet/intel/ice/ice_main.c | 94 +-
drivers/net/ethernet/intel/ice/ice_sched.c | 69 +-
drivers/net/ethernet/intel/ice/ice_switch.c | 27 +
drivers/net/ethernet/intel/ice/ice_switch.h | 4 +
drivers/net/ethernet/intel/ice/ice_type.h | 4 +
.../net/ethernet/intel/ice/ice_virtchnl_pf.c | 25 -
21 files changed, 2462 insertions(+), 54 deletions(-)
create mode 100644 drivers/net/ethernet/intel/ice/ice_idc.c
create mode 100644 drivers/net/ethernet/intel/ice/ice_idc.h
create mode 100644 drivers/net/ethernet/intel/ice/ice_idc_int.h
--
2.21.0
^ permalink raw reply
* [net-next 1/3] ice: Initialize and register platform device to provide RDMA
From: Jeff Kirsher @ 2019-07-04 2:12 UTC (permalink / raw)
To: davem, dledford, jgg
Cc: Tony Nguyen, netdev, linux-rdma, nhorman, sassmann, poswald,
mustafa.ismail, shiraz.saleem, Dave Ertman, Andrew Bowers,
Jeff Kirsher
In-Reply-To: <20190704021252.15534-1-jeffrey.t.kirsher@intel.com>
From: Tony Nguyen <anthony.l.nguyen@intel.com>
The RDMA block does not advertise on the PCI bus or any other bus.
Thus the ice driver needs to provide access to the RDMA hardware block
via a virtual bus; utilize the platform bus to provide this access.
This patch initializes the driver to support RDMA as well as creates
and registers a platform device for the RDMA driver to register to. At
this point the driver is fully initialized to register a platform
driver, however, can not yet register as the ops have not been
implemented.
We refer to the interaction of this platform device as Inter-Driver
Communication (IDC); where the platform device is referred to as the peer
device and the platform driver is referred to as the peer driver.
Signed-off-by: Dave Ertman <david.m.ertman@intel.com>
Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ice/Makefile | 1 +
drivers/net/ethernet/intel/ice/ice.h | 13 +
.../net/ethernet/intel/ice/ice_adminq_cmd.h | 1 +
drivers/net/ethernet/intel/ice/ice_common.c | 5 +
drivers/net/ethernet/intel/ice/ice_dcb_lib.c | 31 ++
drivers/net/ethernet/intel/ice/ice_dcb_lib.h | 2 +
.../net/ethernet/intel/ice/ice_hw_autogen.h | 1 +
drivers/net/ethernet/intel/ice/ice_idc.c | 427 ++++++++++++++++++
drivers/net/ethernet/intel/ice/ice_idc.h | 360 +++++++++++++++
drivers/net/ethernet/intel/ice/ice_idc_int.h | 67 +++
drivers/net/ethernet/intel/ice/ice_main.c | 42 ++
drivers/net/ethernet/intel/ice/ice_type.h | 1 +
12 files changed, 951 insertions(+)
create mode 100644 drivers/net/ethernet/intel/ice/ice_idc.c
create mode 100644 drivers/net/ethernet/intel/ice/ice_idc.h
create mode 100644 drivers/net/ethernet/intel/ice/ice_idc_int.h
diff --git a/drivers/net/ethernet/intel/ice/Makefile b/drivers/net/ethernet/intel/ice/Makefile
index 2d140ba83781..1500f7724b6f 100644
--- a/drivers/net/ethernet/intel/ice/Makefile
+++ b/drivers/net/ethernet/intel/ice/Makefile
@@ -15,6 +15,7 @@ ice-y := ice_main.o \
ice_sched.o \
ice_lib.o \
ice_txrx.o \
+ ice_idc.o \
ice_ethtool.o
ice-$(CONFIG_PCI_IOV) += ice_virtchnl_pf.o ice_sriov.o
ice-$(CONFIG_DCB) += ice_dcb.o ice_dcb_lib.o
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 9ee6b55553c0..6ee2774699f0 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -38,6 +38,7 @@
#include "ice_switch.h"
#include "ice_common.h"
#include "ice_sched.h"
+#include "ice_idc_int.h"
#include "ice_virtchnl_pf.h"
#include "ice_sriov.h"
@@ -84,6 +85,7 @@ extern const char ice_drv_ver[];
#define ICE_MAX_SMALL_RSS_QS 8
#define ICE_RES_VALID_BIT 0x8000
#define ICE_RES_MISC_VEC_ID (ICE_RES_VALID_BIT - 1)
+#define ICE_RES_RDMA_VEC_ID (ICE_RES_MISC_VEC_ID - 1)
#define ICE_INVAL_Q_INDEX 0xffff
#define ICE_INVAL_VFID 256
#define ICE_MAX_VF_COUNT 256
@@ -331,6 +333,7 @@ struct ice_q_vector {
enum ice_pf_flags {
ICE_FLAG_MSIX_ENA,
ICE_FLAG_FLTR_SYNC,
+ ICE_FLAG_IWARP_ENA,
ICE_FLAG_RSS_ENA,
ICE_FLAG_SRIOV_ENA,
ICE_FLAG_SRIOV_CAPABLE,
@@ -373,6 +376,9 @@ struct ice_pf {
struct mutex avail_q_mutex; /* protects access to avail_[rx|tx]qs */
struct mutex sw_mutex; /* lock for protecting VSI alloc flow */
u32 msg_enable;
+ /* Total number of MSIX vectors reserved for base driver */
+ u32 num_rdma_msix;
+ u32 rdma_base_vector;
u32 hw_csum_rx_error;
u32 oicr_idx; /* Other interrupt cause MSIX vector index */
u32 num_avail_sw_msix; /* remaining MSIX SW vectors left unclaimed */
@@ -399,6 +405,7 @@ struct ice_pf {
unsigned long tx_timeout_last_recovery;
u32 tx_timeout_recovery_level;
char int_name[ICE_INT_NAME_STR_LEN];
+ struct ice_peer_dev_int **peers;
u32 sw_int_count;
};
@@ -406,6 +413,8 @@ struct ice_netdev_priv {
struct ice_vsi *vsi;
};
+extern struct ida ice_peer_index_ida;
+
/**
* ice_irq_dynamic_ena - Enable default interrupt generation settings
* @hw: pointer to HW struct
@@ -463,6 +472,10 @@ int ice_set_rss(struct ice_vsi *vsi, u8 *seed, u8 *lut, u16 lut_size);
int ice_get_rss(struct ice_vsi *vsi, u8 *seed, u8 *lut, u16 lut_size);
void ice_fill_rss_lut(u8 *lut, u16 rss_table_size, u16 rss_size);
void ice_print_link_msg(struct ice_vsi *vsi, bool isup);
+int ice_init_peer_devices(struct ice_pf *pf);
+int
+ice_for_each_peer(struct ice_pf *pf, void *data,
+ int (*fn)(struct ice_peer_dev_int *, void *));
#ifdef CONFIG_DCB
int ice_pf_ena_all_vsi(struct ice_pf *pf, bool locked);
void ice_pf_dis_all_vsi(struct ice_pf *pf, bool locked);
diff --git a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
index 765e3c2ed045..1e1bd5f0c2a2 100644
--- a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
+++ b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
@@ -96,6 +96,7 @@ struct ice_aqc_list_caps_elem {
#define ICE_AQC_CAPS_TXQS 0x0042
#define ICE_AQC_CAPS_MSIX 0x0043
#define ICE_AQC_CAPS_MAX_MTU 0x0047
+#define ICE_AQC_CAPS_IWARP 0x0051
u8 major_ver;
u8 minor_ver;
diff --git a/drivers/net/ethernet/intel/ice/ice_common.c b/drivers/net/ethernet/intel/ice/ice_common.c
index 2e0731c1e1a3..2788b2ede440 100644
--- a/drivers/net/ethernet/intel/ice/ice_common.c
+++ b/drivers/net/ethernet/intel/ice/ice_common.c
@@ -1613,6 +1613,11 @@ ice_parse_caps(struct ice_hw *hw, void *buf, u32 cap_count,
"%s: MSIX first vector index = %d\n", prefix,
caps->msix_vector_first_id);
break;
+ case ICE_AQC_CAPS_IWARP:
+ caps->iwarp = (number == 1);
+ ice_debug(hw, ICE_DBG_INIT,
+ "%s: iWARP = %d\n", prefix, caps->iwarp);
+ break;
case ICE_AQC_CAPS_MAX_MTU:
caps->max_mtu = number;
ice_debug(hw, ICE_DBG_INIT, "%s: max MTU = %d\n",
diff --git a/drivers/net/ethernet/intel/ice/ice_dcb_lib.c b/drivers/net/ethernet/intel/ice/ice_dcb_lib.c
index fe88b127ca42..c6466e16eef1 100644
--- a/drivers/net/ethernet/intel/ice/ice_dcb_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_dcb_lib.c
@@ -501,6 +501,37 @@ ice_tx_prepare_vlan_flags_dcb(struct ice_ring *tx_ring,
return 0;
}
+/**
+ * ice_setup_dcb_qos_info - Setup DCB QoS information
+ * @pf: ptr to ice_pf
+ * @qos_info: QoS param instance
+ */
+void ice_setup_dcb_qos_info(struct ice_pf *pf, struct ice_qos_params *qos_info)
+{
+ struct ice_dcbx_cfg *dcbx_cfg;
+ u32 up2tc;
+ int i;
+
+ dcbx_cfg = &pf->hw.port_info->local_dcbx_cfg;
+ up2tc = rd32(&pf->hw, PRTDCB_TUP2TC);
+ qos_info->num_apps = dcbx_cfg->numapps;
+
+ qos_info->num_tc = ice_dcb_get_num_tc(dcbx_cfg);
+
+ for (i = 0; i < ICE_IDC_MAX_USER_PRIORITY; i++)
+ qos_info->up2tc[i] = (up2tc >> (i * 3)) & 0x7;
+
+ for (i = 0; i < IEEE_8021QAZ_MAX_TCS; i++)
+ qos_info->tc_info[i].rel_bw =
+ dcbx_cfg->etscfg.tcbwtable[i];
+
+ for (i = 0; i < qos_info->num_apps; i++) {
+ qos_info->apps[i].priority = dcbx_cfg->app[i].priority;
+ qos_info->apps[i].prot_id = dcbx_cfg->app[i].prot_id;
+ qos_info->apps[i].selector = dcbx_cfg->app[i].selector;
+ }
+}
+
/**
* ice_dcb_need_recfg - Check if DCB needs reconfig
* @pf: board private structure
diff --git a/drivers/net/ethernet/intel/ice/ice_dcb_lib.h b/drivers/net/ethernet/intel/ice/ice_dcb_lib.h
index 819081053ff5..bce3899afb3a 100644
--- a/drivers/net/ethernet/intel/ice/ice_dcb_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_dcb_lib.h
@@ -19,6 +19,7 @@ void ice_update_dcb_stats(struct ice_pf *pf);
int
ice_tx_prepare_vlan_flags_dcb(struct ice_ring *tx_ring,
struct ice_tx_buf *first);
+void ice_setup_dcb_qos_info(struct ice_pf *pf, struct ice_qos_params *qos_info);
void
ice_dcb_process_lldp_set_mib_change(struct ice_pf *pf,
struct ice_rq_event_info *event);
@@ -56,6 +57,7 @@ ice_tx_prepare_vlan_flags_dcb(struct ice_ring __always_unused *tx_ring,
#define ice_update_dcb_stats(pf) do {} while (0)
#define ice_vsi_cfg_dcb_rings(vsi) do {} while (0)
+#define ice_setup_dcb_qos_info(pf, qos_info) do {} while (0)
#define ice_dcb_process_lldp_set_mib_change(pf, event) do {} while (0)
#define ice_set_cgd_num(tlan_ctx, ring) do {} while (0)
#endif /* CONFIG_DCB */
diff --git a/drivers/net/ethernet/intel/ice/ice_hw_autogen.h b/drivers/net/ethernet/intel/ice/ice_hw_autogen.h
index 6c5ce05742b1..0fa393f909dd 100644
--- a/drivers/net/ethernet/intel/ice/ice_hw_autogen.h
+++ b/drivers/net/ethernet/intel/ice/ice_hw_autogen.h
@@ -55,6 +55,7 @@
#define PRTDCB_GENS 0x00083020
#define PRTDCB_GENS_DCBX_STATUS_S 0
#define PRTDCB_GENS_DCBX_STATUS_M ICE_M(0x7, 0)
+#define PRTDCB_TUP2TC 0x001D26C0
#define GLFLXP_RXDID_FLAGS(_i, _j) (0x0045D000 + ((_i) * 4 + (_j) * 256))
#define GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_S 0
#define GLFLXP_RXDID_FLAGS_FLEXIFLAG_4N_M ICE_M(0x3F, 0)
diff --git a/drivers/net/ethernet/intel/ice/ice_idc.c b/drivers/net/ethernet/intel/ice/ice_idc.c
new file mode 100644
index 000000000000..0bf66e9fa159
--- /dev/null
+++ b/drivers/net/ethernet/intel/ice/ice_idc.c
@@ -0,0 +1,427 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2019, Intel Corporation. */
+
+/* Inter-Driver Communication */
+#include "ice.h"
+#include "ice_lib.h"
+#include "ice_dcb_lib.h"
+
+DEFINE_IDA(ice_peer_index_ida);
+
+static const struct peer_dev_id peer_dev_ids[] = ASSIGN_PEER_INFO;
+
+/**
+ * ice_peer_state_change - manage state machine for peer
+ * @peer_dev: pointer to peer's configuration
+ * @new_state: the state requested to transition into
+ * @locked: boolean to determine if call made with mutex held
+ *
+ * This function handles all state transitions for peer devices.
+ * The state machine is as follows:
+ *
+ * +<-----------------------+<-----------------------------+
+ * |<-------+<----------+ +
+ * \/ + + +
+ * INIT --------------> PROBED --> OPENING CLOSED --> REMOVED
+ * + +
+ * OPENED --> CLOSING
+ * + +
+ * PREP_RST +
+ * + +
+ * PREPPED +
+ * +---------->+
+ */
+static void
+ice_peer_state_change(struct ice_peer_dev_int *peer_dev, long new_state,
+ bool locked)
+{
+ if (!locked)
+ mutex_lock(&peer_dev->peer_dev_state_mutex);
+
+ switch (new_state) {
+ case ICE_PEER_DEV_STATE_INIT:
+ if (test_and_clear_bit(ICE_PEER_DEV_STATE_REMOVED,
+ peer_dev->state)) {
+ set_bit(ICE_PEER_DEV_STATE_INIT, peer_dev->state);
+ dev_info(&peer_dev->peer_dev.platform_dev.dev,
+ "state transition from _REMOVED to _INIT\n");
+ } else {
+ set_bit(ICE_PEER_DEV_STATE_INIT, peer_dev->state);
+ dev_info(&peer_dev->peer_dev.platform_dev.dev,
+ "state set to _INIT\n");
+ }
+ break;
+ case ICE_PEER_DEV_STATE_PROBED:
+ if (test_and_clear_bit(ICE_PEER_DEV_STATE_INIT,
+ peer_dev->state)) {
+ set_bit(ICE_PEER_DEV_STATE_PROBED, peer_dev->state);
+ dev_info(&peer_dev->peer_dev.platform_dev.dev,
+ "state transition from _INIT to _PROBED\n");
+ } else if (test_and_clear_bit(ICE_PEER_DEV_STATE_REMOVED,
+ peer_dev->state)) {
+ set_bit(ICE_PEER_DEV_STATE_PROBED, peer_dev->state);
+ dev_info(&peer_dev->peer_dev.platform_dev.dev,
+ "state transition from _REMOVED to _PROBED\n");
+ } else if (test_and_clear_bit(ICE_PEER_DEV_STATE_OPENING,
+ peer_dev->state)) {
+ set_bit(ICE_PEER_DEV_STATE_PROBED, peer_dev->state);
+ dev_info(&peer_dev->peer_dev.platform_dev.dev,
+ "state transition from _OPENING to _PROBED\n");
+ }
+ break;
+ case ICE_PEER_DEV_STATE_OPENING:
+ if (test_and_clear_bit(ICE_PEER_DEV_STATE_PROBED,
+ peer_dev->state)) {
+ set_bit(ICE_PEER_DEV_STATE_OPENING, peer_dev->state);
+ dev_info(&peer_dev->peer_dev.platform_dev.dev,
+ "state transition from _PROBED to _OPENING\n");
+ } else if (test_and_clear_bit(ICE_PEER_DEV_STATE_CLOSED,
+ peer_dev->state)) {
+ set_bit(ICE_PEER_DEV_STATE_OPENING, peer_dev->state);
+ dev_info(&peer_dev->peer_dev.platform_dev.dev,
+ "state transition from _CLOSED to _OPENING\n");
+ }
+ break;
+ case ICE_PEER_DEV_STATE_OPENED:
+ if (test_and_clear_bit(ICE_PEER_DEV_STATE_OPENING,
+ peer_dev->state)) {
+ set_bit(ICE_PEER_DEV_STATE_OPENED, peer_dev->state);
+ dev_info(&peer_dev->peer_dev.platform_dev.dev,
+ "state transition from _OPENING to _OPENED\n");
+ }
+ break;
+ case ICE_PEER_DEV_STATE_PREP_RST:
+ if (test_and_clear_bit(ICE_PEER_DEV_STATE_OPENED,
+ peer_dev->state)) {
+ set_bit(ICE_PEER_DEV_STATE_PREP_RST, peer_dev->state);
+ dev_info(&peer_dev->peer_dev.platform_dev.dev,
+ "state transition from _OPENED to _PREP_RST\n");
+ }
+ break;
+ case ICE_PEER_DEV_STATE_PREPPED:
+ if (test_and_clear_bit(ICE_PEER_DEV_STATE_PREP_RST,
+ peer_dev->state)) {
+ set_bit(ICE_PEER_DEV_STATE_PREPPED, peer_dev->state);
+ dev_info(&peer_dev->peer_dev.platform_dev.dev,
+ "state transition _PREP_RST to _PREPPED\n");
+ }
+ break;
+ case ICE_PEER_DEV_STATE_CLOSING:
+ if (test_and_clear_bit(ICE_PEER_DEV_STATE_OPENED,
+ peer_dev->state)) {
+ set_bit(ICE_PEER_DEV_STATE_CLOSING, peer_dev->state);
+ dev_info(&peer_dev->peer_dev.platform_dev.dev,
+ "state transition from _OPENED to _CLOSING\n");
+ }
+ if (test_and_clear_bit(ICE_PEER_DEV_STATE_PREPPED,
+ peer_dev->state)) {
+ set_bit(ICE_PEER_DEV_STATE_CLOSING, peer_dev->state);
+ dev_info(&peer_dev->peer_dev.platform_dev.dev,
+ "state transition _PREPPED to _CLOSING\n");
+ }
+ /* NOTE - up to peer to handle this situation correctly */
+ if (test_and_clear_bit(ICE_PEER_DEV_STATE_PREP_RST,
+ peer_dev->state)) {
+ set_bit(ICE_PEER_DEV_STATE_CLOSING, peer_dev->state);
+ dev_warn(&peer_dev->peer_dev.platform_dev.dev,
+ "WARN: Peer state PREP_RST to _CLOSING\n");
+ }
+ break;
+ case ICE_PEER_DEV_STATE_CLOSED:
+ if (test_and_clear_bit(ICE_PEER_DEV_STATE_CLOSING,
+ peer_dev->state)) {
+ set_bit(ICE_PEER_DEV_STATE_CLOSED, peer_dev->state);
+ dev_info(&peer_dev->peer_dev.platform_dev.dev,
+ "state transition from _CLOSING to _CLOSED\n");
+ }
+ break;
+ case ICE_PEER_DEV_STATE_REMOVED:
+ if (test_and_clear_bit(ICE_PEER_DEV_STATE_OPENED,
+ peer_dev->state) ||
+ test_and_clear_bit(ICE_PEER_DEV_STATE_CLOSED,
+ peer_dev->state)) {
+ set_bit(ICE_PEER_DEV_STATE_REMOVED, peer_dev->state);
+ dev_info(&peer_dev->peer_dev.platform_dev.dev,
+ "state from _OPENED/_CLOSED to _REMOVED\n");
+ /* Clear registration for events when peer removed */
+ bitmap_zero(peer_dev->events, ICE_PEER_DEV_STATE_NBITS);
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!locked)
+ mutex_unlock(&peer_dev->peer_dev_state_mutex);
+}
+
+/**
+ * ice_for_each_peer - iterate across and call function for each peer dev
+ * @pf: pointer to private board struct
+ * @data: data to pass to function on each call
+ * @fn: pointer to function to call for each peer
+ *
+ * This function is to be used similarly to ice_for_each_peer
+ */
+int
+ice_for_each_peer(struct ice_pf *pf, void *data,
+ int (*fn)(struct ice_peer_dev_int *, void *))
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(peer_dev_ids); i++) {
+ struct ice_peer_dev_int *peer_dev_int;
+ int ret;
+
+ peer_dev_int = pf->peers[i];
+ if (peer_dev_int) {
+ ret = fn(peer_dev_int, data);
+ if (ret)
+ return ret;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * ice_unreg_peer_device - unregister specified device
+ * @peer_dev_int: ptr to peer device internal
+ * @data: ptr to opaque data
+ *
+ * This function invokes device unregistration, removes ID associated with
+ * the specified device.
+ */
+int
+ice_unreg_peer_device(struct ice_peer_dev_int *peer_dev_int,
+ void __always_unused *data)
+{
+ struct ice_pf *pf;
+
+ if (!peer_dev_int)
+ return 0;
+
+ platform_device_unregister(&peer_dev_int->peer_dev.platform_dev);
+
+ pf = pci_get_drvdata(peer_dev_int->peer_dev.pdev);
+ if (!pf)
+ return 0;
+
+ if (peer_dev_int->ice_peer_wq) {
+ if (peer_dev_int->peer_prep_task.func)
+ cancel_work_sync(&peer_dev_int->peer_prep_task);
+
+ if (peer_dev_int->peer_close_task.func)
+ cancel_work_sync(&peer_dev_int->peer_close_task);
+ destroy_workqueue(peer_dev_int->ice_peer_wq);
+ }
+
+ /* Cleanup the allocated ID for this peer device */
+ ida_simple_remove(&ice_peer_index_ida, peer_dev_int->peer_dev.index);
+
+ devm_kfree(&pf->pdev->dev, peer_dev_int);
+
+ return 0;
+}
+
+/**
+ * ice_unroll_peer - destroy peers and peer_wq in case of error
+ * @peer_dev_int: ptr to peer device internal struct
+ * @data: ptr to opaque data
+ *
+ * This function releases resources in the event of a failure in creating
+ * peer devices or their individual work_queues. Meant to be called from
+ * a ice_for_each_peer invocation
+ */
+int
+ice_unroll_peer(struct ice_peer_dev_int *peer_dev_int,
+ void __always_unused *data)
+{
+ struct ice_pf *pf;
+
+ pf = pci_get_drvdata(peer_dev_int->peer_dev.pdev);
+
+ if (peer_dev_int->ice_peer_wq)
+ destroy_workqueue(peer_dev_int->ice_peer_wq);
+ devm_kfree(&pf->pdev->dev, peer_dev_int);
+
+ return 0;
+}
+
+/**
+ * ice_reserve_peer_qvector - Reserve vector resources for peer drivers
+ * @pf: board private structure to initialize
+ */
+static int ice_reserve_peer_qvector(struct ice_pf *pf)
+{
+ if (test_bit(ICE_FLAG_IWARP_ENA, pf->flags)) {
+ int index;
+
+ index = ice_get_res(pf, pf->irq_tracker, pf->num_rdma_msix,
+ ICE_RES_RDMA_VEC_ID);
+ if (index < 0)
+ return index;
+ pf->num_avail_sw_msix -= pf->num_rdma_msix;
+ pf->rdma_base_vector = index;
+ }
+ return 0;
+}
+
+/**
+ * ice_peer_device_release - release function for platform peer device
+ * @dev: pointer to device structure
+ *
+ * This callback function is accessed by platform bus infrastructure when
+ * all references on the peer platform device we registered are removed
+ * (e.g. the peer priver has been removed). We will use this opportunity
+ * to set the peer's state to init to be ready if the peer driver re-loads.
+ */
+static void ice_peer_device_release(struct device *dev)
+{
+ struct ice_peer_dev_int *peer_dev_int;
+ struct ice_peer_dev *peer_dev;
+
+ peer_dev = dev_to_ice_peer(dev);
+ peer_dev_int = container_of(peer_dev, struct ice_peer_dev_int,
+ peer_dev);
+
+ ice_peer_state_change(peer_dev_int, ICE_PEER_DEV_STATE_INIT, false);
+}
+
+/**
+ * ice_init_peer_devices - initializes peer devices
+ * @pf: ptr to ice_pf
+ *
+ * This function initializes peer devices and associates them with specified
+ * pci_dev as their parent.
+ */
+int ice_init_peer_devices(struct ice_pf *pf)
+{
+ struct pci_dev *pdev = pf->pdev;
+ struct msix_entry *entry = NULL;
+ struct ice_port_info *port_info;
+ struct ice_vsi *vsi;
+ int status = 0;
+ int i;
+
+ /* Reserve vector resources */
+ status = ice_reserve_peer_qvector(pf);
+ if (status < 0) {
+ dev_err(&pdev->dev,
+ "failed to reserve vectors for peer drivers\n");
+ return status;
+ }
+ for (i = 0; i < ARRAY_SIZE(peer_dev_ids); i++) {
+ struct ice_peer_dev_int *peer_dev_int;
+ struct platform_device *platform_dev;
+ struct ice_qos_params *qos_info;
+ struct ice_peer_dev *peer_dev;
+ int j;
+
+ /* don't create an RDMA platform_device if NIC does not
+ * support RDMA functionality
+ */
+ if (peer_dev_ids[i].id == ICE_PEER_RDMA_DEV &&
+ !test_bit(ICE_FLAG_IWARP_ENA, pf->flags)) {
+ dev_warn(&pf->pdev->dev,
+ "RDMA not possible with this device config\n");
+ continue;
+ }
+
+ peer_dev_int = devm_kzalloc(&pdev->dev, sizeof(*peer_dev_int),
+ GFP_KERNEL);
+ if (!peer_dev_int)
+ return -ENOMEM;
+ pf->peers[i] = peer_dev_int;
+
+ mutex_init(&peer_dev_int->peer_dev_state_mutex);
+
+ peer_dev = &peer_dev_int->peer_dev;
+ peer_dev->peer_ops = NULL;
+ peer_dev->hw_addr = (u8 __iomem *)pf->hw.hw_addr;
+ peer_dev->ver.major = ICE_PEER_MAJOR_VER;
+ peer_dev->ver.minor = ICE_PEER_MINOR_VER;
+ peer_dev->ver.support = ICE_IDC_FEATURES;
+ peer_dev->peer_dev_id = peer_dev_ids[i].id;
+ port_info = pf->hw.port_info;
+ vsi = pf->vsi[0];
+ peer_dev->pf_vsi_num = vsi->vsi_num;
+ peer_dev->netdev = vsi->netdev;
+ peer_dev->initial_mtu = vsi->netdev->mtu;
+ ether_addr_copy(peer_dev->lan_addr, port_info->mac.lan_addr);
+
+ platform_dev = &peer_dev->platform_dev;
+ platform_dev->name = peer_dev_ids[i].name;
+ platform_dev->id = PLATFORM_DEVID_AUTO;
+ platform_dev->id_auto = true;
+ platform_dev->dev.release = ice_peer_device_release;
+ platform_dev->dev.parent = &pdev->dev;
+ peer_dev_int->ice_peer_wq =
+ alloc_ordered_workqueue("ice_peer_wq_%d", WQ_UNBOUND,
+ i);
+ if (!peer_dev_int->ice_peer_wq)
+ return -ENOMEM;
+
+ /* Assign a unique index and hence name for peer device */
+ status = ida_simple_get(&ice_peer_index_ida, 0, 0, GFP_KERNEL);
+ if (status < 0) {
+ dev_err(&pdev->dev,
+ "failed to get unique index for device\n");
+ devm_kfree(&pdev->dev, peer_dev);
+ return status;
+ }
+ peer_dev->index = status;
+ peer_dev->pdev = pdev;
+ peer_dev->ari_ena = pci_ari_enabled(pdev->bus);
+ peer_dev->bus_num = PCI_BUS_NUM(pdev->devfn);
+ if (!peer_dev->ari_ena) {
+ peer_dev->dev_num = PCI_SLOT(pdev->devfn);
+ peer_dev->fn_num = PCI_FUNC(pdev->devfn);
+ } else {
+ peer_dev->dev_num = 0;
+ peer_dev->fn_num = pdev->devfn & 0xff;
+ }
+
+ qos_info = &peer_dev->initial_qos_info;
+
+ /* setup qos_info fields with defaults */
+ qos_info->num_apps = 0;
+ qos_info->num_tc = 1;
+
+ for (j = 0; j < ICE_IDC_MAX_USER_PRIORITY; j++)
+ qos_info->up2tc[j] = 0;
+
+ qos_info->tc_info[0].rel_bw = 100;
+ for (j = 1; j < IEEE_8021QAZ_MAX_TCS; j++)
+ qos_info->tc_info[j].rel_bw = 0;
+
+ /* for DCB, override the qos_info defaults. */
+ ice_setup_dcb_qos_info(pf, qos_info);
+
+ /* make sure peer specific resources such as msix_count and
+ * msix_entries are initialized
+ */
+ switch (peer_dev_ids[i].id) {
+ case ICE_PEER_RDMA_DEV:
+ if (test_bit(ICE_FLAG_IWARP_ENA, pf->flags)) {
+ peer_dev->msix_count = pf->num_rdma_msix;
+ entry = &pf->msix_entries[pf->rdma_base_vector];
+ }
+ break;
+ default:
+ break;
+ }
+
+ peer_dev->msix_entries = entry;
+ ice_peer_state_change(peer_dev_int, ICE_PEER_DEV_STATE_INIT,
+ false);
+ status = platform_device_register(platform_dev);
+ if (status) {
+ pr_err("Failed to register peer device %s\n",
+ platform_dev->name);
+ return status;
+ }
+ }
+
+ return status;
+}
diff --git a/drivers/net/ethernet/intel/ice/ice_idc.h b/drivers/net/ethernet/intel/ice/ice_idc.h
new file mode 100644
index 000000000000..d29e946fe2f1
--- /dev/null
+++ b/drivers/net/ethernet/intel/ice/ice_idc.h
@@ -0,0 +1,360 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2019, Intel Corporation. */
+
+#ifndef _ICE_IDC_H_
+#define _ICE_IDC_H_
+
+#include <linux/kernel.h>
+#include <linux/device.h>
+#include <linux/if_ether.h>
+#include <linux/netdevice.h>
+#include <linux/dcbnl.h>
+#include <linux/platform_device.h>
+
+/* This major and minor version represent IDC API version information.
+ * During peer driver registration, peer driver specifies major and minor
+ * version information (via. peer_driver:ver_info). It gets checked against
+ * following defines and if mismatch, then peer driver registration
+ * fails and appropriate message gets logged.
+ */
+#define ICE_PEER_MAJOR_VER 6
+#define ICE_PEER_MINOR_VER 1
+
+enum ice_peer_features {
+ ICE_PEER_FEATURE_ADK_SUPPORT,
+ ICE_PEER_FEATURE_PTP_SUPPORT,
+ ICE_PEER_FEATURE_SRIOV_SUPPORT,
+ ICE_PEER_FEATURE_PCIIOV_SUPPORT,
+ ICE_PEER_FEATURE_NBITS
+};
+
+#define ICE_SRIOV_SUP BIT(ICE_PEER_FEATURE_SRIOV_SUPPORT)
+
+#ifdef CONFIG_PCI_IOV
+#define ICE_PCIIOV_SUP BIT(ICE_PEER_FEATURE_PCIIOV_SUPPORT)
+#else
+#define ICE_PCIIOV_SUP 0
+#endif /* CONFIG_PCI_IOV */
+
+#define ICE_IDC_FEATURES (ICE_SRIOV_SUP | ICE_PCIIOV_SUP)
+
+enum ice_event_type {
+ ICE_EVENT_LINK_CHANGE = 0x0,
+ ICE_EVENT_MTU_CHANGE,
+ ICE_EVENT_TC_CHANGE,
+ ICE_EVENT_API_CHANGE,
+ ICE_EVENT_MBX_CHANGE,
+ ICE_EVENT_NBITS /* must be last */
+};
+
+enum ice_res_type {
+ ICE_INVAL_RES = 0x0,
+ ICE_VSI,
+ ICE_VEB,
+ ICE_EVENT_Q,
+ ICE_EGRESS_CMPL_Q,
+ ICE_CMPL_EVENT_Q,
+ ICE_ASYNC_EVENT_Q,
+ ICE_DOORBELL_Q,
+ ICE_RDMA_QSETS_TXSCHED,
+};
+
+enum ice_peer_reset_type {
+ ICE_PEER_PFR = 0,
+ ICE_PEER_CORER,
+ ICE_PEER_CORER_SW_CORE,
+ ICE_PEER_CORER_SW_FULL,
+ ICE_PEER_GLOBR,
+};
+
+/* reason notified to peer driver as part of event handling */
+enum ice_close_reason {
+ ICE_REASON_INVAL = 0x0,
+ ICE_REASON_HW_UNRESPONSIVE,
+ ICE_REASON_INTERFACE_DOWN, /* Administrative down */
+ ICE_REASON_PEER_DRV_UNREG, /* peer driver getting unregistered */
+ ICE_REASON_PEER_DEV_UNINIT,
+ ICE_REASON_GLOBR_REQ,
+ ICE_REASON_CORER_REQ,
+ ICE_REASON_EMPR_REQ,
+ ICE_REASON_PFR_REQ,
+ ICE_REASON_HW_RESET_PENDING,
+ ICE_REASON_PARAM_CHANGE,
+};
+
+enum ice_rdma_filter {
+ ICE_RDMA_FILTER_INVAL = 0x0,
+ ICE_RDMA_FILTER_IWARP,
+ ICE_RDMA_FILTER_ROCEV2,
+ ICE_RDMA_FILTER_BOTH,
+};
+
+/* This information is needed to handle peer driver registration,
+ * instead of adding more params to peer_drv_registration function,
+ * let's get it thru' peer_drv object.
+ */
+struct ice_ver_info {
+ u16 major;
+ u16 minor;
+ u64 support;
+};
+
+/* Struct to hold per DCB APP info */
+struct ice_dcb_app_info {
+ u8 priority;
+ u8 selector;
+ u16 prot_id;
+};
+
+struct ice_peer_dev;
+struct ice_peer_dev_int;
+
+#define ICE_IDC_MAX_USER_PRIORITY 8
+#define ICE_IDC_MAX_APPS 8
+
+/* Struct to hold per RDMA Qset info */
+struct ice_rdma_qset_params {
+ u32 teid; /* qset TEID */
+ u16 qs_handle; /* RDMA driver provides this */
+ u16 vsi_id; /* VSI index */
+ u8 tc; /* TC branch the QSet should belong to */
+ u8 reserved[3];
+};
+
+struct ice_res_base {
+ /* Union for future provision e.g. other res_type */
+ union {
+ struct ice_rdma_qset_params qsets;
+ } res;
+};
+
+struct ice_res {
+ /* Type of resource. Filled by peer driver */
+ enum ice_res_type res_type;
+ /* Count requested by peer driver */
+ u16 cnt_req;
+
+ /* Number of resources allocated. Filled in by callee.
+ * Based on this value, caller to fill up "resources"
+ */
+ u16 res_allocated;
+
+ /* Unique handle to resources allocated. Zero if call fails.
+ * Allocated by callee and for now used by caller for internal
+ * tracking purpose.
+ */
+ u32 res_handle;
+
+ /* Peer driver has to allocate sufficient memory, to accommodate
+ * cnt_requested before calling this function.
+ * Memory has to be zero initialized. It is input/output param.
+ * As a result of alloc_res API, this structures will be populated.
+ */
+ struct ice_res_base res[1];
+};
+
+struct ice_qos_info {
+ u64 tc_ctx;
+ u8 rel_bw;
+ u8 prio_type;
+ u8 egress_virt_up;
+ u8 ingress_virt_up;
+};
+
+/* Struct to hold QoS info */
+struct ice_qos_params {
+ struct ice_qos_info tc_info[IEEE_8021QAZ_MAX_TCS];
+ u8 up2tc[ICE_IDC_MAX_USER_PRIORITY];
+ u8 vsi_relative_bw;
+ u8 vsi_priority_type;
+ u32 num_apps;
+ struct ice_dcb_app_info apps[ICE_IDC_MAX_APPS];
+ u8 num_tc;
+};
+
+union ice_event_info {
+ /* ICE_EVENT_LINK_CHANGE */
+ struct {
+ struct net_device *lwr_nd;
+ u16 vsi_num; /* HW index of VSI corresponding to lwr ndev */
+ u8 new_link_state;
+ u8 lport;
+ } link_info;
+ /* ICE_EVENT_MTU_CHANGE */
+ u16 mtu;
+ /* ICE_EVENT_TC_CHANGE */
+ struct ice_qos_params port_qos;
+ /* ICE_EVENT_API_CHANGE */
+ u8 api_rdy;
+ /* ICE_EVENT_MBX_CHANGE */
+ u8 mbx_rdy;
+};
+
+/* ice_event elements are to be passed back and forth between the ice driver
+ * and the peer drivers. They are to be used to both register/unregister
+ * for event reporting and to report an event (events can be either ice
+ * generated or peer generated).
+ *
+ * For (un)registering for events, the structure needs to be populated with:
+ * reporter - pointer to the ice_peer_dev struct of the peer (un)registering
+ * type - bitmap with bits set for event types to (un)register for
+ *
+ * For reporting events, the structure needs to be populated with:
+ * reporter - pointer to peer that generated the event (NULL for ice)
+ * type - bitmap with single bit set for this event type
+ * info - union containing data relevant to this event type
+ */
+struct ice_event {
+ struct ice_peer_dev *reporter;
+ DECLARE_BITMAP(type, ICE_EVENT_NBITS);
+ union ice_event_info info;
+};
+
+/* Following APIs are implemented by ICE driver and invoked by peer drivers */
+struct ice_ops {
+ /* APIs to allocate resources such as VEB, VSI, Doorbell queues,
+ * completion queues, Tx/Rx queues, etc...
+ */
+ int (*alloc_res)(struct ice_peer_dev *peer_dev,
+ struct ice_res *res,
+ int partial_acceptable);
+ int (*free_res)(struct ice_peer_dev *peer_dev,
+ struct ice_res *res);
+
+ int (*is_vsi_ready)(struct ice_peer_dev *peer_dev);
+ int (*peer_register)(struct ice_peer_dev *peer_dev);
+ int (*peer_unregister)(struct ice_peer_dev *peer_dev);
+ int (*request_reset)(struct ice_peer_dev *dev,
+ enum ice_peer_reset_type reset_type);
+
+ void (*notify_state_change)(struct ice_peer_dev *dev,
+ struct ice_event *event);
+
+ /* Notification APIs */
+ void (*reg_for_notification)(struct ice_peer_dev *dev,
+ struct ice_event *event);
+ void (*unreg_for_notification)(struct ice_peer_dev *dev,
+ struct ice_event *event);
+ int (*update_vsi_filter)(struct ice_peer_dev *peer_dev,
+ enum ice_rdma_filter filter, bool enable);
+ int (*vc_send)(struct ice_peer_dev *peer_dev, u32 vf_id, u8 *msg,
+ u16 len);
+};
+
+/* Following APIs are implemented by peer drivers and invoked by ICE driver */
+struct ice_peer_ops {
+ void (*event_handler)(struct ice_peer_dev *peer_dev,
+ struct ice_event *event);
+
+ /* Why we have 'open' and when it is expected to be called:
+ * 1. symmetric set of API w.r.t close
+ * 2. To be invoked form driver initialization path
+ * - call peer_driver:open once ice driver is fully initialized
+ * 3. To be invoked upon RESET complete
+ *
+ * Calls to open are performed from ice_finish_init_peer_device
+ * which is invoked from the service task. This helps keep devices
+ * from having their open called until the ice driver is ready and
+ * has scheduled its service task.
+ */
+ int (*open)(struct ice_peer_dev *peer_dev);
+
+ /* Peer's close function is to be called when the peer needs to be
+ * quiesced. This can be for a variety of reasons (enumerated in the
+ * ice_close_reason enum struct). A call to close will only be
+ * followed by a call to either remove or open. No IDC calls from the
+ * peer should be accepted until it is re-opened.
+ *
+ * The *reason* parameter is the reason for the call to close. This
+ * can be for any reason enumerated in the ice_close_reason struct.
+ * It's primary reason is for the peer's bookkeeping and in case the
+ * peer want to perform any different tasks dictated by the reason.
+ */
+ void (*close)(struct ice_peer_dev *peer_dev,
+ enum ice_close_reason reason);
+
+ int (*vc_receive)(struct ice_peer_dev *peer_dev, u32 vf_id, u8 *msg,
+ u16 len);
+ /* tell RDMA peer to prepare for TC change in a blocking call
+ * that will directly precede the change event
+ */
+ void (*prep_tc_change)(struct ice_peer_dev *peer_dev);
+};
+
+#define ICE_PEER_RDMA_NAME "ice_rdma"
+#define ICE_PEER_RDMA_DEV 0x00000010
+#define ICE_MAX_NUM_PEERS 4
+
+/* The const struct that instantiates peer_dev_id needs to be initialized
+ * in the .c with the macro ASSIGN_PEER_INFO.
+ * For example:
+ * static const struct peer_dev_id peer_dev_ids[] = ASSIGN_PEER_INFO;
+ */
+struct peer_dev_id {
+ char *name;
+ int id;
+};
+
+#define ASSIGN_PEER_INFO \
+{ \
+ { ICE_PEER_RDMA_NAME, ICE_PEER_RDMA_DEV }, \
+}
+
+/* structure representing peer device */
+struct ice_peer_dev {
+ struct platform_device platform_dev;
+ struct ice_ver_info ver;
+ struct pci_dev *pdev; /* PCI device of corresponding to main function */
+ /* KVA / Linear address corresponding to BAR0 of underlying
+ * pci_device.
+ */
+ u8 __iomem *hw_addr;
+ int peer_dev_id;
+
+ unsigned int index;
+
+ u8 ftype; /* PF(false) or VF (true) */
+
+ /* Data VSI created by driver */
+ u16 pf_vsi_num;
+
+ u8 lan_addr[ETH_ALEN]; /* default MAC address of main netdev */
+ u16 initial_mtu; /* Initial MTU of main netdev */
+ struct ice_qos_params initial_qos_info;
+ struct net_device *netdev;
+ /* PCI info */
+ u8 ari_ena;
+ u16 bus_num;
+ u16 dev_num;
+ u16 fn_num;
+
+ /* Based on peer driver type, this shall point to corresponding MSIx
+ * entries in pf->msix_entries (which were allocated as part of driver
+ * initialization) e.g. for RDMA driver, msix_entries reserved will be
+ * num_online_cpus + 1.
+ */
+ u16 msix_count; /* How many vectors are reserved for this device */
+ struct msix_entry *msix_entries;
+
+ /* Following struct contains function pointers to be initialized
+ * by ICE driver and called by peer driver
+ */
+ const struct ice_ops *ops;
+
+ /* Following struct contains function pointers to be initialized
+ * by peer driver and called by ICE driver
+ */
+ const struct ice_peer_ops *peer_ops;
+
+ /* Pointer to peer_drv struct to be populated by peer driver */
+ struct ice_peer_drv *peer_drv;
+};
+
+static inline struct ice_peer_dev *dev_to_ice_peer(struct device *_dev)
+{
+ struct platform_device *platform_dev;
+
+ platform_dev = container_of(_dev, struct platform_device, dev);
+ return container_of(platform_dev, struct ice_peer_dev, platform_dev);
+}
+#endif /* _ICE_IDC_H_*/
diff --git a/drivers/net/ethernet/intel/ice/ice_idc_int.h b/drivers/net/ethernet/intel/ice/ice_idc_int.h
new file mode 100644
index 000000000000..f8ff8cac7a26
--- /dev/null
+++ b/drivers/net/ethernet/intel/ice/ice_idc_int.h
@@ -0,0 +1,67 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2019, Intel Corporation. */
+
+#ifndef _ICE_IDC_INT_H_
+#define _ICE_IDC_INT_H_
+
+#include "ice_idc.h"
+
+int ice_unroll_peer(struct ice_peer_dev_int *peer_dev_int, void *data);
+int ice_unreg_peer_device(struct ice_peer_dev_int *peer_dev_int, void *data);
+
+enum ice_peer_dev_state {
+ ICE_PEER_DEV_STATE_INIT,
+ ICE_PEER_DEV_STATE_PROBED,
+ ICE_PEER_DEV_STATE_OPENING,
+ ICE_PEER_DEV_STATE_OPENED,
+ ICE_PEER_DEV_STATE_PREP_RST,
+ ICE_PEER_DEV_STATE_PREPPED,
+ ICE_PEER_DEV_STATE_CLOSING,
+ ICE_PEER_DEV_STATE_CLOSED,
+ ICE_PEER_DEV_STATE_REMOVED,
+ ICE_PEER_DEV_STATE_API_RDY,
+ ICE_PEER_DEV_STATE_NBITS, /* must be last */
+};
+
+enum ice_peer_drv_state {
+ ICE_PEER_DRV_STATE_MBX_RDY,
+ ICE_PEER_DRV_STATE_NBITS, /* must be last */
+};
+
+struct ice_peer_drv_int {
+ struct ice_peer_drv *peer_drv;
+ struct ice_peer_dev *peer_dev;
+
+ /* States associated with peer driver */
+ DECLARE_BITMAP(state, ICE_PEER_DRV_STATE_NBITS);
+
+ /* if this peer_dev is the originator of an event, these are the
+ * most recent events of each type
+ */
+ struct ice_event current_events[ICE_EVENT_NBITS];
+};
+
+struct ice_peer_dev_int {
+ struct ice_peer_dev peer_dev; /* public structure */
+ struct ice_peer_drv_int *peer_drv_int; /* driver private structure */
+
+ /* if this peer_dev is the originator of an event, these are the
+ * most recent events of each type
+ */
+ struct ice_event current_events[ICE_EVENT_NBITS];
+ /* Events a peer has registered to be notified about */
+ DECLARE_BITMAP(events, ICE_EVENT_NBITS);
+
+ /* States associated with peer device */
+ DECLARE_BITMAP(state, ICE_PEER_DEV_STATE_NBITS);
+ struct mutex peer_dev_state_mutex; /* peer_dev state mutex */
+
+ /* per peer workqueue */
+ struct workqueue_struct *ice_peer_wq;
+
+ struct work_struct peer_prep_task;
+ struct work_struct peer_close_task;
+
+ enum ice_close_reason rst_type;
+};
+#endif /* !_ICE_IDC_INT_H_ */
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 28ec0d57941d..08bce0d67634 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -2092,6 +2092,8 @@ static void ice_init_pf(struct ice_pf *pf)
{
bitmap_zero(pf->flags, ICE_PF_FLAGS_NBITS);
set_bit(ICE_FLAG_MSIX_ENA, pf->flags);
+ if (pf->hw.func_caps.common_cap.iwarp)
+ set_bit(ICE_FLAG_IWARP_ENA, pf->flags);
#ifdef CONFIG_PCI_IOV
if (pf->hw.func_caps.common_cap.sr_iov_1_1) {
struct ice_hw *hw = &pf->hw;
@@ -2145,6 +2147,17 @@ static int ice_ena_msix_range(struct ice_pf *pf)
v_budget += pf->num_lan_msix;
v_left -= pf->num_lan_msix;
+ /* reserve vectors for RDMA peer driver */
+ if (test_bit(ICE_FLAG_IWARP_ENA, pf->flags)) {
+ /* RDMA peer driver needs one extra to handle misc causes */
+ needed = min_t(int, num_online_cpus(), v_left) + 1;
+ if (v_left < needed)
+ goto no_vecs_left_err;
+ pf->num_rdma_msix = needed;
+ v_budget += needed;
+ v_left -= needed;
+ }
+
pf->msix_entries = devm_kcalloc(&pf->pdev->dev, v_budget,
sizeof(*pf->msix_entries), GFP_KERNEL);
@@ -2171,6 +2184,8 @@ static int ice_ena_msix_range(struct ice_pf *pf)
"not enough vectors. requested = %d, obtained = %d\n",
v_budget, v_actual);
if (v_actual >= (pf->num_lan_msix + 1)) {
+ clear_bit(ICE_FLAG_IWARP_ENA, pf->flags);
+ pf->num_rdma_msix = 0;
pf->num_avail_sw_msix = v_actual -
(pf->num_lan_msix + 1);
} else if (v_actual >= 2) {
@@ -2189,6 +2204,11 @@ static int ice_ena_msix_range(struct ice_pf *pf)
devm_kfree(&pf->pdev->dev, pf->msix_entries);
goto exit_err;
+no_vecs_left_err:
+ dev_err(&pf->pdev->dev,
+ "not enough vectors. requested = %d, available = %d\n",
+ needed, v_left);
+ err = -ERANGE;
exit_err:
pf->num_lan_msix = 0;
clear_bit(ICE_FLAG_MSIX_ENA, pf->flags);
@@ -2351,6 +2371,12 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)
ice_init_pf(pf);
+ pf->peers = devm_kcalloc(dev, ICE_MAX_NUM_PEERS, sizeof(*pf->peers),
+ GFP_KERNEL);
+ if (!pf->peers) {
+ err = -ENOMEM;
+ goto err_init_peer_unroll;
+ }
err = ice_init_pf_dcb(pf, false);
if (err) {
clear_bit(ICE_FLAG_DCB_CAPABLE, pf->flags);
@@ -2432,10 +2458,20 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)
goto err_alloc_sw_unroll;
}
+ err = ice_init_peer_devices(pf);
+ if (err) {
+ dev_err(dev, "Failed to initialize peer devices: 0x%x\n", err);
+ err = -EIO;
+ goto err_init_peer_unroll;
+ }
+
ice_verify_cacheline_size(pf);
return 0;
+ /* Unwind non-managed device resources, etc. if something failed */
+err_init_peer_unroll:
+ ice_for_each_peer(pf, NULL, ice_unroll_peer);
err_alloc_sw_unroll:
set_bit(__ICE_SERVICE_DIS, pf->state);
set_bit(__ICE_DOWN, pf->state);
@@ -2477,6 +2513,7 @@ static void ice_remove(struct pci_dev *pdev)
if (test_bit(ICE_FLAG_SRIOV_ENA, pf->flags))
ice_free_vfs(pf);
ice_vsi_release_all(pf);
+ ice_for_each_peer(pf, NULL, ice_unreg_peer_device);
ice_free_irq_msix_misc(pf);
ice_for_each_vsi(pf, i) {
if (!pf->vsi[i])
@@ -2676,6 +2713,7 @@ static int __init ice_module_init(void)
if (status) {
pr_err("failed to register PCI driver, err %d\n", status);
destroy_workqueue(ice_wq);
+ ida_destroy(&ice_peer_index_ida);
}
return status;
@@ -2692,6 +2730,10 @@ static void __exit ice_module_exit(void)
{
pci_unregister_driver(&ice_driver);
destroy_workqueue(ice_wq);
+ /* release all cached layer within ida tree, associated with
+ * ice_peer_index_ida object
+ */
+ ida_destroy(&ice_peer_index_ida);
pr_info("module unloaded\n");
}
module_exit(ice_module_exit);
diff --git a/drivers/net/ethernet/intel/ice/ice_type.h b/drivers/net/ethernet/intel/ice/ice_type.h
index 24bbef8bbe69..99822b4e8621 100644
--- a/drivers/net/ethernet/intel/ice/ice_type.h
+++ b/drivers/net/ethernet/intel/ice/ice_type.h
@@ -161,6 +161,7 @@ struct ice_hw_common_caps {
u8 rss_table_entry_width; /* RSS Entry width in bits */
u8 dcb;
+ u8 iwarp;
};
/* Function specific capabilities */
--
2.21.0
^ permalink raw reply related
* [net-next 3/3] i40e: Register RDMA client devices to the virtual platform bus
From: Jeff Kirsher @ 2019-07-04 2:12 UTC (permalink / raw)
To: davem, dledford, jgg
Cc: Shiraz Saleem, netdev, linux-rdma, nhorman, sassmann,
mustafa.ismail, david.m.ertman, Andrew Bowers, Jeff Kirsher
In-Reply-To: <20190704021252.15534-1-jeffrey.t.kirsher@intel.com>
From: Shiraz Saleem <shiraz.saleem@intel.com>
Register RDMA client devices to the virtual platform bus. The
unified RDMA driver 'irdma' registers as a platform driver and
will bind to these devices. This model is inspired by the discussion
in [1]. It realizes a single RDMA driver capable of working with
multiple LAN drivers over multi-generation Intel HW supporting RDMA.
There is also no load ordering dependencies between i40e and irdma.
Summary of changes:
* Register a platform client device per PF and unregister the platform
device when PF goes away.
* Add 2 new client ops.
* i40e_client_device_register() which is called during RDMA
probe() per PF. Validate client drv OPs and schedule service
task to call open()
* i40e_client_device_unregister() called during RDMA remove()
per PF. Call client close() and release_qvlist.
* The global register/unregister calls exported for i40iw are retained
until i40iw is removed from the kernel.
[1] https://patchwork.kernel.org/patch/10815567/
Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e_client.c | 116 ++++++++++++++----
drivers/net/ethernet/intel/i40e/i40e_client.h | 8 ++
2 files changed, 102 insertions(+), 22 deletions(-)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_client.c b/drivers/net/ethernet/intel/i40e/i40e_client.c
index e81530ca08d0..133950d190a6 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_client.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_client.c
@@ -30,11 +30,17 @@ static int i40e_client_update_vsi_ctxt(struct i40e_info *ldev,
bool is_vf, u32 vf_id,
u32 flag, u32 valid_flag);
+static int i40e_client_device_register(struct i40e_info *ldev);
+
+static void i40e_client_device_unregister(struct i40e_info *ldev);
+
static struct i40e_ops i40e_lan_ops = {
.virtchnl_send = i40e_client_virtchnl_send,
.setup_qvlist = i40e_client_setup_qvlist,
.request_reset = i40e_client_request_reset,
.update_vsi_ctxt = i40e_client_update_vsi_ctxt,
+ .client_device_register = i40e_client_device_register,
+ .client_device_unregister = i40e_client_device_unregister,
};
/**
@@ -71,6 +77,10 @@ int i40e_client_get_params(struct i40e_vsi *vsi, struct i40e_params *params)
return 0;
}
+static void i40e_client_device_release(struct device *dev)
+{
+}
+
/**
* i40e_notify_client_of_vf_msg - call the client vf message callback
* @vsi: the VSI with the message
@@ -287,9 +297,7 @@ static void i40e_client_add_instance(struct i40e_pf *pf)
struct i40e_client_instance *cdev = NULL;
struct netdev_hw_addr *mac = NULL;
struct i40e_vsi *vsi = pf->vsi[pf->lan_vsi];
-
- if (!registered_client || pf->cinst)
- return;
+ struct platform_device *platform_dev;
cdev = kzalloc(sizeof(*cdev), GFP_KERNEL);
if (!cdev)
@@ -308,6 +316,12 @@ static void i40e_client_add_instance(struct i40e_pf *pf)
cdev->lan_info.fw_maj_ver = pf->hw.aq.fw_maj_ver;
cdev->lan_info.fw_min_ver = pf->hw.aq.fw_min_ver;
cdev->lan_info.fw_build = pf->hw.aq.fw_build;
+ platform_dev = &cdev->lan_info.platform_dev;
+ platform_dev->name = "i40e_rdma";
+ platform_dev->id = PLATFORM_DEVID_AUTO;
+ platform_dev->id_auto = true;
+ platform_dev->dev.release = i40e_client_device_release;
+ platform_dev->dev.parent = &pf->pdev->dev;
set_bit(__I40E_CLIENT_INSTANCE_NONE, &cdev->state);
if (i40e_client_get_params(vsi, &cdev->lan_info.params)) {
@@ -323,10 +337,12 @@ static void i40e_client_add_instance(struct i40e_pf *pf)
else
dev_err(&pf->pdev->dev, "MAC address list is empty!\n");
- cdev->client = registered_client;
+ cdev->client = NULL;
pf->cinst = cdev;
- i40e_client_update_msix_info(pf);
+ cdev->lan_info.msix_count = pf->num_iwarp_msix;
+ cdev->lan_info.msix_entries = &pf->msix_entries[pf->iwarp_base_vector];
+ platform_device_register(platform_dev);
}
/**
@@ -347,7 +363,7 @@ void i40e_client_del_instance(struct i40e_pf *pf)
**/
void i40e_client_subtask(struct i40e_pf *pf)
{
- struct i40e_client *client = registered_client;
+ struct i40e_client *client;
struct i40e_client_instance *cdev;
struct i40e_vsi *vsi = pf->vsi[pf->lan_vsi];
int ret = 0;
@@ -361,9 +377,11 @@ void i40e_client_subtask(struct i40e_pf *pf)
test_bit(__I40E_CONFIG_BUSY, pf->state))
return;
- if (!client || !cdev)
+ if (!cdev || !cdev->client)
return;
+ client = cdev->client;
+
/* Here we handle client opens. If the client is down, and
* the netdev is registered, then open the client.
*/
@@ -424,16 +442,7 @@ int i40e_lan_add_device(struct i40e_pf *pf)
pf->hw.pf_id, pf->hw.bus.bus_id,
pf->hw.bus.device, pf->hw.bus.func);
- /* If a client has already been registered, we need to add an instance
- * of it to our new LAN device.
- */
- if (registered_client)
- i40e_client_add_instance(pf);
-
- /* Since in some cases register may have happened before a device gets
- * added, we can schedule a subtask to go initiate the clients if
- * they can be launched at probe time.
- */
+ i40e_client_add_instance(pf);
set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
i40e_service_event_schedule(pf);
@@ -453,6 +462,8 @@ int i40e_lan_del_device(struct i40e_pf *pf)
struct i40e_device *ldev, *tmp;
int ret = -ENODEV;
+ platform_device_unregister(&pf->cinst->lan_info.platform_dev);
+
/* First, remove any client instance. */
i40e_client_del_instance(pf);
@@ -505,10 +516,7 @@ static void i40e_client_release(struct i40e_client *client)
"Client %s instance for PF id %d closed\n",
client->name, pf->hw.pf_id);
}
- /* delete the client instance */
- i40e_client_del_instance(pf);
- dev_info(&pf->pdev->dev, "Deleted client instance of Client %s\n",
- client->name);
+ cdev->client = NULL;
clear_bit(__I40E_SERVICE_SCHED, pf->state);
}
mutex_unlock(&i40e_device_mutex);
@@ -527,7 +535,7 @@ static void i40e_client_prepare(struct i40e_client *client)
mutex_lock(&i40e_device_mutex);
list_for_each_entry(ldev, &i40e_devices, list) {
pf = ldev->pf;
- i40e_client_add_instance(pf);
+ pf->cinst->client = registered_client;
/* Start the client subtask */
set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
i40e_service_event_schedule(pf);
@@ -733,6 +741,70 @@ static int i40e_client_update_vsi_ctxt(struct i40e_info *ldev,
return err;
}
+static int i40e_client_device_register(struct i40e_info *ldev)
+{
+ struct i40e_client *client;
+ struct i40e_pf *pf;
+
+ if (!ldev) {
+ pr_err("Failed to reg client dev: ldev ptr NULL\n");
+ return -EINVAL;
+ }
+
+ client = ldev->client;
+ pf = ldev->pf;
+ if (!client) {
+ pr_err("Failed to reg client dev: client ptr NULL\n");
+ return -EINVAL;
+ }
+
+ if (!ldev->ops || !client->ops) {
+ pr_err("Failed to reg client dev: client dev peer_ops/ops NULL\n");
+ return -EINVAL;
+ }
+
+ if (client->version.major != I40E_CLIENT_VERSION_MAJOR ||
+ client->version.minor != I40E_CLIENT_VERSION_MINOR) {
+ pr_err("i40e: Failed to register client %s due to mismatched client interface version\n",
+ client->name);
+ pr_err("Client is using version: %02d.%02d.%02d while LAN driver supports %s\n",
+ client->version.major, client->version.minor,
+ client->version.build,
+ i40e_client_interface_version_str);
+ return -EINVAL;
+ }
+
+ pf->cinst->client = ldev->client;
+ set_bit(__I40E_CLIENT_SERVICE_REQUESTED, pf->state);
+ i40e_service_event_schedule(pf);
+
+ return 0;
+}
+
+static void i40e_client_device_unregister(struct i40e_info *ldev)
+{
+ struct i40e_pf *pf = ldev->pf;
+ struct i40e_client_instance *cdev = pf->cinst;
+
+ while (test_and_set_bit(__I40E_SERVICE_SCHED, pf->state))
+ usleep_range(500, 1000);
+
+ if (!cdev || !cdev->client || !cdev->client->ops ||
+ !cdev->client->ops->close) {
+ dev_err(&pf->pdev->dev, "Cannot close client device\n");
+ return;
+ }
+ cdev->client->ops->close(&cdev->lan_info, cdev->client, false);
+ clear_bit(__I40E_CLIENT_INSTANCE_OPENED, &cdev->state);
+ i40e_client_release_qvlist(&cdev->lan_info);
+ pf->cinst->client = NULL;
+ clear_bit(__I40E_SERVICE_SCHED, pf->state);
+}
+
+/* Retain legacy global registration/unregistration calls till i40iw is
+ * deprecated from the kernel. The irdma unified driver does not use these
+ * exported symbols.
+ */
/**
* i40e_register_client - Register a i40e client driver with the L2 driver
* @client: pointer to the i40e_client struct
diff --git a/drivers/net/ethernet/intel/i40e/i40e_client.h b/drivers/net/ethernet/intel/i40e/i40e_client.h
index 72994baf4941..0cfdb35e93d6 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_client.h
+++ b/drivers/net/ethernet/intel/i40e/i40e_client.h
@@ -4,6 +4,8 @@
#ifndef _I40E_CLIENT_H_
#define _I40E_CLIENT_H_
+#include <linux/platform_device.h>
+
#define I40E_CLIENT_STR_LENGTH 10
/* Client interface version should be updated anytime there is a change in the
@@ -80,6 +82,7 @@ struct i40e_params {
/* Structure to hold Lan device info for a client device */
struct i40e_info {
+ struct platform_device platform_dev;
struct i40e_client_version version;
u8 lanmac[6];
struct net_device *netdev;
@@ -97,6 +100,7 @@ struct i40e_info {
struct i40e_qvlist_info *qvlist_info;
struct i40e_params params;
struct i40e_ops *ops;
+ struct i40e_client *client;
u16 msix_count; /* number of msix vectors*/
/* Array down below will be dynamically allocated based on msix_count */
@@ -132,6 +136,10 @@ struct i40e_ops {
struct i40e_client *client,
bool is_vf, u32 vf_id,
u32 flag, u32 valid_flag);
+
+ int (*client_device_register)(struct i40e_info *ldev);
+
+ void (*client_device_unregister)(struct i40e_info *ldev);
};
struct i40e_client_ops {
--
2.21.0
^ permalink raw reply related
* [net-next 2/3] ice: Implement peer communications
From: Jeff Kirsher @ 2019-07-04 2:12 UTC (permalink / raw)
To: davem, dledford, jgg
Cc: Tony Nguyen, netdev, linux-rdma, nhorman, sassmann,
mustafa.ismail, shiraz.saleem, Dave Ertman, Andrew Bowers,
Jeff Kirsher
In-Reply-To: <20190704021252.15534-1-jeffrey.t.kirsher@intel.com>
From: Tony Nguyen <anthony.l.nguyen@intel.com>
Set and implement operations for the peer device and peer driver to
communicate with each other, via ice_ops and ice_peer_ops, to request
resources and manage event notification.
Signed-off-by: Dave Ertman <david.m.ertman@intel.com>
Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
Tested-by: Andrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
drivers/net/ethernet/intel/ice/ice.h | 2 +
.../net/ethernet/intel/ice/ice_adminq_cmd.h | 32 +
drivers/net/ethernet/intel/ice/ice_common.c | 189 ++++
drivers/net/ethernet/intel/ice/ice_common.h | 9 +
drivers/net/ethernet/intel/ice/ice_dcb_lib.c | 33 +
drivers/net/ethernet/intel/ice/ice_idc.c | 927 ++++++++++++++++++
drivers/net/ethernet/intel/ice/ice_idc.h | 17 +
drivers/net/ethernet/intel/ice/ice_idc_int.h | 15 +
drivers/net/ethernet/intel/ice/ice_lib.c | 35 +-
drivers/net/ethernet/intel/ice/ice_lib.h | 2 +
drivers/net/ethernet/intel/ice/ice_main.c | 52 +-
drivers/net/ethernet/intel/ice/ice_sched.c | 69 +-
drivers/net/ethernet/intel/ice/ice_switch.c | 27 +
drivers/net/ethernet/intel/ice/ice_switch.h | 4 +
drivers/net/ethernet/intel/ice/ice_type.h | 3 +
.../net/ethernet/intel/ice/ice_virtchnl_pf.c | 25 -
16 files changed, 1409 insertions(+), 32 deletions(-)
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 6ee2774699f0..4855d3425043 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -302,6 +302,7 @@ struct ice_vsi {
u16 num_rxq; /* Used Rx queues */
u16 num_rx_desc;
u16 num_tx_desc;
+ u16 qset_handle[ICE_MAX_TRAFFIC_CLASS];
struct ice_tc_cfg tc_cfg;
} ____cacheline_internodealigned_in_smp;
@@ -471,6 +472,7 @@ struct ice_vsi *ice_lb_vsi_setup(struct ice_pf *pf, struct ice_port_info *pi);
int ice_set_rss(struct ice_vsi *vsi, u8 *seed, u8 *lut, u16 lut_size);
int ice_get_rss(struct ice_vsi *vsi, u8 *seed, u8 *lut, u16 lut_size);
void ice_fill_rss_lut(u8 *lut, u16 rss_table_size, u16 rss_size);
+int ice_schedule_reset(struct ice_pf *pf, enum ice_reset_req reset);
void ice_print_link_msg(struct ice_vsi *vsi, bool isup);
int ice_init_peer_devices(struct ice_pf *pf);
int
diff --git a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
index 1e1bd5f0c2a2..8e4f20b696ae 100644
--- a/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
+++ b/drivers/net/ethernet/intel/ice/ice_adminq_cmd.h
@@ -1441,6 +1441,36 @@ struct ice_aqc_dis_txq {
struct ice_aqc_dis_txq_item qgrps[1];
};
+/* Add Tx RDMA Queue Set (indirect 0x0C33) */
+struct ice_aqc_add_rdma_qset {
+ u8 num_qset_grps;
+ u8 reserved[7];
+ __le32 addr_high;
+ __le32 addr_low;
+};
+
+/* This is the descriptor of each qset entry for the Add Tx RDMA Queue Set
+ * command (0x0C33). Only used within struct ice_aqc_add_rdma_qset.
+ */
+struct ice_aqc_add_tx_rdma_qset_entry {
+ __le16 tx_qset_id;
+ u8 rsvd[2];
+ __le32 qset_teid;
+ struct ice_aqc_txsched_elem info;
+};
+
+/* The format of the command buffer for Add Tx RDMA Queue Set(0x0C33)
+ * is an array of the following structs. Please note that the length of
+ * each struct ice_aqc_add_rdma_qset is variable due to the variable
+ * number of queues in each group!
+ */
+struct ice_aqc_add_rdma_qset_data {
+ __le32 parent_teid;
+ __le16 num_qsets;
+ u8 rsvd[2];
+ struct ice_aqc_add_tx_rdma_qset_entry rdma_qsets[1];
+};
+
/* Configure Firmware Logging Command (indirect 0xFF09)
* Logging Information Read Response (indirect 0xFF10)
* Note: The 0xFF10 command has no input parameters.
@@ -1576,6 +1606,7 @@ struct ice_aq_desc {
struct ice_aqc_get_set_rss_key get_set_rss_key;
struct ice_aqc_add_txqs add_txqs;
struct ice_aqc_dis_txqs dis_txqs;
+ struct ice_aqc_add_rdma_qset add_rdma_qset;
struct ice_aqc_add_get_update_free_vsi vsi_cmd;
struct ice_aqc_add_update_free_vsi_resp add_update_free_vsi_res;
struct ice_aqc_fw_logging fw_logging;
@@ -1697,6 +1728,7 @@ enum ice_adminq_opc {
/* Tx queue handling commands/events */
ice_aqc_opc_add_txqs = 0x0C30,
ice_aqc_opc_dis_txqs = 0x0C31,
+ ice_aqc_opc_add_rdma_qset = 0x0C33,
/* debug commands */
ice_aqc_opc_fw_logging = 0xFF09,
diff --git a/drivers/net/ethernet/intel/ice/ice_common.c b/drivers/net/ethernet/intel/ice/ice_common.c
index 2788b2ede440..6cc8cfd97f88 100644
--- a/drivers/net/ethernet/intel/ice/ice_common.c
+++ b/drivers/net/ethernet/intel/ice/ice_common.c
@@ -2711,6 +2711,59 @@ ice_aq_dis_lan_txq(struct ice_hw *hw, u8 num_qgrps,
return status;
}
+/**
+ * ice_aq_add_rdma_qsets
+ * @hw: pointer to the hardware structure
+ * @num_qset_grps: Number of RDMA Qset groups
+ * @qset_list: list of qset groups to be added
+ * @buf_size: size of buffer for indirect command
+ * @cd: pointer to command details structure or NULL
+ *
+ * Add Tx RDMA Qsets (0x0C33)
+ */
+static enum ice_status
+ice_aq_add_rdma_qsets(struct ice_hw *hw, u8 num_qset_grps,
+ struct ice_aqc_add_rdma_qset_data *qset_list,
+ u16 buf_size, struct ice_sq_cd *cd)
+{
+ struct ice_aqc_add_rdma_qset_data *list;
+ u16 i, sum_header_size, sum_q_size = 0;
+ struct ice_aqc_add_rdma_qset *cmd;
+ struct ice_aq_desc desc;
+
+ cmd = &desc.params.add_rdma_qset;
+
+ ice_fill_dflt_direct_cmd_desc(&desc, ice_aqc_opc_add_rdma_qset);
+
+ if (!qset_list)
+ return ICE_ERR_PARAM;
+
+ if (num_qset_grps > ICE_LAN_TXQ_MAX_QGRPS)
+ return ICE_ERR_PARAM;
+
+ sum_header_size = num_qset_grps *
+ (sizeof(*qset_list) - sizeof(*qset_list->rdma_qsets));
+
+ list = qset_list;
+ for (i = 0; i < num_qset_grps; i++) {
+ struct ice_aqc_add_tx_rdma_qset_entry *qset = list->rdma_qsets;
+ u16 num_qsets = le16_to_cpu(list->num_qsets);
+
+ sum_q_size += num_qsets * sizeof(*qset);
+ list = (struct ice_aqc_add_rdma_qset_data *)
+ (qset + num_qsets);
+ }
+
+ if (buf_size != (sum_header_size + sum_q_size))
+ return ICE_ERR_PARAM;
+
+ desc.flags |= cpu_to_le16(ICE_AQ_FLAG_RD);
+
+ cmd->num_qset_grps = num_qset_grps;
+
+ return ice_aq_send_cmd(hw, &desc, qset_list, buf_size, cd);
+}
+
/* End of FW Admin Queue command wrappers */
/**
@@ -3179,6 +3232,142 @@ ice_cfg_vsi_lan(struct ice_port_info *pi, u16 vsi_handle, u8 tc_bitmap,
ICE_SCHED_NODE_OWNER_LAN);
}
+/**
+ * ice_cfg_vsi_rdma - configure the VSI RDMA queues
+ * @pi: port information structure
+ * @vsi_handle: software VSI handle
+ * @tc_bitmap: TC bitmap
+ * @max_rdmaqs: max RDMA queues array per TC
+ *
+ * This function adds/updates the VSI RDMA queues per TC.
+ */
+enum ice_status
+ice_cfg_vsi_rdma(struct ice_port_info *pi, u16 vsi_handle, u8 tc_bitmap,
+ u16 *max_rdmaqs)
+{
+ return ice_cfg_vsi_qs(pi, vsi_handle, tc_bitmap, max_rdmaqs,
+ ICE_SCHED_NODE_OWNER_RDMA);
+}
+
+/**
+ * ice_ena_vsi_rdma_qset
+ * @pi: port information structure
+ * @vsi_handle: software VSI handle
+ * @tc: TC number
+ * @rdma_qset: pointer to RDMA qset
+ * @num_qsets: number of RDMA qsets
+ * @qset_teid: pointer to qset node teids
+ *
+ * This function adds RDMA qset
+ */
+enum ice_status
+ice_ena_vsi_rdma_qset(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+ u16 *rdma_qset, u16 num_qsets, u32 *qset_teid)
+{
+ struct ice_aqc_txsched_elem_data node = { 0 };
+ struct ice_aqc_add_rdma_qset_data *buf;
+ struct ice_sched_node *parent;
+ enum ice_status status;
+ struct ice_hw *hw;
+ u16 buf_size;
+ u8 i;
+
+ if (!pi || pi->port_state != ICE_SCHED_PORT_STATE_READY)
+ return ICE_ERR_CFG;
+ hw = pi->hw;
+
+ if (!ice_is_vsi_valid(hw, vsi_handle))
+ return ICE_ERR_PARAM;
+
+ buf_size = sizeof(*buf) + sizeof(*buf->rdma_qsets) * (num_qsets - 1);
+ buf = devm_kzalloc(ice_hw_to_dev(hw), buf_size, GFP_KERNEL);
+ if (!buf)
+ return ICE_ERR_NO_MEMORY;
+ mutex_lock(&pi->sched_lock);
+
+ parent = ice_sched_get_free_qparent(pi, vsi_handle, tc,
+ ICE_SCHED_NODE_OWNER_RDMA);
+ if (!parent) {
+ status = ICE_ERR_PARAM;
+ goto rdma_error_exit;
+ }
+ buf->parent_teid = parent->info.node_teid;
+ node.parent_teid = parent->info.node_teid;
+
+ buf->num_qsets = cpu_to_le16(num_qsets);
+ for (i = 0; i < num_qsets; i++) {
+ buf->rdma_qsets[i].tx_qset_id = cpu_to_le16(rdma_qset[i]);
+ buf->rdma_qsets[i].info.valid_sections =
+ ICE_AQC_ELEM_VALID_GENERIC;
+ }
+ status = ice_aq_add_rdma_qsets(hw, 1, buf, buf_size, NULL);
+ if (status) {
+ ice_debug(hw, ICE_DBG_RDMA, "add RDMA qset failed\n");
+ goto rdma_error_exit;
+ }
+ node.data.elem_type = ICE_AQC_ELEM_TYPE_LEAF;
+ for (i = 0; i < num_qsets; i++) {
+ node.node_teid = buf->rdma_qsets[i].qset_teid;
+ status = ice_sched_add_node(pi, hw->num_tx_sched_layers - 1,
+ &node);
+ if (status)
+ break;
+ qset_teid[i] = le32_to_cpu(node.node_teid);
+ }
+rdma_error_exit:
+ mutex_unlock(&pi->sched_lock);
+ devm_kfree(ice_hw_to_dev(hw), buf);
+ return status;
+}
+
+/**
+ * ice_dis_vsi_rdma_qset - free RDMA resources
+ * @pi: port_info struct
+ * @count: number of RDMA qsets to free
+ * @qset_teid: TEID of qset node
+ * @q_id: list of queue IDs being disabled
+ */
+enum ice_status
+ice_dis_vsi_rdma_qset(struct ice_port_info *pi, u16 count, u32 *qset_teid,
+ u16 *q_id)
+{
+ struct ice_aqc_dis_txq_item qg_list;
+ enum ice_status status = 0;
+ u16 qg_size;
+ int i;
+
+ if (!pi || pi->port_state != ICE_SCHED_PORT_STATE_READY)
+ return ICE_ERR_CFG;
+
+ qg_size = sizeof(qg_list);
+
+ mutex_lock(&pi->sched_lock);
+
+ for (i = 0; i < count; i++) {
+ struct ice_sched_node *node;
+
+ node = ice_sched_find_node_by_teid(pi->root, qset_teid[i]);
+ if (!node)
+ continue;
+
+ qg_list.parent_teid = node->info.parent_teid;
+ qg_list.num_qs = 1;
+ qg_list.q_id[0] =
+ cpu_to_le16(q_id[i] |
+ ICE_AQC_Q_DIS_BUF_ELEM_TYPE_RDMA_QSET);
+
+ status = ice_aq_dis_lan_txq(pi->hw, 1, &qg_list, qg_size,
+ ICE_NO_RESET, 0, NULL);
+ if (status)
+ break;
+
+ ice_free_sched_node(pi, node);
+ }
+
+ mutex_unlock(&pi->sched_lock);
+ return status;
+}
+
/**
* ice_replay_pre_init - replay pre initialization
* @hw: pointer to the HW struct
diff --git a/drivers/net/ethernet/intel/ice/ice_common.h b/drivers/net/ethernet/intel/ice/ice_common.h
index d1f8353fe6bb..f56c05119378 100644
--- a/drivers/net/ethernet/intel/ice/ice_common.h
+++ b/drivers/net/ethernet/intel/ice/ice_common.h
@@ -108,6 +108,15 @@ ice_aq_set_port_id_led(struct ice_port_info *pi, bool is_orig_mode,
struct ice_sq_cd *cd);
enum ice_status
+ice_cfg_vsi_rdma(struct ice_port_info *pi, u16 vsi_handle, u8 tc_bitmap,
+ u16 *max_rdmaqs);
+enum ice_status
+ice_ena_vsi_rdma_qset(struct ice_port_info *pi, u16 vsi_handle, u8 tc,
+ u16 *rdma_qset, u16 num_qsets, u32 *qset_teid);
+enum ice_status
+ice_dis_vsi_rdma_qset(struct ice_port_info *pi, u16 count, u32 *qset_teid,
+ u16 *q_id);
+enum ice_status
ice_dis_vsi_txq(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u8 num_queues,
u16 *q_handle, u16 *q_ids, u32 *q_teids,
enum ice_disq_rst_src rst_src, u16 vmvf_num,
diff --git a/drivers/net/ethernet/intel/ice/ice_dcb_lib.c b/drivers/net/ethernet/intel/ice/ice_dcb_lib.c
index c6466e16eef1..ded039a1e3e7 100644
--- a/drivers/net/ethernet/intel/ice/ice_dcb_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_dcb_lib.c
@@ -106,6 +106,7 @@ void ice_vsi_cfg_dcb_rings(struct ice_vsi *vsi)
static void ice_pf_dcb_recfg(struct ice_pf *pf)
{
struct ice_dcbx_cfg *dcbcfg = &pf->hw.port_info->local_dcbx_cfg;
+ struct ice_event *event;
u8 tc_map = 0;
int v, ret;
@@ -129,6 +130,35 @@ static void ice_pf_dcb_recfg(struct ice_pf *pf)
ice_vsi_map_rings_to_vectors(pf->vsi[v]);
}
+ event = devm_kzalloc(&pf->pdev->dev, sizeof(*event), GFP_KERNEL);
+ if (!event)
+ return;
+
+ set_bit(ICE_EVENT_TC_CHANGE, event->type);
+ event->reporter = NULL;
+ ice_setup_dcb_qos_info(pf, &event->info.port_qos);
+ ice_for_each_peer(pf, event, ice_peer_check_for_reg);
+ devm_kfree(&pf->pdev->dev, event);
+}
+
+/**
+ * ice_peer_prep_tc_change - Pre-notify RDMA Peer in blocking call of TC change
+ * @peer_dev_int: ptr to peer device internal struct
+ * @data: ptr to opaque data
+ */
+static int
+ice_peer_prep_tc_change(struct ice_peer_dev_int *peer_dev_int,
+ void __always_unused *data)
+{
+ struct ice_peer_dev *peer_dev = &peer_dev_int->peer_dev;
+
+ if (!ice_validate_peer_dev(peer_dev))
+ return 0;
+
+ if (peer_dev->peer_ops && peer_dev->peer_ops->prep_tc_change)
+ peer_dev->peer_ops->prep_tc_change(peer_dev);
+
+ return 0;
}
/**
@@ -160,6 +190,9 @@ int ice_pf_dcb_cfg(struct ice_pf *pf, struct ice_dcbx_cfg *new_cfg, bool locked)
return ret;
}
+ /* Notify capable peers about impending change to TCs */
+ ice_for_each_peer(pf, NULL, ice_peer_prep_tc_change);
+
/* Store old config in case FW config fails */
old_cfg = devm_kzalloc(&pf->pdev->dev, sizeof(*old_cfg), GFP_KERNEL);
memcpy(old_cfg, curr_cfg, sizeof(*old_cfg));
diff --git a/drivers/net/ethernet/intel/ice/ice_idc.c b/drivers/net/ethernet/intel/ice/ice_idc.c
index 0bf66e9fa159..550c3839b581 100644
--- a/drivers/net/ethernet/intel/ice/ice_idc.c
+++ b/drivers/net/ethernet/intel/ice/ice_idc.c
@@ -10,6 +10,44 @@ DEFINE_IDA(ice_peer_index_ida);
static const struct peer_dev_id peer_dev_ids[] = ASSIGN_PEER_INFO;
+/**
+ * ice_validate_peer_dev - validate peer device state
+ * @peer_dev: ptr to peer device
+ *
+ * This helper function checks if PF is in a minimal state and if the peer
+ * device is valid. This should be called before engaging in peer operations.
+ *
+ * Returns true if the peer device is valid, false otherwise.
+ */
+#ifndef CONFIG_DCB
+static
+#endif
+bool ice_validate_peer_dev(struct ice_peer_dev *peer_dev)
+{
+ struct ice_peer_dev_int *peer_dev_int;
+ struct ice_pf *pf;
+
+ if (!peer_dev || !peer_dev->pdev)
+ return false;
+
+ if (!peer_dev->peer_ops)
+ return false;
+
+ pf = pci_get_drvdata(peer_dev->pdev);
+ if (!pf)
+ return false;
+
+ peer_dev_int = peer_to_ice_dev_int(peer_dev);
+ if (!peer_dev_int)
+ return false;
+
+ if (test_bit(ICE_PEER_DEV_STATE_REMOVED, peer_dev_int->state) ||
+ test_bit(ICE_PEER_DEV_STATE_INIT, peer_dev_int->state))
+ return false;
+
+ return true;
+}
+
/**
* ice_peer_state_change - manage state machine for peer
* @peer_dev: pointer to peer's configuration
@@ -155,6 +193,155 @@ ice_peer_state_change(struct ice_peer_dev_int *peer_dev, long new_state,
mutex_unlock(&peer_dev->peer_dev_state_mutex);
}
+/**
+ * ice_peer_close - close a peer device
+ * @peer_dev_int: device to close
+ * @data: pointer to opaque data
+ *
+ * This function will also set the state bit for the peer to CLOSED. This
+ * function is meant to be called from a ice_for_each_peer().
+ */
+int ice_peer_close(struct ice_peer_dev_int *peer_dev_int, void *data)
+{
+ enum ice_close_reason reason = *(enum ice_close_reason *)(data);
+ struct ice_peer_dev *peer_dev = &peer_dev_int->peer_dev;
+ struct ice_pf *pf;
+ int i;
+
+ /* return 0 so ice_for_each_peer will continue closing other peers */
+ if (!ice_validate_peer_dev(peer_dev))
+ return 0;
+ pf = pci_get_drvdata(peer_dev->pdev);
+
+ if (test_bit(__ICE_DOWN, pf->state) ||
+ test_bit(__ICE_SUSPENDED, pf->state) ||
+ test_bit(__ICE_NEEDS_RESTART, pf->state))
+ return 0;
+
+ mutex_lock(&peer_dev_int->peer_dev_state_mutex);
+
+ /* no peer driver, already closed, closing or opening nothing to do */
+ if (test_bit(ICE_PEER_DEV_STATE_CLOSED, peer_dev_int->state) ||
+ test_bit(ICE_PEER_DEV_STATE_CLOSING, peer_dev_int->state) ||
+ test_bit(ICE_PEER_DEV_STATE_OPENING, peer_dev_int->state) ||
+ test_bit(ICE_PEER_DEV_STATE_REMOVED, peer_dev_int->state))
+ goto peer_close_out;
+
+ /* Set the peer state to CLOSING */
+ ice_peer_state_change(peer_dev_int, ICE_PEER_DEV_STATE_CLOSING, true);
+
+ for (i = 0; i < ICE_EVENT_NBITS; i++)
+ bitmap_zero(peer_dev_int->current_events[i].type,
+ ICE_EVENT_NBITS);
+
+ if (peer_dev->peer_ops && peer_dev->peer_ops->close)
+ peer_dev->peer_ops->close(peer_dev, reason);
+
+ /* Set the peer state to CLOSED */
+ ice_peer_state_change(peer_dev_int, ICE_PEER_DEV_STATE_CLOSED, true);
+
+peer_close_out:
+ mutex_unlock(&peer_dev_int->peer_dev_state_mutex);
+
+ return 0;
+}
+
+/**
+ * ice_close_peer_for_reset - queue work to close peer for reset
+ * @peer_dev_int: pointer peer dev internal struct
+ * @data: pointer to opaque data used for reset type
+ */
+int ice_close_peer_for_reset(struct ice_peer_dev_int *peer_dev_int, void *data)
+{
+ struct ice_peer_dev *peer_dev = &peer_dev_int->peer_dev;
+ enum ice_reset_req reset;
+
+ if (!ice_validate_peer_dev(peer_dev))
+ return 0;
+
+ reset = *(enum ice_reset_req *)data;
+
+ switch (reset) {
+ case ICE_RESET_GLOBR:
+ peer_dev_int->rst_type = ICE_REASON_GLOBR_REQ;
+ break;
+ case ICE_RESET_CORER:
+ peer_dev_int->rst_type = ICE_REASON_CORER_REQ;
+ break;
+ case ICE_RESET_PFR:
+ peer_dev_int->rst_type = ICE_REASON_PFR_REQ;
+ break;
+ default:
+ /* reset type is invalid */
+ return 1;
+ }
+ queue_work(peer_dev_int->ice_peer_wq, &peer_dev_int->peer_close_task);
+ return 0;
+}
+
+/**
+ * ice_check_peer_drv_for_events - check peer_drv for events to report
+ * @peer_dev: peer device to report to
+ */
+static void ice_check_peer_drv_for_events(struct ice_peer_dev *peer_dev)
+{
+ const struct ice_peer_ops *p_ops = peer_dev->peer_ops;
+ struct ice_peer_dev_int *peer_dev_int;
+ struct ice_peer_drv_int *peer_drv_int;
+ int i;
+
+ peer_dev_int = peer_to_ice_dev_int(peer_dev);
+ peer_drv_int = peer_dev_int->peer_drv_int;
+
+ for_each_set_bit(i, peer_dev_int->events, ICE_EVENT_NBITS) {
+ struct ice_event *curr = &peer_drv_int->current_events[i];
+
+ if (!bitmap_empty(curr->type, ICE_EVENT_NBITS) &&
+ p_ops->event_handler)
+ p_ops->event_handler(peer_dev, curr);
+ }
+}
+
+/**
+ * ice_check_peer_for_events - check peer_devs for events new peer reg'd for
+ * @src_peer_int: peer to check for events
+ * @data: ptr to opaque data, to be used for the peer struct that opened
+ *
+ * This function is to be called when a peer device is opened.
+ *
+ * Since a new peer opening would have missed any events that would
+ * have happened before its opening, we need to walk the peers and see
+ * if any of them have events that the new peer cares about
+ *
+ * This function is meant to be called by a device_for_each_child.
+ */
+static int
+ice_check_peer_for_events(struct ice_peer_dev_int *src_peer_int, void *data)
+{
+ struct ice_peer_dev *new_peer = (struct ice_peer_dev *)data;
+ struct ice_peer_dev *src_peer = &src_peer_int->peer_dev;
+ const struct ice_peer_ops *p_ops = new_peer->peer_ops;
+ struct ice_peer_dev_int *new_peer_int;
+ int i;
+
+ if (!ice_validate_peer_dev(new_peer) ||
+ !ice_validate_peer_dev(src_peer))
+ return 0;
+
+ new_peer_int = peer_to_ice_dev_int(new_peer);
+
+ for_each_set_bit(i, new_peer_int->events, ICE_EVENT_NBITS) {
+ struct ice_event *curr = &src_peer_int->current_events[i];
+
+ if (!bitmap_empty(curr->type, ICE_EVENT_NBITS) &&
+ new_peer->index != src_peer->index &&
+ p_ops->event_handler)
+ p_ops->event_handler(new_peer, curr);
+ }
+
+ return 0;
+}
+
/**
* ice_for_each_peer - iterate across and call function for each peer dev
* @pf: pointer to private board struct
@@ -184,6 +371,88 @@ ice_for_each_peer(struct ice_pf *pf, void *data,
return 0;
}
+/**
+ * ice_finish_init_peer_device - complete peer device initialization
+ * @peer_dev_int: ptr to peer device internal struct
+ * @data: ptr to opaque data
+ *
+ * This function completes remaining initialization of peer_devices
+ */
+int
+ice_finish_init_peer_device(struct ice_peer_dev_int *peer_dev_int,
+ void __always_unused *data)
+{
+ struct ice_peer_dev *peer_dev = &peer_dev_int->peer_dev;
+ struct ice_peer_drv *peer_drv;
+ struct ice_pf *pf;
+ int ret = 0;
+
+ /* peer_dev will not always be populated at the time of this check */
+ if (!ice_validate_peer_dev(peer_dev))
+ return ret;
+
+ peer_drv = peer_dev->peer_drv;
+ pf = pci_get_drvdata(peer_dev->pdev);
+ /* There will be several assessments of the peer_dev's state in this
+ * chunk of logic. We need to hold the peer_dev_int's state mutex
+ * for the entire part so that the flow progresses without another
+ * context changing things mid-flow
+ */
+ mutex_lock(&peer_dev_int->peer_dev_state_mutex);
+
+ if (!peer_dev->peer_ops) {
+ dev_err(&pf->pdev->dev,
+ "peer_ops not defined on peer dev\n");
+ goto init_unlock;
+ }
+
+ if (!peer_dev->peer_ops->open) {
+ dev_err(&pf->pdev->dev,
+ "peer_ops:open not defined on peer dev\n");
+ goto init_unlock;
+ }
+
+ if (!peer_dev->peer_ops->close) {
+ dev_err(&pf->pdev->dev,
+ "peer_ops:close not defined on peer dev\n");
+ goto init_unlock;
+ }
+
+ /* Peer driver expected to set driver_id during registration */
+ if (!peer_drv->driver_id) {
+ dev_err(&pf->pdev->dev,
+ "Peer driver did not set driver_id\n");
+ goto init_unlock;
+ }
+
+ if ((test_bit(ICE_PEER_DEV_STATE_CLOSED, peer_dev_int->state) ||
+ test_bit(ICE_PEER_DEV_STATE_PROBED, peer_dev_int->state)) &&
+ ice_pf_state_is_nominal(pf)) {
+ ice_peer_state_change(peer_dev_int, ICE_PEER_DEV_STATE_OPENING,
+ true);
+ ret = peer_dev->peer_ops->open(peer_dev);
+ if (ret) {
+ dev_err(&pf->pdev->dev,
+ "Peer %d failed to open\n",
+ peer_dev->peer_dev_id);
+ ice_peer_state_change(peer_dev_int,
+ ICE_PEER_DEV_STATE_PROBED, true);
+ goto init_unlock;
+ }
+
+ ice_peer_state_change(peer_dev_int, ICE_PEER_DEV_STATE_OPENED,
+ true);
+ ret = ice_for_each_peer(pf, peer_dev,
+ ice_check_peer_for_events);
+ ice_check_peer_drv_for_events(peer_dev);
+ }
+
+init_unlock:
+ mutex_unlock(&peer_dev_int->peer_dev_state_mutex);
+
+ return ret;
+}
+
/**
* ice_unreg_peer_device - unregister specified device
* @peer_dev_int: ptr to peer device internal
@@ -196,6 +465,7 @@ int
ice_unreg_peer_device(struct ice_peer_dev_int *peer_dev_int,
void __always_unused *data)
{
+ struct ice_peer_drv_int *peer_drv_int;
struct ice_pf *pf;
if (!peer_dev_int)
@@ -207,6 +477,8 @@ ice_unreg_peer_device(struct ice_peer_dev_int *peer_dev_int,
if (!pf)
return 0;
+ peer_drv_int = peer_dev_int->peer_drv_int;
+
if (peer_dev_int->ice_peer_wq) {
if (peer_dev_int->peer_prep_task.func)
cancel_work_sync(&peer_dev_int->peer_prep_task);
@@ -219,6 +491,8 @@ ice_unreg_peer_device(struct ice_peer_dev_int *peer_dev_int,
/* Cleanup the allocated ID for this peer device */
ida_simple_remove(&ice_peer_index_ida, peer_dev_int->peer_dev.index);
+ devm_kfree(&pf->pdev->dev, peer_drv_int);
+
devm_kfree(&pf->pdev->dev, peer_dev_int);
return 0;
@@ -248,6 +522,607 @@ ice_unroll_peer(struct ice_peer_dev_int *peer_dev_int,
return 0;
}
+/**
+ * ice_find_vsi - Find the VSI from VSI ID
+ * @pf: The PF pointer to search in
+ * @vsi_num: The VSI ID to search for
+ */
+static struct ice_vsi *ice_find_vsi(struct ice_pf *pf, u16 vsi_num)
+{
+ int i;
+
+ ice_for_each_vsi(pf, i)
+ if (pf->vsi[i] && pf->vsi[i]->vsi_num == vsi_num)
+ return pf->vsi[i];
+ return NULL;
+}
+
+/**
+ * ice_peer_alloc_rdma_qsets - Allocate Leaf Nodes for RDMA Qset
+ * @peer_dev: peer that is requesting the Leaf Nodes
+ * @res: Resources to be allocated
+ * @partial_acceptable: If partial allocation is acceptable to the peer
+ *
+ * This function allocates Leaf Nodes for given RDMA Qset resources
+ * for the peer device.
+ */
+static int
+ice_peer_alloc_rdma_qsets(struct ice_peer_dev *peer_dev, struct ice_res *res,
+ int __maybe_unused partial_acceptable)
+{
+ u16 max_rdmaqs[ICE_MAX_TRAFFIC_CLASS];
+ enum ice_status status;
+ struct ice_vsi *vsi;
+ struct ice_pf *pf;
+ int i, ret = 0;
+ u32 *qset_teid;
+ u16 *qs_handle;
+
+ if (!ice_validate_peer_dev(peer_dev) || !res)
+ return -EINVAL;
+
+ pf = pci_get_drvdata(peer_dev->pdev);
+
+ if (res->cnt_req > ICE_MAX_TXQ_PER_TXQG)
+ return -EINVAL;
+
+ qset_teid = devm_kcalloc(&pf->pdev->dev, res->cnt_req,
+ sizeof(*qset_teid), GFP_KERNEL);
+ if (!qset_teid)
+ return -ENOMEM;
+
+ qs_handle = devm_kcalloc(&pf->pdev->dev, res->cnt_req,
+ sizeof(*qs_handle), GFP_KERNEL);
+ if (!qs_handle) {
+ devm_kfree(&pf->pdev->dev, qset_teid);
+ return -ENOMEM;
+ }
+
+ ice_for_each_traffic_class(i)
+ max_rdmaqs[i] = 0;
+
+ for (i = 0; i < res->cnt_req; i++) {
+ struct ice_rdma_qset_params *qset;
+
+ qset = &res->res[i].res.qsets;
+ if (qset->vsi_id != peer_dev->pf_vsi_num) {
+ dev_err(&pf->pdev->dev,
+ "RDMA QSet invalid VSI requested\n");
+ ret = -EINVAL;
+ goto out;
+ }
+ max_rdmaqs[qset->tc]++;
+ qs_handle[i] = qset->qs_handle;
+ }
+
+ vsi = ice_find_vsi(pf, peer_dev->pf_vsi_num);
+ if (!vsi) {
+ dev_err(&pf->pdev->dev, "RDMA QSet invalid VSI\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
+ status = ice_cfg_vsi_rdma(vsi->port_info, vsi->idx, vsi->tc_cfg.ena_tc,
+ max_rdmaqs);
+ if (status) {
+ dev_err(&pf->pdev->dev, "Failed VSI RDMA qset config\n");
+ ret = -EINVAL;
+ goto out;
+ }
+
+ for (i = 0; i < res->cnt_req; i++) {
+ struct ice_rdma_qset_params *qset;
+
+ qset = &res->res[i].res.qsets;
+ status = ice_ena_vsi_rdma_qset(vsi->port_info, vsi->idx,
+ qset->tc, &qs_handle[i], 1,
+ &qset_teid[i]);
+ if (status) {
+ dev_err(&pf->pdev->dev,
+ "Failed VSI RDMA qset enable\n");
+ ret = -EINVAL;
+ goto out;
+ }
+ vsi->qset_handle[qset->tc] = qset->qs_handle;
+ qset->teid = qset_teid[i];
+ }
+
+out:
+ devm_kfree(&pf->pdev->dev, qset_teid);
+ devm_kfree(&pf->pdev->dev, qs_handle);
+ return ret;
+}
+
+/**
+ * ice_peer_free_rdma_qsets - Free leaf nodes for RDMA Qset
+ * @peer_dev: peer that requested qsets to be freed
+ * @res: Resource to be freed
+ */
+static int
+ice_peer_free_rdma_qsets(struct ice_peer_dev *peer_dev, struct ice_res *res)
+{
+ enum ice_status status;
+ int count, i, ret = 0;
+ struct ice_vsi *vsi;
+ struct ice_pf *pf;
+ u16 vsi_id;
+ u32 *teid;
+ u16 *q_id;
+
+ if (!ice_validate_peer_dev(peer_dev) || !res)
+ return -EINVAL;
+
+ pf = pci_get_drvdata(peer_dev->pdev);
+
+ count = res->res_allocated;
+ if (count > ICE_MAX_TXQ_PER_TXQG)
+ return -EINVAL;
+
+ teid = devm_kcalloc(&pf->pdev->dev, count, sizeof(*teid), GFP_KERNEL);
+ if (!teid)
+ return -ENOMEM;
+
+ q_id = devm_kcalloc(&pf->pdev->dev, count, sizeof(*q_id), GFP_KERNEL);
+ if (!q_id) {
+ devm_kfree(&pf->pdev->dev, teid);
+ return -ENOMEM;
+ }
+
+ vsi_id = res->res[0].res.qsets.vsi_id;
+ vsi = ice_find_vsi(pf, vsi_id);
+ if (!vsi) {
+ dev_err(&pf->pdev->dev, "RDMA Invalid VSI\n");
+ ret = -EINVAL;
+ goto rdma_free_out;
+ }
+
+ for (i = 0; i < count; i++) {
+ struct ice_rdma_qset_params *qset;
+
+ qset = &res->res[i].res.qsets;
+ if (qset->vsi_id != vsi_id) {
+ dev_err(&pf->pdev->dev, "RDMA Invalid VSI ID\n");
+ ret = -EINVAL;
+ goto rdma_free_out;
+ }
+ q_id[i] = qset->qs_handle;
+ teid[i] = qset->teid;
+
+ vsi->qset_handle[qset->tc] = 0;
+ }
+
+ status = ice_dis_vsi_rdma_qset(vsi->port_info, count, teid, q_id);
+ if (status)
+ ret = -EINVAL;
+
+rdma_free_out:
+ devm_kfree(&pf->pdev->dev, teid);
+ devm_kfree(&pf->pdev->dev, q_id);
+
+ return ret;
+}
+
+/**
+ * ice_peer_alloc_res - Allocate requested resources for peer device
+ * @peer_dev: peer that is requesting resources
+ * @res: Resources to be allocated
+ * @partial_acceptable: If partial allocation is acceptable to the peer
+ *
+ * This function allocates requested resources for the peer device.
+ */
+static int
+ice_peer_alloc_res(struct ice_peer_dev *peer_dev, struct ice_res *res,
+ int partial_acceptable)
+{
+ struct ice_pf *pf;
+ int ret;
+
+ if (!ice_validate_peer_dev(peer_dev) || !res)
+ return -EINVAL;
+
+ pf = pci_get_drvdata(peer_dev->pdev);
+ if (!ice_pf_state_is_nominal(pf))
+ return -EBUSY;
+
+ switch (res->res_type) {
+ case ICE_RDMA_QSETS_TXSCHED:
+ ret = ice_peer_alloc_rdma_qsets(peer_dev, res,
+ partial_acceptable);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
+/**
+ * ice_peer_free_res - Free given resources
+ * @peer_dev: peer that is requesting freeing of resources
+ * @res: Resources to be freed
+ *
+ * Free/Release resources allocated to given peer device.
+ */
+static int
+ice_peer_free_res(struct ice_peer_dev *peer_dev, struct ice_res *res)
+{
+ int ret;
+
+ if (!ice_validate_peer_dev(peer_dev) || !res)
+ return -EINVAL;
+
+ switch (res->res_type) {
+ case ICE_RDMA_QSETS_TXSCHED:
+ ret = ice_peer_free_rdma_qsets(peer_dev, res);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
+/**
+ * ice_peer_reg_for_notif - register a peer to receive specific notifications
+ * @peer_dev: peer that is registering for event notifications
+ * @events: mask of event types peer is registering for
+ */
+static void
+ice_peer_reg_for_notif(struct ice_peer_dev *peer_dev, struct ice_event *events)
+{
+ struct ice_peer_dev_int *peer_dev_int;
+ struct ice_pf *pf;
+
+ if (!ice_validate_peer_dev(peer_dev) || !events)
+ return;
+
+ peer_dev_int = peer_to_ice_dev_int(peer_dev);
+ pf = pci_get_drvdata(peer_dev->pdev);
+
+ bitmap_or(peer_dev_int->events, peer_dev_int->events, events->type,
+ ICE_EVENT_NBITS);
+
+ /* Check to see if any events happened previous to peer registering */
+ ice_for_each_peer(pf, peer_dev, ice_check_peer_for_events);
+ ice_check_peer_drv_for_events(peer_dev);
+}
+
+/**
+ * ice_peer_unreg_for_notif - unreg a peer from receiving certain notifications
+ * @peer_dev: peer that is unregistering from event notifications
+ * @events: mask of event types peer is unregistering for
+ */
+static void
+ice_peer_unreg_for_notif(struct ice_peer_dev *peer_dev,
+ struct ice_event *events)
+{
+ struct ice_peer_dev_int *peer_dev_int;
+
+ if (!ice_validate_peer_dev(peer_dev) || !events)
+ return;
+
+ peer_dev_int = peer_to_ice_dev_int(peer_dev);
+
+ bitmap_andnot(peer_dev_int->events, peer_dev_int->events, events->type,
+ ICE_EVENT_NBITS);
+}
+
+/**
+ * ice_peer_check_for_reg - check to see if any peers are reg'd for event
+ * @peer_dev_int: ptr to peer device internal struct
+ * @data: ptr to opaque data, to be used for ice_event to report
+ *
+ * This function is to be called by device_for_each_child to handle an
+ * event reported by a peer or the ice driver.
+ */
+int ice_peer_check_for_reg(struct ice_peer_dev_int *peer_dev_int, void *data)
+{
+ struct ice_event *event = (struct ice_event *)data;
+ DECLARE_BITMAP(comp_events, ICE_EVENT_NBITS);
+ struct ice_peer_dev *peer_dev;
+ bool check = true;
+
+ peer_dev = &peer_dev_int->peer_dev;
+
+ if (!ice_validate_peer_dev(peer_dev) || !data)
+ /* If invalid dev, in this case return 0 instead of error
+ * because caller ignores this return value
+ */
+ return 0;
+
+ if (event->reporter)
+ check = event->reporter->index != peer_dev->index;
+
+ if (bitmap_and(comp_events, event->type, peer_dev_int->events,
+ ICE_EVENT_NBITS) &&
+ (test_bit(ICE_PEER_DEV_STATE_OPENED, peer_dev_int->state) ||
+ test_bit(ICE_PEER_DEV_STATE_PREP_RST, peer_dev_int->state) ||
+ test_bit(ICE_PEER_DEV_STATE_PREPPED, peer_dev_int->state)) &&
+ check &&
+ peer_dev->peer_ops->event_handler)
+ peer_dev->peer_ops->event_handler(peer_dev, event);
+
+ return 0;
+}
+
+/**
+ * ice_peer_report_state_change - accept report of a peer state change
+ * @peer_dev: peer that is sending notification about state change
+ * @event: ice_event holding info on what the state change is
+ *
+ * We also need to parse the list of peers to see if anyone is registered
+ * for notifications about this state change event, and if so, notify them.
+ */
+static void
+ice_peer_report_state_change(struct ice_peer_dev *peer_dev,
+ struct ice_event *event)
+{
+ struct ice_peer_dev_int *peer_dev_int;
+ struct ice_peer_drv_int *peer_drv_int;
+ int e_type, drv_event = 0;
+ struct ice_pf *pf;
+
+ if (!ice_validate_peer_dev(peer_dev) || !event)
+ return;
+
+ pf = pci_get_drvdata(peer_dev->pdev);
+ peer_dev_int = peer_to_ice_dev_int(peer_dev);
+ peer_drv_int = peer_dev_int->peer_drv_int;
+
+ e_type = find_first_bit(event->type, ICE_EVENT_NBITS);
+ if (!e_type)
+ return;
+
+ switch (e_type) {
+ /* Check for peer_drv events */
+ case ICE_EVENT_MBX_CHANGE:
+ drv_event = 1;
+ if (event->info.mbx_rdy)
+ set_bit(ICE_PEER_DRV_STATE_MBX_RDY,
+ peer_drv_int->state);
+ else
+ clear_bit(ICE_PEER_DRV_STATE_MBX_RDY,
+ peer_drv_int->state);
+ break;
+
+ /* Check for peer_dev events */
+ case ICE_EVENT_API_CHANGE:
+ if (event->info.api_rdy)
+ set_bit(ICE_PEER_DEV_STATE_API_RDY,
+ peer_dev_int->state);
+ else
+ clear_bit(ICE_PEER_DEV_STATE_API_RDY,
+ peer_dev_int->state);
+ break;
+
+ default:
+ return;
+ }
+
+ /* store the event and state to notify any new peers opening */
+ if (drv_event)
+ memcpy(&peer_drv_int->current_events[e_type], event,
+ sizeof(*event));
+ else
+ memcpy(&peer_dev_int->current_events[e_type], event,
+ sizeof(*event));
+
+ ice_for_each_peer(pf, event, ice_peer_check_for_reg);
+}
+
+/**
+ * ice_peer_unregister - request to unregister peer
+ * @peer_dev: peer device
+ *
+ * This function triggers close/remove on peer_dev allowing peer
+ * to unregister.
+ */
+static int ice_peer_unregister(struct ice_peer_dev *peer_dev)
+{
+ enum ice_close_reason reason = ICE_REASON_PEER_DEV_UNINIT;
+ struct ice_peer_dev_int *peer_dev_int;
+ struct ice_pf *pf;
+ int ret;
+
+ if (!ice_validate_peer_dev(peer_dev))
+ return -EINVAL;
+
+ pf = pci_get_drvdata(peer_dev->pdev);
+ if (ice_is_reset_in_progress(pf->state))
+ return -EBUSY;
+
+ peer_dev_int = peer_to_ice_dev_int(peer_dev);
+
+ ret = ice_peer_close(peer_dev_int, &reason);
+ if (ret)
+ return ret;
+
+ peer_dev->peer_ops = NULL;
+
+ ice_peer_state_change(peer_dev_int, ICE_PEER_DEV_STATE_REMOVED, false);
+
+ return ret;
+}
+
+/**
+ * ice_peer_register - Called by peer to open communication with LAN
+ * @peer_dev: ptr to peer device
+ *
+ * registering peer is expected to populate the ice_peerdrv->name field
+ * before calling this function.
+ */
+static int ice_peer_register(struct ice_peer_dev *peer_dev)
+{
+ struct ice_peer_drv_int *peer_drv_int;
+ struct ice_peer_dev_int *peer_dev_int;
+ struct ice_peer_drv *peer_drv;
+
+ if (!peer_dev) {
+ pr_err("Failed to reg peer dev: peer_dev ptr NULL\n");
+ return -EINVAL;
+ }
+
+ if (!peer_dev->pdev) {
+ pr_err("Failed to reg peer dev: peer dev pdev NULL\n");
+ return -EINVAL;
+ }
+
+ if (!peer_dev->peer_ops || !peer_dev->ops) {
+ pr_err("Failed to reg peer dev: peer dev peer_ops/ops NULL\n");
+ return -EINVAL;
+ }
+
+ peer_drv = peer_dev->peer_drv;
+ if (!peer_drv) {
+ pr_err("Failed to reg peer dev: peer drv NULL\n");
+ return -EINVAL;
+ }
+
+ if (peer_drv->ver.major != ICE_PEER_MAJOR_VER ||
+ peer_drv->ver.minor != ICE_PEER_MINOR_VER) {
+ pr_err("failed to register due to version mismatch:\n");
+ pr_err("expected major ver %d, caller specified major ver %d\n",
+ ICE_PEER_MAJOR_VER, peer_drv->ver.major);
+ pr_err("expected minor ver %d, caller specified minor ver %d\n",
+ ICE_PEER_MINOR_VER, peer_drv->ver.minor);
+ return -EINVAL;
+ }
+
+ peer_dev_int = peer_to_ice_dev_int(peer_dev);
+ peer_drv_int = peer_dev_int->peer_drv_int;
+ if (!peer_drv_int) {
+ pr_err("Failed to match peer_drv_int to peer_dev\n");
+ return -EINVAL;
+ }
+
+ peer_drv_int->peer_drv = peer_drv;
+
+ ice_peer_state_change(peer_dev_int, ICE_PEER_DEV_STATE_PROBED, false);
+
+ return 0;
+}
+
+/**
+ * ice_peer_request_reset - accept request from peer to perform a reset
+ * @peer_dev: peer device that is request a reset
+ * @reset_type: type of reset the peer is requesting
+ */
+static int
+ice_peer_request_reset(struct ice_peer_dev *peer_dev,
+ enum ice_peer_reset_type reset_type)
+{
+ enum ice_reset_req reset;
+ struct ice_pf *pf;
+
+ if (!ice_validate_peer_dev(peer_dev))
+ return -EINVAL;
+
+ pf = pci_get_drvdata(peer_dev->pdev);
+
+ switch (reset_type) {
+ case ICE_PEER_PFR:
+ reset = ICE_RESET_PFR;
+ break;
+ case ICE_PEER_CORER:
+ reset = ICE_RESET_CORER;
+ break;
+ case ICE_PEER_GLOBR:
+ reset = ICE_RESET_GLOBR;
+ break;
+ default:
+ dev_err(&pf->pdev->dev, "incorrect reset request from peer\n");
+ return -EINVAL;
+ }
+
+ return ice_schedule_reset(pf, reset);
+}
+
+/**
+ * ice_peer_is_vsi_ready - query if VSI in nominal state
+ * @peer_dev: pointer to ice_peer_dev struct
+ */
+static int ice_peer_is_vsi_ready(struct ice_peer_dev *peer_dev)
+{
+ DECLARE_BITMAP(check_bits, __ICE_STATE_NBITS) = { 0 };
+ struct ice_netdev_priv *np;
+ struct ice_vsi *vsi;
+
+ /* If the peer_dev or associated values are not valid, then return
+ * 0 as there is no ready port associated with the values passed in
+ * as parameters.
+ */
+
+ if (!ice_validate_peer_dev(peer_dev))
+ return 0;
+
+ if (!peer_dev->netdev)
+ return 0;
+
+ np = netdev_priv(peer_dev->netdev);
+ vsi = np->vsi;
+ if (!vsi)
+ return 0;
+
+ bitmap_set(check_bits, 0, __ICE_STATE_NOMINAL_CHECK_BITS);
+ if (bitmap_intersects(vsi->state, check_bits, __ICE_STATE_NBITS))
+ return 0;
+
+ return 1;
+}
+
+/**
+ * ice_peer_update_vsi_filter - update filters for RDMA VSI
+ * @peer_dev: pointer to RDMA peer device
+ * @filter: selection of filters to enable or disable
+ * @enable: bool whether to enable or disable filters
+ */
+static int
+ice_peer_update_vsi_filter(struct ice_peer_dev *peer_dev,
+ enum ice_rdma_filter __maybe_unused filter,
+ bool enable)
+{
+ struct ice_pf *pf;
+ int ret, v;
+ u16 idx;
+
+ if (!ice_validate_peer_dev(peer_dev))
+ return -EINVAL;
+
+ pf = pci_get_drvdata(peer_dev->pdev);
+
+ ice_for_each_vsi(pf, v)
+ if (peer_dev->pf_vsi_num == pf->vsi[v]->vsi_num) {
+ idx = pf->vsi[v]->idx;
+ break;
+ }
+ if (v >= pf->num_alloc_vsi)
+ return -EINVAL;
+
+ ret = ice_cfg_iwarp_fltr(&pf->hw, idx, enable);
+
+ if (ret)
+ dev_err(&pf->pdev->dev, "Failed to %sable iWARP filtering\n",
+ enable ? "en" : "dis");
+
+ return ret;
+}
+
+/* Initialize the ice_ops struct, which is used in 'ice_init_peer_devices' */
+static const struct ice_ops ops = {
+ .alloc_res = ice_peer_alloc_res,
+ .free_res = ice_peer_free_res,
+ .is_vsi_ready = ice_peer_is_vsi_ready,
+ .reg_for_notification = ice_peer_reg_for_notif,
+ .unreg_for_notification = ice_peer_unreg_for_notif,
+ .notify_state_change = ice_peer_report_state_change,
+ .request_reset = ice_peer_request_reset,
+ .peer_register = ice_peer_register,
+ .peer_unregister = ice_peer_unregister,
+ .update_vsi_filter = ice_peer_update_vsi_filter,
+};
+
/**
* ice_reserve_peer_qvector - Reserve vector resources for peer drivers
* @pf: board private structure to initialize
@@ -267,6 +1142,41 @@ static int ice_reserve_peer_qvector(struct ice_pf *pf)
return 0;
}
+/**
+ * ice_peer_close_task - call peer's close asynchronously
+ * @work: pointer to work_struct contained by the peer_dev_int struct
+ *
+ * This method (asynchronous) of calling a peer's close function is
+ * meant to be used in the reset path.
+ */
+static void ice_peer_close_task(struct work_struct *work)
+{
+ struct ice_peer_dev_int *peer_dev_int;
+ struct ice_peer_dev *peer_dev;
+
+ peer_dev_int = container_of(work, struct ice_peer_dev_int,
+ peer_close_task);
+
+ peer_dev = &peer_dev_int->peer_dev;
+ if (!peer_dev || !peer_dev->peer_ops)
+ return;
+
+ /* If this peer_dev is going to close, we do not want any state changes
+ * to happen until after we successfully finish or abort the close.
+ * Grab the peer_dev_state_mutex to protect this flow
+ */
+ mutex_lock(&peer_dev_int->peer_dev_state_mutex);
+
+ ice_peer_state_change(peer_dev_int, ICE_PEER_DEV_STATE_CLOSING, true);
+
+ if (peer_dev->peer_ops->close)
+ peer_dev->peer_ops->close(peer_dev, peer_dev_int->rst_type);
+
+ ice_peer_state_change(peer_dev_int, ICE_PEER_DEV_STATE_CLOSED, true);
+
+ mutex_unlock(&peer_dev_int->peer_dev_state_mutex);
+}
+
/**
* ice_peer_device_release - release function for platform peer device
* @dev: pointer to device structure
@@ -313,6 +1223,7 @@ int ice_init_peer_devices(struct ice_pf *pf)
}
for (i = 0; i < ARRAY_SIZE(peer_dev_ids); i++) {
struct ice_peer_dev_int *peer_dev_int;
+ struct ice_peer_drv_int *peer_drv_int;
struct platform_device *platform_dev;
struct ice_qos_params *qos_info;
struct ice_peer_dev *peer_dev;
@@ -334,6 +1245,19 @@ int ice_init_peer_devices(struct ice_pf *pf)
return -ENOMEM;
pf->peers[i] = peer_dev_int;
+ peer_drv_int = devm_kzalloc(&pdev->dev, sizeof(*peer_drv_int),
+ GFP_KERNEL);
+ if (!peer_drv_int)
+ return -ENOMEM;
+
+ peer_drv_int->peer_dev = &peer_dev_int->peer_dev;
+ peer_dev_int->peer_drv_int = peer_drv_int;
+
+ /* Initialize driver values */
+ for (j = 0; j < ICE_EVENT_NBITS; j++)
+ bitmap_zero(peer_drv_int->current_events[j].type,
+ ICE_EVENT_NBITS);
+
mutex_init(&peer_dev_int->peer_dev_state_mutex);
peer_dev = &peer_dev_int->peer_dev;
@@ -361,6 +1285,7 @@ int ice_init_peer_devices(struct ice_pf *pf)
i);
if (!peer_dev_int->ice_peer_wq)
return -ENOMEM;
+ INIT_WORK(&peer_dev_int->peer_close_task, ice_peer_close_task);
/* Assign a unique index and hence name for peer device */
status = ida_simple_get(&ice_peer_index_ida, 0, 0, GFP_KERNEL);
@@ -397,6 +1322,8 @@ int ice_init_peer_devices(struct ice_pf *pf)
/* for DCB, override the qos_info defaults. */
ice_setup_dcb_qos_info(pf, qos_info);
+ /* Initialize ice_ops */
+ peer_dev->ops = &ops;
/* make sure peer specific resources such as msix_count and
* msix_entries are initialized
diff --git a/drivers/net/ethernet/intel/ice/ice_idc.h b/drivers/net/ethernet/intel/ice/ice_idc.h
index d29e946fe2f1..455c1192ca4f 100644
--- a/drivers/net/ethernet/intel/ice/ice_idc.h
+++ b/drivers/net/ethernet/intel/ice/ice_idc.h
@@ -357,4 +357,21 @@ static inline struct ice_peer_dev *dev_to_ice_peer(struct device *_dev)
platform_dev = container_of(_dev, struct platform_device, dev);
return container_of(platform_dev, struct ice_peer_dev, platform_dev);
}
+
+/* structure representing peer driver
+ * Peer driver to initialize those function ptrs and
+ * it will be invoked by ICE as part of driver_registration
+ * via bus infrastructure
+ */
+struct ice_peer_drv {
+ u16 driver_id;
+#define ICE_PEER_LAN_DRIVER 0
+#define ICE_PEER_RDMA_DRIVER 4
+#define ICE_PEER_ADK_DRIVER 5
+
+ struct ice_ver_info ver;
+ const char *name;
+
+};
+
#endif /* _ICE_IDC_H_*/
diff --git a/drivers/net/ethernet/intel/ice/ice_idc_int.h b/drivers/net/ethernet/intel/ice/ice_idc_int.h
index f8ff8cac7a26..e67955a36342 100644
--- a/drivers/net/ethernet/intel/ice/ice_idc_int.h
+++ b/drivers/net/ethernet/intel/ice/ice_idc_int.h
@@ -6,8 +6,16 @@
#include "ice_idc.h"
+int ice_close_peer_for_reset(struct ice_peer_dev_int *peer_dev_int, void *data);
int ice_unroll_peer(struct ice_peer_dev_int *peer_dev_int, void *data);
int ice_unreg_peer_device(struct ice_peer_dev_int *peer_dev_int, void *data);
+int ice_peer_close(struct ice_peer_dev_int *peer_dev_int, void *data);
+int ice_peer_check_for_reg(struct ice_peer_dev_int *peer_dev_int, void *data);
+int
+ice_finish_init_peer_device(struct ice_peer_dev_int *peer_dev_int, void *data);
+#ifdef CONFIG_DCB
+bool ice_validate_peer_dev(struct ice_peer_dev *peer);
+#endif /* CONFIG_DCB */
enum ice_peer_dev_state {
ICE_PEER_DEV_STATE_INIT,
@@ -64,4 +72,11 @@ struct ice_peer_dev_int {
enum ice_close_reason rst_type;
};
+
+static inline struct
+ice_peer_dev_int *peer_to_ice_dev_int(struct ice_peer_dev *peer_dev)
+{
+ return container_of(peer_dev, struct ice_peer_dev_int, peer_dev);
+}
+
#endif /* !_ICE_IDC_INT_H_ */
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c
index a19f5920733b..65a3f814b87d 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.c
+++ b/drivers/net/ethernet/intel/ice/ice_lib.c
@@ -1464,6 +1464,31 @@ int ice_add_mac_to_list(struct ice_vsi *vsi, struct list_head *add_list,
return 0;
}
+/**
+ * ice_pf_state_is_nominal - checks the PF for nominal state
+ * @pf: pointer to PF to check
+ *
+ * Check the PF's state for a collection of bits that would indicate
+ * the PF is in a state that would inhibit normal operation for
+ * driver functionality.
+ *
+ * Returns true if PF is in a nominal state.
+ * Returns false otherwise
+ */
+bool ice_pf_state_is_nominal(struct ice_pf *pf)
+{
+ DECLARE_BITMAP(check_bits, __ICE_STATE_NBITS) = { 0 };
+
+ if (!pf)
+ return false;
+
+ bitmap_set(check_bits, 0, __ICE_STATE_NOMINAL_CHECK_BITS);
+ if (bitmap_intersects(pf->state, check_bits, __ICE_STATE_NBITS))
+ return false;
+
+ return true;
+}
+
/**
* ice_update_eth_stats - Update VSI-specific ethernet statistics counters
* @vsi: the VSI to be updated
@@ -2684,9 +2709,17 @@ void ice_vsi_free_rx_rings(struct ice_vsi *vsi)
*/
void ice_vsi_close(struct ice_vsi *vsi)
{
+ enum ice_close_reason reason = ICE_REASON_INTERFACE_DOWN;
+ struct device *dev = &vsi->back->pdev->dev;
+ int ret = 0;
+
+ if (vsi->type == ICE_VSI_PF)
+ ret = ice_for_each_peer(vsi->back, &reason, ice_peer_close);
+
+ if (ret)
+ dev_dbg(dev, "Peer device did not implement close function\n");
if (!test_and_set_bit(__ICE_DOWN, vsi->state))
ice_down(vsi);
-
ice_vsi_free_irq(vsi);
ice_vsi_free_tx_rings(vsi);
ice_vsi_free_rx_rings(vsi);
diff --git a/drivers/net/ethernet/intel/ice/ice_lib.h b/drivers/net/ethernet/intel/ice/ice_lib.h
index 6e43ef03bfc3..5353a71fcca0 100644
--- a/drivers/net/ethernet/intel/ice/ice_lib.h
+++ b/drivers/net/ethernet/intel/ice/ice_lib.h
@@ -11,6 +11,8 @@ int ice_add_mac_to_list(struct ice_vsi *vsi, struct list_head *add_list,
void ice_free_fltr_list(struct device *dev, struct list_head *h);
+bool ice_pf_state_is_nominal(struct ice_pf *pf);
+
void ice_update_eth_stats(struct ice_vsi *vsi);
int ice_vsi_cfg_rxqs(struct ice_vsi *vsi);
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 08bce0d67634..62b45c918ab5 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -576,6 +576,7 @@ static void ice_reset_subtask(struct ice_pf *pf)
/* return if no valid reset type requested */
if (reset_type == ICE_RESET_INVAL)
return;
+ ice_for_each_peer(pf, &reset_type, ice_close_peer_for_reset);
ice_prepare_for_reset(pf);
/* make sure we are ready to rebuild */
@@ -1336,6 +1337,8 @@ static void ice_service_task(struct work_struct *work)
return;
}
+ /* Invoke remaining initialization of peer devices */
+ ice_for_each_peer(pf, NULL, ice_finish_init_peer_device);
ice_check_for_hang_subtask(pf);
ice_sync_fltr_subtask(pf);
ice_handle_mdd_event(pf);
@@ -1375,6 +1378,42 @@ static void ice_set_ctrlq_len(struct ice_hw *hw)
hw->mailboxq.sq_buf_size = ICE_MBXQ_MAX_BUF_LEN;
}
+/**
+ * ice_schedule_reset - schedule a reset
+ * @pf: board private structure
+ * @reset: reset being requested
+ */
+int ice_schedule_reset(struct ice_pf *pf, enum ice_reset_req reset)
+{
+ /* bail out if earlier reset has failed */
+ if (test_bit(__ICE_RESET_FAILED, pf->state)) {
+ dev_dbg(&pf->pdev->dev, "earlier reset has failed\n");
+ return -EIO;
+ }
+ /* bail if reset/recovery already in progress */
+ if (ice_is_reset_in_progress(pf->state)) {
+ dev_dbg(&pf->pdev->dev, "Reset already in progress\n");
+ return -EBUSY;
+ }
+
+ switch (reset) {
+ case ICE_RESET_PFR:
+ set_bit(__ICE_PFR_REQ, pf->state);
+ break;
+ case ICE_RESET_CORER:
+ set_bit(__ICE_CORER_REQ, pf->state);
+ break;
+ case ICE_RESET_GLOBR:
+ set_bit(__ICE_GLOBR_REQ, pf->state);
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ ice_service_task_schedule(pf);
+ return 0;
+}
+
/**
* ice_irq_affinity_notify - Callback for affinity changes
* @notify: context as to what irq was changed
@@ -2496,6 +2535,7 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)
static void ice_remove(struct pci_dev *pdev)
{
struct ice_pf *pf = pci_get_drvdata(pdev);
+ enum ice_close_reason reason;
int i;
if (!pf)
@@ -2507,8 +2547,10 @@ static void ice_remove(struct pci_dev *pdev)
msleep(100);
}
- set_bit(__ICE_DOWN, pf->state);
ice_service_task_stop(pf);
+ reason = ICE_REASON_INTERFACE_DOWN;
+ ice_for_each_peer(pf, &reason, ice_peer_close);
+ set_bit(__ICE_DOWN, pf->state);
if (test_bit(ICE_FLAG_SRIOV_ENA, pf->flags))
ice_free_vfs(pf);
@@ -3946,6 +3988,7 @@ static int ice_change_mtu(struct net_device *netdev, int new_mtu)
struct ice_netdev_priv *np = netdev_priv(netdev);
struct ice_vsi *vsi = np->vsi;
struct ice_pf *pf = vsi->back;
+ struct ice_event *event;
u8 count = 0;
if (new_mtu == netdev->mtu) {
@@ -3997,6 +4040,13 @@ static int ice_change_mtu(struct net_device *netdev, int new_mtu)
}
}
+ event = devm_kzalloc(&pf->pdev->dev, sizeof(*event), GFP_KERNEL);
+ set_bit(ICE_EVENT_MTU_CHANGE, event->type);
+ event->reporter = NULL;
+ event->info.mtu = new_mtu;
+ ice_for_each_peer(pf, event, ice_peer_check_for_reg);
+ devm_kfree(&pf->pdev->dev, event);
+
netdev_info(netdev, "changed MTU to %d\n", new_mtu);
return 0;
}
diff --git a/drivers/net/ethernet/intel/ice/ice_sched.c b/drivers/net/ethernet/intel/ice/ice_sched.c
index 2a232504379d..1bc5b8e61eee 100644
--- a/drivers/net/ethernet/intel/ice/ice_sched.c
+++ b/drivers/net/ethernet/intel/ice/ice_sched.c
@@ -576,6 +576,50 @@ ice_alloc_lan_q_ctx(struct ice_hw *hw, u16 vsi_handle, u8 tc, u16 new_numqs)
return 0;
}
+/**
+ * ice_alloc_rdma_q_ctx - allocate RDMA queue contexts for the given VSI and TC
+ * @hw: pointer to the HW struct
+ * @vsi_handle: VSI handle
+ * @tc: TC number
+ * @new_numqs: number of queues
+ */
+static enum ice_status
+ice_alloc_rdma_q_ctx(struct ice_hw *hw, u16 vsi_handle, u8 tc, u16 new_numqs)
+{
+ struct ice_vsi_ctx *vsi_ctx;
+ struct ice_q_ctx *q_ctx;
+
+ vsi_ctx = ice_get_vsi_ctx(hw, vsi_handle);
+ if (!vsi_ctx)
+ return ICE_ERR_PARAM;
+ /* allocate RDMA queue contexts */
+ if (!vsi_ctx->rdma_q_ctx[tc]) {
+ vsi_ctx->rdma_q_ctx[tc] = devm_kcalloc(ice_hw_to_dev(hw),
+ new_numqs,
+ sizeof(*q_ctx),
+ GFP_KERNEL);
+ if (!vsi_ctx->rdma_q_ctx[tc])
+ return ICE_ERR_NO_MEMORY;
+ vsi_ctx->num_rdma_q_entries[tc] = new_numqs;
+ return 0;
+ }
+ /* num queues are increased, update the queue contexts */
+ if (new_numqs > vsi_ctx->num_rdma_q_entries[tc]) {
+ u16 prev_num = vsi_ctx->num_rdma_q_entries[tc];
+
+ q_ctx = devm_kcalloc(ice_hw_to_dev(hw), new_numqs,
+ sizeof(*q_ctx), GFP_KERNEL);
+ if (!q_ctx)
+ return ICE_ERR_NO_MEMORY;
+ memcpy(q_ctx, vsi_ctx->rdma_q_ctx[tc],
+ prev_num * sizeof(*q_ctx));
+ devm_kfree(ice_hw_to_dev(hw), vsi_ctx->rdma_q_ctx[tc]);
+ vsi_ctx->rdma_q_ctx[tc] = q_ctx;
+ vsi_ctx->num_rdma_q_entries[tc] = new_numqs;
+ }
+ return 0;
+}
+
/**
* ice_sched_clear_agg - clears the aggregator related information
* @hw: pointer to the hardware structure
@@ -1447,13 +1491,22 @@ ice_sched_update_vsi_child_nodes(struct ice_port_info *pi, u16 vsi_handle,
if (!vsi_ctx)
return ICE_ERR_PARAM;
- prev_numqs = vsi_ctx->sched.max_lanq[tc];
+ if (owner == ICE_SCHED_NODE_OWNER_LAN)
+ prev_numqs = vsi_ctx->sched.max_lanq[tc];
+ else
+ prev_numqs = vsi_ctx->sched.max_rdmaq[tc];
/* num queues are not changed or less than the previous number */
if (new_numqs <= prev_numqs)
return status;
- status = ice_alloc_lan_q_ctx(hw, vsi_handle, tc, new_numqs);
- if (status)
- return status;
+ if (owner == ICE_SCHED_NODE_OWNER_LAN) {
+ status = ice_alloc_lan_q_ctx(hw, vsi_handle, tc, new_numqs);
+ if (status)
+ return status;
+ } else {
+ status = ice_alloc_rdma_q_ctx(hw, vsi_handle, tc, new_numqs);
+ if (status)
+ return status;
+ }
if (new_numqs)
ice_sched_calc_vsi_child_nodes(hw, new_numqs, new_num_nodes);
@@ -1468,7 +1521,10 @@ ice_sched_update_vsi_child_nodes(struct ice_port_info *pi, u16 vsi_handle,
new_num_nodes, owner);
if (status)
return status;
- vsi_ctx->sched.max_lanq[tc] = new_numqs;
+ if (owner == ICE_SCHED_NODE_OWNER_LAN)
+ vsi_ctx->sched.max_lanq[tc] = new_numqs;
+ else
+ vsi_ctx->sched.max_rdmaq[tc] = new_numqs;
return 0;
}
@@ -1534,6 +1590,7 @@ ice_sched_cfg_vsi(struct ice_port_info *pi, u16 vsi_handle, u8 tc, u16 maxqs,
* recreate the child nodes all the time in these cases.
*/
vsi_ctx->sched.max_lanq[tc] = 0;
+ vsi_ctx->sched.max_rdmaq[tc] = 0;
}
/* update the VSI child nodes */
@@ -1665,6 +1722,8 @@ ice_sched_rm_vsi_cfg(struct ice_port_info *pi, u16 vsi_handle, u8 owner)
}
if (owner == ICE_SCHED_NODE_OWNER_LAN)
vsi_ctx->sched.max_lanq[i] = 0;
+ else
+ vsi_ctx->sched.max_rdmaq[i] = 0;
}
status = 0;
diff --git a/drivers/net/ethernet/intel/ice/ice_switch.c b/drivers/net/ethernet/intel/ice/ice_switch.c
index 8271fd651725..92c44973af51 100644
--- a/drivers/net/ethernet/intel/ice/ice_switch.c
+++ b/drivers/net/ethernet/intel/ice/ice_switch.c
@@ -346,6 +346,10 @@ static void ice_clear_vsi_q_ctx(struct ice_hw *hw, u16 vsi_handle)
devm_kfree(ice_hw_to_dev(hw), vsi->lan_q_ctx[i]);
vsi->lan_q_ctx[i] = NULL;
}
+ if (vsi->rdma_q_ctx[i]) {
+ devm_kfree(ice_hw_to_dev(hw), vsi->rdma_q_ctx[i]);
+ vsi->rdma_q_ctx[i] = NULL;
+ }
}
}
@@ -467,6 +471,29 @@ ice_update_vsi(struct ice_hw *hw, u16 vsi_handle, struct ice_vsi_ctx *vsi_ctx,
return ice_aq_update_vsi(hw, vsi_ctx, cd);
}
+/**
+ * ice_cfg_iwarp_fltr - enable/disable iWARP filtering on VSI
+ * @hw: pointer to HW struct
+ * @vsi_handle: VSI SW index
+ * @enable: boolean for enable/disable
+ */
+enum ice_status
+ice_cfg_iwarp_fltr(struct ice_hw *hw, u16 vsi_handle, bool enable)
+{
+ struct ice_vsi_ctx *ctx;
+
+ ctx = ice_get_vsi_ctx(hw, vsi_handle);
+ if (!ctx)
+ return ICE_ERR_DOES_NOT_EXIST;
+
+ if (enable)
+ ctx->info.q_opt_flags |= ICE_AQ_VSI_Q_OPT_PE_FLTR_EN;
+ else
+ ctx->info.q_opt_flags &= ~ICE_AQ_VSI_Q_OPT_PE_FLTR_EN;
+
+ return ice_update_vsi(hw, vsi_handle, ctx, NULL);
+}
+
/**
* ice_aq_alloc_free_vsi_list
* @hw: pointer to the HW struct
diff --git a/drivers/net/ethernet/intel/ice/ice_switch.h b/drivers/net/ethernet/intel/ice/ice_switch.h
index cb123fbe30be..a81a9dd509d7 100644
--- a/drivers/net/ethernet/intel/ice/ice_switch.h
+++ b/drivers/net/ethernet/intel/ice/ice_switch.h
@@ -31,6 +31,8 @@ struct ice_vsi_ctx {
u8 vf_num;
u16 num_lan_q_entries[ICE_MAX_TRAFFIC_CLASS];
struct ice_q_ctx *lan_q_ctx[ICE_MAX_TRAFFIC_CLASS];
+ u16 num_rdma_q_entries[ICE_MAX_TRAFFIC_CLASS];
+ struct ice_q_ctx *rdma_q_ctx[ICE_MAX_TRAFFIC_CLASS];
};
enum ice_sw_fwd_act_type {
@@ -225,6 +227,8 @@ void ice_remove_vsi_fltr(struct ice_hw *hw, u16 vsi_handle);
enum ice_status
ice_add_vlan(struct ice_hw *hw, struct list_head *m_list);
enum ice_status ice_remove_vlan(struct ice_hw *hw, struct list_head *v_list);
+enum ice_status
+ice_cfg_iwarp_fltr(struct ice_hw *hw, u16 vsi_handle, bool enable);
/* Promisc/defport setup for VSIs */
enum ice_status
diff --git a/drivers/net/ethernet/intel/ice/ice_type.h b/drivers/net/ethernet/intel/ice/ice_type.h
index 99822b4e8621..737181f006ee 100644
--- a/drivers/net/ethernet/intel/ice/ice_type.h
+++ b/drivers/net/ethernet/intel/ice/ice_type.h
@@ -31,6 +31,7 @@ static inline bool ice_is_tc_ena(u8 bitmap, u8 tc)
#define ICE_DBG_LAN BIT_ULL(8)
#define ICE_DBG_SW BIT_ULL(13)
#define ICE_DBG_SCHED BIT_ULL(14)
+#define ICE_DBG_RDMA BIT_ULL(15)
#define ICE_DBG_RES BIT_ULL(17)
#define ICE_DBG_AQ_MSG BIT_ULL(24)
#define ICE_DBG_AQ_CMD BIT_ULL(27)
@@ -242,6 +243,7 @@ struct ice_sched_node {
u8 tc_num;
u8 owner;
#define ICE_SCHED_NODE_OWNER_LAN 0
+#define ICE_SCHED_NODE_OWNER_RDMA 2
};
/* Access Macros for Tx Sched Elements data */
@@ -267,6 +269,7 @@ struct ice_sched_vsi_info {
struct ice_sched_node *ag_node[ICE_MAX_TRAFFIC_CLASS];
struct list_head list_entry;
u16 max_lanq[ICE_MAX_TRAFFIC_CLASS];
+ u16 max_rdmaq[ICE_MAX_TRAFFIC_CLASS];
};
/* driver defines the policy */
diff --git a/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c b/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c
index 5d24b539648f..8feb0ddad729 100644
--- a/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c
+++ b/drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c
@@ -1310,31 +1310,6 @@ static int ice_alloc_vfs(struct ice_pf *pf, u16 num_alloc_vfs)
return ret;
}
-/**
- * ice_pf_state_is_nominal - checks the PF for nominal state
- * @pf: pointer to PF to check
- *
- * Check the PF's state for a collection of bits that would indicate
- * the PF is in a state that would inhibit normal operation for
- * driver functionality.
- *
- * Returns true if PF is in a nominal state.
- * Returns false otherwise
- */
-static bool ice_pf_state_is_nominal(struct ice_pf *pf)
-{
- DECLARE_BITMAP(check_bits, __ICE_STATE_NBITS) = { 0 };
-
- if (!pf)
- return false;
-
- bitmap_set(check_bits, 0, __ICE_STATE_NOMINAL_CHECK_BITS);
- if (bitmap_intersects(pf->state, check_bits, __ICE_STATE_NBITS))
- return false;
-
- return true;
-}
-
/**
* ice_pci_sriov_ena - Enable or change number of VFs
* @pf: pointer to the PF structure
--
2.21.0
^ permalink raw reply related
* [rdma 01/16] RDMA/irdma: Add driver framework definitions
From: Jeff Kirsher @ 2019-07-04 2:12 UTC (permalink / raw)
To: dledford, jgg, davem
Cc: Mustafa Ismail, linux-rdma, netdev, nhorman, sassmann, poswald,
david.m.ertman, Shiraz Saleem
In-Reply-To: <20190704021259.15489-1-jeffrey.t.kirsher@intel.com>
From: Mustafa Ismail <mustafa.ismail@intel.com>
Register irdma as a platform driver capable of supporting platform
devices from multi-generation RDMA capable Intel HW. Establish the
interface with all supported netdev peer devices and initialize HW.
Signed-off-by: Mustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
---
drivers/infiniband/hw/irdma/i40iw_if.c | 256 ++++++++++
drivers/infiniband/hw/irdma/irdma_if.c | 426 +++++++++++++++++
drivers/infiniband/hw/irdma/main.c | 531 ++++++++++++++++++++
drivers/infiniband/hw/irdma/main.h | 639 +++++++++++++++++++++++++
4 files changed, 1852 insertions(+)
create mode 100644 drivers/infiniband/hw/irdma/i40iw_if.c
create mode 100644 drivers/infiniband/hw/irdma/irdma_if.c
create mode 100644 drivers/infiniband/hw/irdma/main.c
create mode 100644 drivers/infiniband/hw/irdma/main.h
diff --git a/drivers/infiniband/hw/irdma/i40iw_if.c b/drivers/infiniband/hw/irdma/i40iw_if.c
new file mode 100644
index 000000000000..ca717ce5035f
--- /dev/null
+++ b/drivers/infiniband/hw/irdma/i40iw_if.c
@@ -0,0 +1,256 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019, Intel Corporation. */
+
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <net/addrconf.h>
+#include "main.h"
+#include "i40iw_hw.h"
+#include "i40e_client.h"
+
+/**
+ * i40iw_request_reset - Request a reset
+ * @rf: RDMA PCI function
+ *
+ */
+void i40iw_request_reset(struct irdma_pci_f *rf)
+{
+ struct i40e_info *ldev = (struct i40e_info *)rf->ldev.if_ldev;
+
+ ldev->ops->request_reset(ldev, rf->ldev.if_client, 1);
+}
+
+/**
+ * i40iw_open - client interface operation open for iwarp/uda device
+ * @ldev: lan device information
+ * @client: iwarp client information, provided during registration
+ *
+ * Called by the lan driver during the processing of client register
+ * Create device resources, set up queues, pble and hmc objects and
+ * register the device with the ib verbs interface
+ * Return 0 if successful, otherwise return error
+ */
+static int i40iw_open(struct i40e_info *ldev, struct i40e_client *client)
+{
+ struct irdma_device *iwdev = NULL;
+ struct irdma_handler *hdl = NULL;
+ struct irdma_priv_ldev *pldev;
+ struct irdma_sc_dev *dev;
+ struct irdma_pci_f *rf;
+ struct irdma_l2params l2params = {};
+ int err_code = -EIO;
+ int i;
+ u16 qset;
+ u16 last_qset = IRDMA_NO_QSET;
+
+ hdl = irdma_find_handler(ldev->pcidev);
+ if (hdl)
+ return 0;
+
+ hdl = kzalloc((sizeof(*hdl) + sizeof(*iwdev)), GFP_KERNEL);
+ if (!hdl)
+ return -ENOMEM;
+
+ iwdev = (struct irdma_device *)((u8 *)hdl + sizeof(*hdl));
+
+ iwdev->param_wq = alloc_ordered_workqueue("l2params", WQ_MEM_RECLAIM);
+ if (!iwdev->param_wq)
+ goto error;
+
+ rf = &hdl->rf;
+ rf->hdl = hdl;
+ dev = &rf->sc_dev;
+ dev->back_dev = rf;
+ rf->rdma_ver = IRDMA_GEN_1;
+ hdl->platform_dev = &ldev->platform_dev;
+ irdma_init_rf_config_params(rf);
+ rf->init_hw = i40iw_init_hw;
+ rf->hw.hw_addr = ldev->hw_addr;
+ rf->pdev = ldev->pcidev;
+ rf->netdev = ldev->netdev;
+ dev->pci_rev = rf->pdev->revision;
+ iwdev->rf = rf;
+ iwdev->hdl = hdl;
+ iwdev->ldev = &rf->ldev;
+ iwdev->init_state = INITIAL_STATE;
+ iwdev->rcv_wnd = IRDMA_CM_DEFAULT_RCV_WND_SCALED;
+ iwdev->rcv_wscale = IRDMA_CM_DEFAULT_RCV_WND_SCALE;
+ iwdev->netdev = ldev->netdev;
+ iwdev->create_ilq = true;
+ iwdev->vsi_num = 0;
+
+ pldev = &rf->ldev;
+ hdl->ldev = pldev;
+ pldev->if_client = client;
+ pldev->if_ldev = ldev;
+ pldev->fn_num = ldev->fid;
+ pldev->ftype = ldev->ftype;
+ pldev->pf_vsi_num = 0;
+ pldev->msix_count = ldev->msix_count;
+ pldev->msix_entries = ldev->msix_entries;
+
+ if (irdma_ctrl_init_hw(rf))
+ goto error;
+
+ l2params.mtu =
+ (ldev->params.mtu) ? ldev->params.mtu : IRDMA_DEFAULT_MTU;
+ for (i = 0; i < I40E_CLIENT_MAX_USER_PRIORITY; i++) {
+ qset = ldev->params.qos.prio_qos[i].qs_handle;
+ l2params.up2tc[i] = ldev->params.qos.prio_qos[i].tc;
+ l2params.qs_handle_list[i] = qset;
+ if (last_qset == IRDMA_NO_QSET)
+ last_qset = qset;
+ else if ((qset != last_qset) && (qset != IRDMA_NO_QSET))
+ iwdev->dcb = true;
+ }
+
+ if (irdma_rt_init_hw(rf, iwdev, &l2params)) {
+ irdma_deinit_ctrl_hw(rf);
+ goto error;
+ }
+
+ irdma_add_handler(hdl);
+ return 0;
+error:
+ kfree(hdl);
+ return err_code;
+}
+
+/**
+ * i40iw_l2params_worker - worker for l2 params change
+ * @work: work pointer for l2 params
+ */
+static void i40iw_l2params_worker(struct work_struct *work)
+{
+ struct l2params_work *dwork =
+ container_of(work, struct l2params_work, work);
+ struct irdma_device *iwdev = dwork->iwdev;
+
+ irdma_change_l2params(&iwdev->vsi, &dwork->l2params);
+ atomic_dec(&iwdev->params_busy);
+ kfree(work);
+}
+
+/**
+ * i40iw_l2param_change - handle qs handles for qos and mss change
+ * @ldev: lan device information
+ * @client: client for parameter change
+ * @params: new parameters from L2
+ */
+static void i40iw_l2param_change(struct i40e_info *ldev,
+ struct i40e_client *client,
+ struct i40e_params *params)
+{
+ struct irdma_handler *hdl;
+ struct irdma_l2params *l2params;
+ struct l2params_work *work;
+ struct irdma_device *iwdev;
+ int i;
+
+ hdl = irdma_find_handler(ldev->pcidev);
+ if (!hdl)
+ return;
+
+ iwdev = (struct irdma_device *)((u8 *)hdl + sizeof(*hdl));
+
+ if (atomic_read(&iwdev->params_busy))
+ return;
+ work = kzalloc(sizeof(*work), GFP_KERNEL);
+ if (!work)
+ return;
+
+ atomic_inc(&iwdev->params_busy);
+ work->iwdev = iwdev;
+ l2params = &work->l2params;
+ for (i = 0; i < I40E_CLIENT_MAX_USER_PRIORITY; i++)
+ l2params->qs_handle_list[i] = params->qos.prio_qos[i].qs_handle;
+
+ l2params->mtu = (params->mtu) ? params->mtu : iwdev->vsi.mtu;
+
+ INIT_WORK(&work->work, i40iw_l2params_worker);
+ queue_work(iwdev->param_wq, &work->work);
+}
+
+/**
+ * i40iw_close - client interface operation close for iwarp/uda device
+ * @ldev: lan device information
+ * @client: client to close
+ * @reset: flag to indicate close on reset
+ *
+ * Called by the lan driver during the processing of client unregister
+ * Destroy and clean up the driver resources
+ */
+static void i40iw_close(struct i40e_info *ldev, struct i40e_client *client,
+ bool reset)
+{
+ struct irdma_handler *hdl;
+ struct irdma_pci_f *rf;
+ struct irdma_device *iwdev;
+
+ hdl = irdma_find_handler(ldev->pcidev);
+ if (!hdl)
+ return;
+ rf = &hdl->rf;
+ iwdev = (struct irdma_device *)((u8 *)hdl + sizeof(*hdl));
+
+ if (iwdev->param_wq)
+ destroy_workqueue(iwdev->param_wq);
+
+ if (reset)
+ iwdev->reset = true;
+
+ irdma_deinit_rt_device(iwdev);
+ irdma_deinit_ctrl_hw(rf);
+ irdma_del_handler(irdma_find_handler(ldev->pcidev));
+ kfree(hdl);
+ pr_info("IRDMA hardware deinitialization complete\n");
+}
+
+/* client interface functions */
+static const struct i40e_client_ops i40e_ops = {
+ .open = i40iw_open,
+ .close = i40iw_close,
+ .l2_param_change = i40iw_l2param_change
+};
+
+static struct i40e_client i40iw_client = {
+ .name = "irdma",
+ .ops = &i40e_ops,
+ .version.major = I40E_CLIENT_VERSION_MAJOR,
+ .version.minor = I40E_CLIENT_VERSION_MINOR,
+ .version.build = I40E_CLIENT_VERSION_BUILD,
+ .type = I40E_CLIENT_IWARP,
+};
+
+int i40iw_probe(struct platform_device *pdev)
+{
+ struct i40e_info *ldev = container_of(pdev, struct i40e_info,
+ platform_dev);
+ if (ldev->version.major != I40E_CLIENT_VERSION_MAJOR ||
+ ldev->version.minor != I40E_CLIENT_VERSION_MINOR) {
+ pr_err("version mismatch:\n");
+ pr_err("expected major ver %d, caller specified major ver %d\n",
+ I40E_CLIENT_VERSION_MAJOR, ldev->version.major);
+ pr_err("expected minor ver %d, caller specified minor ver %d\n",
+ I40E_CLIENT_VERSION_MINOR, ldev->version.minor);
+ return -EINVAL;
+ }
+
+ if (!ldev->ops->client_device_register)
+ return -EINVAL;
+
+ ldev->client = &i40iw_client;
+
+ return ldev->ops->client_device_register(ldev);
+}
+
+void i40iw_remove(struct platform_device *pdev)
+{
+ struct i40e_info *ldev = container_of(pdev, struct i40e_info,
+ platform_dev);
+
+ if (ldev->ops->client_device_unregister)
+ ldev->ops->client_device_unregister(ldev);
+}
diff --git a/drivers/infiniband/hw/irdma/irdma_if.c b/drivers/infiniband/hw/irdma/irdma_if.c
new file mode 100644
index 000000000000..d30fe2e694c3
--- /dev/null
+++ b/drivers/infiniband/hw/irdma/irdma_if.c
@@ -0,0 +1,426 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019, Intel Corporation. */
+
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <ice_idc.h>
+#include "main.h"
+#include "ws.h"
+#include "icrdma_hw.h"
+
+/**
+ * irdma_lan_register_qset - Register qset with LAN driver
+ * @vsi: vsi structure
+ * @tc_node: Traffic class node
+ */
+enum irdma_status_code irdma_lan_register_qset(struct irdma_sc_vsi *vsi,
+ struct irdma_ws_node *tc_node)
+{
+ struct irdma_device *iwdev = vsi->back_vsi;
+ struct ice_peer_dev *ldev = (struct ice_peer_dev *)iwdev->ldev->if_ldev;
+ struct ice_res rdma_qset_res = {};
+ int ret;
+
+ if (ldev->ops->alloc_res) {
+ rdma_qset_res.cnt_req = 1;
+ rdma_qset_res.res_type = ICE_RDMA_QSETS_TXSCHED;
+ rdma_qset_res.res[0].res.qsets.qs_handle = tc_node->qs_handle;
+ rdma_qset_res.res[0].res.qsets.tc = tc_node->traffic_class;
+ rdma_qset_res.res[0].res.qsets.vsi_id = vsi->vsi_idx;
+ ret = ldev->ops->alloc_res(ldev, &rdma_qset_res, 0);
+ if (ret) {
+ dev_dbg(rfdev_to_dev(vsi->dev),
+ "WS: LAN alloc_res for rdma qset failed.\n");
+ return IRDMA_ERR_NO_MEMORY;
+ }
+
+ tc_node->l2_sched_node_id = rdma_qset_res.res[0].res.qsets.teid;
+ vsi->qos[tc_node->user_pri].l2_sched_node_id =
+ rdma_qset_res.res[0].res.qsets.teid;
+ }
+
+ return 0;
+}
+
+/**
+ * irdma_lan_unregister_qset - Unregister qset with LAN driver
+ * @vsi: vsi structure
+ * @tc_node: Traffic class node
+ */
+void irdma_lan_unregister_qset(struct irdma_sc_vsi *vsi,
+ struct irdma_ws_node *tc_node)
+{
+ struct irdma_device *iwdev = vsi->back_vsi;
+ struct ice_peer_dev *ldev = (struct ice_peer_dev *)iwdev->ldev->if_ldev;
+ struct ice_res rdma_qset_res = {};
+
+ if (ldev->ops->free_res) {
+ rdma_qset_res.res_allocated = 1;
+ rdma_qset_res.res_type = ICE_RDMA_QSETS_TXSCHED;
+ rdma_qset_res.res[0].res.qsets.vsi_id = vsi->vsi_idx;
+ rdma_qset_res.res[0].res.qsets.teid = tc_node->l2_sched_node_id;
+ rdma_qset_res.res[0].res.qsets.qs_handle = tc_node->qs_handle;
+
+ if (ldev->ops->free_res(ldev, &rdma_qset_res))
+ dev_dbg(rfdev_to_dev(vsi->dev),
+ "WS: LAN free_res for rdma qset failed.\n");
+ }
+}
+
+/**
+ * irdma_log_invalid_mtu: log warning on invalid mtu
+ * @mtu: maximum tranmission unit
+ */
+static void irdma_log_invalid_mtu(u16 mtu)
+{
+ if (mtu < IRDMA_MIN_MTU_IPV4)
+ pr_warn("Current MTU setting of %d is too low for RDMA traffic. Minimum MTU is 576 for IPv4 and 1280 for IPv6\n",
+ mtu);
+ else if (mtu < IRDMA_MIN_MTU_IPV6)
+ pr_warn("Current MTU setting of %d is too low for IPv6 RDMA traffic, the minimum is 1280\n",
+ mtu);
+}
+
+/**
+ * irdma_prep_tc_change - Prepare for TC changes
+ * @ldev: Peer device structure
+ */
+static void irdma_prep_tc_change(struct ice_peer_dev *ldev)
+{
+ struct irdma_device *iwdev;
+
+ iwdev = irdma_get_device(ldev->netdev);
+ if (!iwdev)
+ return;
+
+ if (iwdev->vsi.tc_change_pending)
+ goto done;
+
+ iwdev->vsi.tc_change_pending = true;
+ irdma_suspend_qps(&iwdev->vsi);
+
+ /* Wait for all qp's to suspend */
+ wait_event_timeout(iwdev->suspend_wq,
+ !atomic_read(&iwdev->vsi.qp_suspend_reqs),
+ IRDMA_EVENT_TIMEOUT);
+ irdma_ws_reset(&iwdev->vsi);
+done:
+ irdma_put_device(iwdev);
+}
+
+/**
+ * irdma_event_handler - Called by LAN driver to notify events
+ * @ldev: Peer device structure
+ * @event: event from LAN driver
+ */
+static void irdma_event_handler(struct ice_peer_dev *ldev,
+ struct ice_event *event)
+{
+ struct irdma_l2params l2params = {};
+ struct irdma_device *iwdev;
+ int i;
+
+ iwdev = irdma_get_device(ldev->netdev);
+ if (!iwdev)
+ return;
+
+ if (test_bit(ICE_EVENT_LINK_CHANGE, event->type)) {
+ dev_dbg(rfdev_to_dev(&iwdev->rf->sc_dev),
+ "CLNT: LINK_CHANGE event\n");
+ } else if (test_bit(ICE_EVENT_MTU_CHANGE, event->type)) {
+ dev_dbg(rfdev_to_dev(&iwdev->rf->sc_dev),
+ "CLNT: new MTU = %d\n", event->info.mtu);
+ if (iwdev->vsi.mtu != event->info.mtu) {
+ l2params.mtu = event->info.mtu;
+ l2params.mtu_changed = true;
+ irdma_log_invalid_mtu(l2params.mtu);
+ irdma_change_l2params(&iwdev->vsi, &l2params);
+ }
+ } else if (test_bit(ICE_EVENT_TC_CHANGE, event->type)) {
+ if (!iwdev->vsi.tc_change_pending)
+ goto done;
+
+ l2params.tc_changed = true;
+ dev_dbg(rfdev_to_dev(&iwdev->rf->sc_dev), "CLNT: TC Change\n");
+ iwdev->dcb = event->info.port_qos.num_tc > 1 ? true : false;
+
+ for (i = 0; i < ICE_IDC_MAX_USER_PRIORITY; ++i)
+ l2params.up2tc[i] = event->info.port_qos.up2tc[i];
+ irdma_change_l2params(&iwdev->vsi, &l2params);
+ } else if (test_bit(ICE_EVENT_API_CHANGE, event->type)) {
+ dev_dbg(rfdev_to_dev(&iwdev->rf->sc_dev),
+ "CLNT: API_CHANGE\n");
+ }
+
+done:
+ irdma_put_device(iwdev);
+}
+
+/**
+ * irdma_open - client interface operation open for RDMA device
+ * @ldev: lan device information
+ *
+ * Called by the lan driver during the processing of client
+ * register.
+ */
+static int irdma_open(struct ice_peer_dev *ldev)
+{
+ struct irdma_handler *hdl;
+ struct irdma_device *iwdev;
+ struct irdma_sc_dev *dev;
+ enum irdma_status_code status;
+ struct ice_event events = {};
+ struct irdma_pci_f *rf;
+ struct irdma_priv_ldev *pldev;
+ struct irdma_l2params l2params = {};
+ int i;
+
+ hdl = irdma_find_handler(ldev->pdev);
+ if (!hdl)
+ return -ENODEV;
+
+ rf = &hdl->rf;
+ if (rf->init_state != CEQ0_CREATED)
+ return -EINVAL;
+
+ iwdev = kzalloc(sizeof(*iwdev), GFP_KERNEL);
+ if (!iwdev)
+ return -ENOMEM;
+
+ iwdev->hdl = hdl;
+ iwdev->rf = rf;
+ iwdev->ldev = &rf->ldev;
+ pldev = &rf->ldev;
+ pldev->pf_vsi_num = ldev->pf_vsi_num;
+
+ /* Set configfs default values */
+ iwdev->push_mode = 0;
+ iwdev->rcv_wnd = IRDMA_CM_DEFAULT_RCV_WND_SCALED;
+ iwdev->rcv_wscale = IRDMA_CM_DEFAULT_RCV_WND_SCALE;
+
+ dev = &hdl->rf.sc_dev;
+ iwdev->netdev = ldev->netdev;
+ iwdev->create_ilq = true;
+ if (rf->protocol_used == IRDMA_ROCE_PROTOCOL_ONLY) {
+ iwdev->roce_mode = true;
+ iwdev->create_ilq = false;
+ }
+ l2params.mtu = ldev->initial_mtu;
+
+ l2params.num_tc = ldev->initial_qos_info.num_tc;
+ l2params.num_apps = ldev->initial_qos_info.num_apps;
+ l2params.vsi_prio_type = ldev->initial_qos_info.vsi_priority_type;
+ l2params.vsi_rel_bw = ldev->initial_qos_info.vsi_relative_bw;
+ for (i = 0; i < l2params.num_tc; i++) {
+ l2params.tc_info[i].egress_virt_up =
+ ldev->initial_qos_info.tc_info[i].egress_virt_up;
+ l2params.tc_info[i].ingress_virt_up =
+ ldev->initial_qos_info.tc_info[i].ingress_virt_up;
+ l2params.tc_info[i].prio_type =
+ ldev->initial_qos_info.tc_info[i].prio_type;
+ l2params.tc_info[i].rel_bw =
+ ldev->initial_qos_info.tc_info[i].rel_bw;
+ l2params.tc_info[i].tc_ctx =
+ ldev->initial_qos_info.tc_info[i].tc_ctx;
+ }
+ for (i = 0; i < ICE_IDC_MAX_USER_PRIORITY; i++)
+ l2params.up2tc[i] = ldev->initial_qos_info.up2tc[i];
+
+ iwdev->vsi_num = ldev->pf_vsi_num;
+ ldev->ops->update_vsi_filter(ldev, ICE_RDMA_FILTER_BOTH, true);
+
+ status = irdma_rt_init_hw(rf, iwdev, &l2params);
+ if (status) {
+ kfree(iwdev);
+ return -EIO;
+ }
+
+ events.reporter = ldev;
+ set_bit(ICE_EVENT_LINK_CHANGE, events.type);
+ set_bit(ICE_EVENT_MTU_CHANGE, events.type);
+ set_bit(ICE_EVENT_TC_CHANGE, events.type);
+ set_bit(ICE_EVENT_API_CHANGE, events.type);
+
+ if (ldev->ops->reg_for_notification)
+ ldev->ops->reg_for_notification(ldev, &events);
+ dev_info(rfdev_to_dev(dev), "IRDMA VSI Open Successful");
+ init_waitqueue_head(&iwdev->suspend_wq);
+
+ return 0;
+}
+
+/**
+ * irdma_close - client interface operation close for iwarp/uda device
+ * @ldev: lan device information
+ * @reason: reason for closing
+ *
+ * Called by the lan driver during the processing of client unregister
+ * Destroy and clean up the driver resources
+ */
+static void irdma_close(struct ice_peer_dev *ldev, enum ice_close_reason reason)
+{
+ struct irdma_device *iwdev;
+
+ iwdev = irdma_get_device(ldev->netdev);
+ if (!iwdev)
+ return;
+
+ irdma_put_device(iwdev);
+ if (reason == ICE_REASON_HW_RESET_PENDING) {
+ iwdev->reset = true;
+ iwdev->rf->reset = true;
+ }
+
+ if (iwdev->init_state >= CEQ0_CREATED)
+ irdma_deinit_rt_device(iwdev);
+
+ kfree(iwdev);
+ ldev->ops->update_vsi_filter(ldev, ICE_RDMA_FILTER_BOTH, false);
+ pr_info("IRDMA VSI close complete\n");
+}
+
+/**
+ * irdma_deinit_pf - Unrolls PF initializations done during irdma_probe()
+ * @rf: RDMA PCI function
+ */
+static void irdma_deinit_pf(struct irdma_pci_f *rf)
+{
+ if (rf->free_qp_wq)
+ destroy_workqueue(rf->free_qp_wq);
+ if (rf->free_cqbuf_wq)
+ destroy_workqueue(rf->free_cqbuf_wq);
+ irdma_deinit_ctrl_hw(rf);
+ irdma_del_handler(rf->hdl);
+ kfree(rf->hdl);
+}
+
+/**
+ * irdma_remove - GEN_2 device remove()
+ * @pdev: platform device
+ *
+ * Called on module unload.
+ */
+int irdma_remove(struct platform_device *pdev)
+{
+ struct ice_peer_dev *ldev = container_of(pdev, struct ice_peer_dev,
+ platform_dev);
+ struct irdma_handler *hdl;
+
+ hdl = irdma_find_handler(ldev->pdev);
+ if (!hdl)
+ return 0;
+
+ if (ldev->ops->peer_unregister)
+ ldev->ops->peer_unregister(ldev);
+
+ irdma_deinit_pf(&hdl->rf);
+ pr_info("IRDMA hardware deinitialization complete\n");
+
+ return 0;
+}
+
+static const struct ice_peer_ops irdma_peer_ops = {
+ .close = irdma_close,
+ .event_handler = irdma_event_handler,
+ .open = irdma_open,
+ .prep_tc_change = irdma_prep_tc_change,
+};
+
+static struct ice_peer_drv irdma_peer_drv = {
+ .driver_id = ICE_PEER_RDMA_DRIVER,
+ .name = KBUILD_MODNAME,
+ .ver.major = ICE_PEER_MAJOR_VER,
+ .ver.minor = ICE_PEER_MINOR_VER,
+};
+
+/**
+ * irdma_probe - GEN_2 device probe()
+ * @pdev: platform device
+ *
+ * Create device resources, set up queues, pble and hmc objects.
+ * Return 0 if successful, otherwise return error
+ */
+int irdma_probe(struct platform_device *pdev)
+{
+ struct ice_peer_dev *ldev = container_of(pdev, struct ice_peer_dev,
+ platform_dev);
+ struct irdma_handler *hdl;
+ struct irdma_pci_f *rf;
+ struct irdma_sc_dev *dev;
+ struct irdma_priv_ldev *pldev;
+ int ret;
+
+ pr_info("probe: ldev=%p, ldev->dev.pdev.bus->number=%d, ldev->netdev=%p\n",
+ ldev, ldev->pdev->bus->number, ldev->netdev);
+
+ if (ldev->ver.major != ICE_PEER_MAJOR_VER ||
+ ldev->ver.minor != ICE_PEER_MINOR_VER) {
+ pr_err("version mismatch:\n");
+ pr_err("expected major ver %d, caller specified major ver %d\n",
+ ICE_PEER_MAJOR_VER, ldev->ver.major);
+ pr_err("expected minor ver %d, caller specified minor ver %d\n",
+ ICE_PEER_MINOR_VER, ldev->ver.minor);
+ return -EINVAL;
+ }
+
+ hdl = irdma_find_handler(ldev->pdev);
+ if (hdl)
+ return -EBUSY;
+
+ if (!ldev->ops->peer_register)
+ return -EINVAL;
+
+ hdl = kzalloc(sizeof(*hdl), GFP_KERNEL);
+ if (!hdl)
+ return -ENOMEM;
+
+ rf = &hdl->rf;
+ pldev = &rf->ldev;
+ hdl->ldev = pldev;
+ hdl->platform_dev = pdev;
+ rf->hdl = hdl;
+ dev = &rf->sc_dev;
+ dev->back_dev = rf;
+ rf->init_hw = icrdma_init_hw;
+ pldev->if_ldev = ldev;
+ rf->rdma_ver = IRDMA_GEN_2;
+ irdma_init_rf_config_params(rf);
+ dev->pci_rev = ldev->pdev->revision;
+ rf->default_vsi.vsi_idx = ldev->pf_vsi_num;
+ /* save information from ldev to priv_ldev*/
+ pldev->fn_num = ldev->fn_num;
+ rf->hw.hw_addr = ldev->hw_addr;
+ rf->pdev = ldev->pdev;
+ rf->netdev = ldev->netdev;
+ pldev->ftype = ldev->ftype;
+ pldev->msix_count = ldev->msix_count;
+ pldev->msix_entries = ldev->msix_entries;
+ irdma_add_handler(hdl);
+ if (irdma_ctrl_init_hw(rf)) {
+ irdma_del_handler(hdl);
+ kfree(hdl);
+ return -EIO;
+ }
+ ldev->peer_ops = &irdma_peer_ops;
+ ldev->peer_drv = &irdma_peer_drv;
+ ret = ldev->ops->peer_register(ldev);
+ if (ret) {
+ irdma_deinit_pf(rf);
+ return ret;
+ }
+
+ return 0;
+}
+
+/**
+ * icrdma_request_reset - Request a reset
+ * @rf: RDMA PCI function
+ *
+ */
+void icrdma_request_reset(struct irdma_pci_f *rf)
+{
+ struct ice_peer_dev *ldev = (struct ice_peer_dev *)rf->ldev.if_ldev;
+
+ if (ldev && ldev->ops && ldev->ops->request_reset)
+ ldev->ops->request_reset(ldev, ICE_PEER_PFR);
+}
diff --git a/drivers/infiniband/hw/irdma/main.c b/drivers/infiniband/hw/irdma/main.c
new file mode 100644
index 000000000000..254e4a3dff31
--- /dev/null
+++ b/drivers/infiniband/hw/irdma/main.c
@@ -0,0 +1,531 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019, Intel Corporation. */
+
+#include "main.h"
+
+/* Legacy i40iw module parameters */
+static int resource_profile;
+module_param(resource_profile, int, 0644);
+MODULE_PARM_DESC(resource_profile, "Resource Profile: 0=PF only, 1=Weighted VF, 2=Even Distribution");
+
+static int max_rdma_vfs = 32;
+module_param(max_rdma_vfs, int, 0644);
+MODULE_PARM_DESC(max_rdma_vfs, "Maximum VF count: 0-32 32=default");
+
+static int mpa_version = 2;
+module_param(mpa_version, int, 0644);
+MODULE_PARM_DESC(mpa_version, "MPA version: deprecated parameter");
+
+static int push_mode;
+module_param(push_mode, int, 0644);
+MODULE_PARM_DESC(push_mode, "Low latency mode: deprecated parameter");
+
+static int debug;
+module_param(debug, int, 0644);
+MODULE_PARM_DESC(debug, "debug flags: deprecated parameter");
+
+MODULE_ALIAS("i40iw");
+MODULE_AUTHOR("Intel Corporation, <e1000-rdma@lists.sourceforge.net>");
+MODULE_DESCRIPTION("Intel(R) Ethernet Connection RDMA Driver");
+MODULE_LICENSE("Dual BSD/GPL");
+
+LIST_HEAD(irdma_handlers);
+DEFINE_SPINLOCK(irdma_handler_lock);
+
+static struct notifier_block irdma_inetaddr_notifier = {
+ .notifier_call = irdma_inetaddr_event
+};
+
+static struct notifier_block irdma_inetaddr6_notifier = {
+ .notifier_call = irdma_inet6addr_event
+};
+
+static struct notifier_block irdma_net_notifier = {
+ .notifier_call = irdma_net_event
+};
+
+static struct notifier_block irdma_netdevice_notifier = {
+ .notifier_call = irdma_netdevice_event
+};
+
+void irdma_init_rf_config_params(struct irdma_pci_f *rf)
+{
+ struct irdma_dl_priv *dl_priv;
+
+ rf->rsrc_profile = (resource_profile < IRDMA_HMC_PROFILE_EQUAL) ?
+ (u8)resource_profile + IRDMA_HMC_PROFILE_DEFAULT :
+ IRDMA_HMC_PROFILE_DEFAULT;
+ rf->max_rdma_vfs = (rf->rsrc_profile != IRDMA_HMC_PROFILE_DEFAULT) ?
+ max_rdma_vfs : 0;
+ rf->max_ena_vfs = rf->max_rdma_vfs;
+ dl_priv = platform_get_drvdata(rf->hdl->platform_dev);
+ rf->limits_sel = dl_priv->limits_sel;
+ rf->protocol_used = dl_priv->roce_ena ? IRDMA_ROCE_PROTOCOL_ONLY :
+ IRDMA_IWARP_PROTOCOL_ONLY;
+}
+
+/**
+ * irdma_get_device - find a iwdev given a netdev
+ * @netdev: pointer to net_device
+ *
+ * This function takes a reference on ibdev and prevents ib
+ * device deregistration. The caller must call a matching
+ * irdma_put_device.
+ */
+struct irdma_device *irdma_get_device(struct net_device *netdev)
+{
+ struct ib_device *ibdev = ib_device_get_by_netdev(netdev,
+ RDMA_DRIVER_I40IW);
+
+ if (!ibdev)
+ return NULL;
+
+ return to_iwdev(ibdev);
+}
+
+/**
+ * irdma_put_device - release ibdev refcnt
+ * @iwdev: device
+ *
+ * release refcnt on ibdev taken with irdma_get_device.
+ */
+void irdma_put_device(struct irdma_device *iwdev)
+{
+ struct ib_device *ibdev = &iwdev->iwibdev->ibdev;
+
+ ib_device_put(ibdev);
+}
+
+/**
+ * irdma_find_ice_handler - find a handler given a client info
+ * @pdev: pointer to pci dev info
+ */
+struct irdma_handler *irdma_find_handler(struct pci_dev *pdev)
+{
+ struct irdma_handler *hdl;
+ unsigned long flags;
+
+ spin_lock_irqsave(&irdma_handler_lock, flags);
+ list_for_each_entry(hdl, &irdma_handlers, list) {
+ if (hdl->rf.pdev->devfn == pdev->devfn &&
+ hdl->rf.pdev->bus->number == pdev->bus->number) {
+ spin_unlock_irqrestore(&irdma_handler_lock, flags);
+ return hdl;
+ }
+ }
+ spin_unlock_irqrestore(&irdma_handler_lock, flags);
+
+ return NULL;
+}
+
+/**
+ * irdma_add_handler - add a handler to the list
+ * @hdl: handler to be added to the handler list
+ */
+void irdma_add_handler(struct irdma_handler *hdl)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&irdma_handler_lock, flags);
+ list_add(&hdl->list, &irdma_handlers);
+ spin_unlock_irqrestore(&irdma_handler_lock, flags);
+}
+
+/**
+ * irdma_del_handler - delete a handler from the list
+ * @hdl: handler to be deleted from the handler list
+ */
+void irdma_del_handler(struct irdma_handler *hdl)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&irdma_handler_lock, flags);
+ list_del(&hdl->list);
+ spin_unlock_irqrestore(&irdma_handler_lock, flags);
+}
+
+/**
+ * irdma_register_notifiers - register tcp ip notifiers
+ */
+void irdma_register_notifiers(void)
+{
+ register_inetaddr_notifier(&irdma_inetaddr_notifier);
+ register_inet6addr_notifier(&irdma_inetaddr6_notifier);
+ register_netevent_notifier(&irdma_net_notifier);
+ register_netdevice_notifier(&irdma_netdevice_notifier);
+}
+
+void irdma_unregister_notifiers(void)
+{
+ unregister_netevent_notifier(&irdma_net_notifier);
+ unregister_inetaddr_notifier(&irdma_inetaddr_notifier);
+ unregister_inet6addr_notifier(&irdma_inetaddr6_notifier);
+ unregister_netdevice_notifier(&irdma_netdevice_notifier);
+}
+
+/**
+ * irdma_add_ipv6_addr - add ipv6 address to the hw arp table
+ * @iwdev: iwarp device
+ */
+static void irdma_add_ipv6_addr(struct irdma_device *iwdev)
+{
+ struct net_device *ip_dev;
+ struct inet6_dev *idev;
+ struct inet6_ifaddr *ifp, *tmp;
+ u32 local_ipaddr6[4];
+
+ rcu_read_lock();
+ for_each_netdev_rcu(&init_net, ip_dev) {
+ if (((rdma_vlan_dev_vlan_id(ip_dev) < 0xFFFF &&
+ rdma_vlan_dev_real_dev(ip_dev) == iwdev->netdev) ||
+ ip_dev == iwdev->netdev) && ip_dev->flags & IFF_UP) {
+ idev = __in6_dev_get(ip_dev);
+ if (!idev) {
+ dev_err(rfdev_to_dev(&iwdev->rf->sc_dev),
+ "ipv6 inet device not found\n");
+ break;
+ }
+ list_for_each_entry_safe(ifp, tmp, &idev->addr_list,
+ if_list) {
+ dev_info(rfdev_to_dev(&iwdev->rf->sc_dev),
+ "IP=%pI6, vlan_id=%d, MAC=%pM\n",
+ &ifp->addr,
+ rdma_vlan_dev_vlan_id(ip_dev),
+ ip_dev->dev_addr);
+
+ irdma_copy_ip_ntohl(local_ipaddr6,
+ ifp->addr.in6_u.u6_addr32);
+ irdma_manage_arp_cache(iwdev->rf,
+ ip_dev->dev_addr,
+ local_ipaddr6, false,
+ IRDMA_ARP_ADD);
+ }
+ }
+ }
+ rcu_read_unlock();
+}
+
+/**
+ * irdma_add_ipv4_addr - add ipv4 address to the hw arp table
+ * @iwdev: iwarp device
+ */
+static void irdma_add_ipv4_addr(struct irdma_device *iwdev)
+{
+ struct net_device *dev;
+ struct in_device *idev;
+ bool got_lock = true;
+ u32 ip_addr;
+
+ if (!rtnl_trylock())
+ got_lock = false;
+
+ for_each_netdev(&init_net, dev) {
+ if (((rdma_vlan_dev_vlan_id(dev) < 0xFFFF &&
+ rdma_vlan_dev_real_dev(dev) == iwdev->netdev) ||
+ dev == iwdev->netdev) && dev->flags & IFF_UP) {
+ idev = in_dev_get(dev);
+ for_ifa(idev)
+ {
+ dev_dbg(rfdev_to_dev(&iwdev->rf->sc_dev),
+ "CM: IP=%pI4, vlan_id=%d, MAC=%pM\n",
+ &ifa->ifa_address,
+ rdma_vlan_dev_vlan_id(dev),
+ dev->dev_addr);
+
+ ip_addr = ntohl(ifa->ifa_address);
+ irdma_manage_arp_cache(iwdev->rf, dev->dev_addr,
+ &ip_addr, true,
+ IRDMA_ARP_ADD);
+ }
+ endfor_ifa(idev);
+ in_dev_put(idev);
+ }
+ }
+ if (got_lock)
+ rtnl_unlock();
+}
+
+/**
+ * irdma_add_ip - add ip addresses
+ * @iwdev: iwarp device
+ *
+ * Add ipv4/ipv6 addresses to the arp cache
+ */
+void irdma_add_ip(struct irdma_device *iwdev)
+{
+ irdma_add_ipv4_addr(iwdev);
+ irdma_add_ipv6_addr(iwdev);
+}
+
+/**
+ * irdma_request_reset - Request a reset
+ * @rf: RDMA PCI function
+ *
+ */
+void irdma_request_reset(struct irdma_pci_f *rf)
+{
+ dev_warn(rfdev_to_dev(&rf->sc_dev),
+ "Requesting a a reset from LAN driver\n");
+ if (rf->rdma_ver == IRDMA_GEN_1)
+ i40iw_request_reset(rf);
+ else
+ icrdma_request_reset(rf);
+}
+
+static int irdma_devlink_rsrc_limits_validate(struct devlink *dl, u32 id,
+ union devlink_param_value val,
+ struct netlink_ext_ack *extack)
+{
+ u8 value = val.vu8;
+
+ if (value > 5) {
+ NL_SET_ERR_MSG_MOD(extack, "resource limits selector range is (0-5)");
+ return -ERANGE;
+ }
+
+ return 0;
+}
+
+enum irdma_dl_param_id {
+ IRDMA_DEVLINK_PARAM_ID_BASE = DEVLINK_PARAM_GENERIC_ID_MAX,
+ IRDMA_DEVLINK_PARAM_ID_LIMITS_SELECTOR,
+ IRDMA_DEVLINK_PARAM_ID_ROCE_ENABLE,
+};
+
+static const struct devlink_param irdma_devlink_params[] = {
+ /* Common */
+ DEVLINK_PARAM_DRIVER(IRDMA_DEVLINK_PARAM_ID_LIMITS_SELECTOR,
+ "resource_limits_selector", DEVLINK_PARAM_TYPE_U8,
+ BIT(DEVLINK_PARAM_CMODE_DRIVERINIT),
+ NULL, NULL, irdma_devlink_rsrc_limits_validate),
+#define IRDMA_DL_COMMON_PARAMS_ARRAY_SZ 1
+ /* GEN_2 only */
+ DEVLINK_PARAM_DRIVER(IRDMA_DEVLINK_PARAM_ID_ROCE_ENABLE,
+ "roce_enable", DEVLINK_PARAM_TYPE_BOOL,
+ BIT(DEVLINK_PARAM_CMODE_DRIVERINIT),
+ NULL, NULL, NULL),
+};
+
+static int irdma_devlink_reload(struct devlink *devlink,
+ struct netlink_ext_ack *extack)
+{
+ struct irdma_dl_priv *priv = devlink_priv(devlink);
+ union devlink_param_value saved_value;
+ const struct platform_device_id *id = platform_get_device_id(priv->pdev);
+ u8 gen_ver = id->driver_data;
+ int ret;
+
+ switch (gen_ver) {
+ case IRDMA_GEN_2:
+ irdma_remove(priv->pdev);
+ devlink_param_driverinit_value_get(devlink,
+ IRDMA_DEVLINK_PARAM_ID_ROCE_ENABLE,
+ &saved_value);
+ priv->roce_ena = saved_value.vbool;
+ devlink_param_driverinit_value_get(devlink,
+ IRDMA_DEVLINK_PARAM_ID_LIMITS_SELECTOR,
+ &saved_value);
+ priv->limits_sel = saved_value.vu8;
+ ret = irdma_probe(priv->pdev);
+ break;
+ case IRDMA_GEN_1:
+ i40iw_remove(priv->pdev);
+ devlink_param_driverinit_value_get(devlink,
+ IRDMA_DEVLINK_PARAM_ID_LIMITS_SELECTOR,
+ &saved_value);
+ priv->limits_sel = saved_value.vu8;
+ ret = i40iw_probe(priv->pdev);
+ break;
+ default:
+ ret = -ENODEV;
+ break;
+ }
+
+ return ret;
+}
+
+static const struct devlink_ops irdma_devlink_ops = {
+ .reload = irdma_devlink_reload,
+};
+
+static void irdma_devlink_unregister(struct platform_device *pdev)
+{
+ const struct platform_device_id *id = platform_get_device_id(pdev);
+ struct irdma_dl_priv *priv = platform_get_drvdata(pdev);
+ struct devlink *devlink = priv_to_devlink(priv);
+ u8 gen_ver = id->driver_data;
+
+ if (gen_ver == IRDMA_GEN_2)
+ devlink_params_unregister(devlink, irdma_devlink_params,
+ ARRAY_SIZE(irdma_devlink_params));
+ else if (gen_ver == IRDMA_GEN_1)
+ devlink_params_unregister(devlink, irdma_devlink_params,
+ IRDMA_DL_COMMON_PARAMS_ARRAY_SZ);
+
+ devlink_unregister(devlink);
+ devlink_free(devlink);
+}
+
+static int irdma_devlink_register(struct platform_device *pdev)
+{
+ const struct platform_device_id *id = platform_get_device_id(pdev);
+ u8 gen_ver = id->driver_data;
+ struct devlink *devlink;
+ struct irdma_dl_priv *priv;
+ union devlink_param_value value;
+ int ret;
+
+ devlink = devlink_alloc(&irdma_devlink_ops, sizeof(struct irdma_dl_priv));
+ if (!devlink)
+ return -ENOMEM;
+
+ priv = devlink_priv(devlink);
+ priv->pdev = pdev;
+ priv->roce_ena = 0;
+ platform_set_drvdata(pdev, priv);
+
+ ret = devlink_register(devlink, &pdev->dev);
+ if (ret)
+ goto err_dl_free;
+
+ switch (gen_ver) {
+ case IRDMA_GEN_2:
+ priv->limits_sel = 0;
+ ret = devlink_params_register(devlink, irdma_devlink_params,
+ ARRAY_SIZE(irdma_devlink_params));
+ if (!ret) {
+ value.vbool = priv->roce_ena;
+ devlink_param_driverinit_value_set(devlink,
+ IRDMA_DEVLINK_PARAM_ID_ROCE_ENABLE,
+ value);
+ }
+ break;
+ case IRDMA_GEN_1:
+ priv->limits_sel = 2;
+ ret = devlink_params_register(devlink, irdma_devlink_params,
+ IRDMA_DL_COMMON_PARAMS_ARRAY_SZ);
+ break;
+ default:
+ ret = -ENODEV;
+ break;
+ }
+
+ if (ret)
+ goto err_dl_unreg;
+
+ value.vu8 = priv->limits_sel;
+ devlink_param_driverinit_value_set(devlink,
+ IRDMA_DEVLINK_PARAM_ID_LIMITS_SELECTOR,
+ value);
+ devlink_params_publish(devlink);
+
+ return 0;
+
+err_dl_unreg:
+ devlink_unregister(devlink);
+err_dl_free:
+ devlink_free(devlink);
+
+ return ret;
+}
+
+static int irdma_bus_probe(struct platform_device *pdev)
+{
+ const struct platform_device_id *id = platform_get_device_id(pdev);
+ u8 gen_ver = id->driver_data;
+ int ret = -ENODEV;
+
+ ret = irdma_devlink_register(pdev);
+ if (ret)
+ return ret;
+
+ switch (gen_ver) {
+ case IRDMA_GEN_2:
+ ret = irdma_probe(pdev);
+ break;
+ case IRDMA_GEN_1:
+ ret = i40iw_probe(pdev);
+ break;
+ default:
+ break;
+ }
+
+ if (ret)
+ irdma_devlink_unregister(pdev);
+
+ return ret;
+}
+
+static int irdma_bus_remove(struct platform_device *pdev)
+{
+ const struct platform_device_id *id = platform_get_device_id(pdev);
+ u8 gen_ver = id->driver_data;
+
+ switch (gen_ver) {
+ case IRDMA_GEN_2:
+ irdma_remove(pdev);
+ break;
+ case IRDMA_GEN_1:
+ i40iw_remove(pdev);
+ break;
+ default:
+ break;
+ }
+
+ irdma_devlink_unregister(pdev);
+
+ return 0;
+}
+
+static const struct platform_device_id irdma_platform_id_table[] = {
+ {"ice_rdma", IRDMA_GEN_2},
+ {"i40e_rdma", IRDMA_GEN_1},
+ {},
+};
+
+MODULE_DEVICE_TABLE(platform, irdma_platform_id_table);
+
+static struct platform_driver irdma_pdriver = {
+ .probe = irdma_bus_probe,
+ .remove = irdma_bus_remove,
+ .id_table = irdma_platform_id_table,
+ .driver = {
+ .name = "irdma",
+ .owner = THIS_MODULE,
+ },
+};
+
+/**
+ * irdma_init_module - driver initialization function
+ *
+ * First function to call when the driver is loaded
+ * Register the driver as ice client and port mapper client
+ */
+static int __init irdma_init_module(void)
+{
+ int ret;
+
+ ret = platform_driver_register(&irdma_pdriver);
+ if (ret) {
+ pr_err("Failed irdma platform_driver_register()\n");
+ return ret;
+ }
+ irdma_register_notifiers();
+
+ return 0;
+}
+
+/**
+ * irdma_exit_module - driver exit clean up function
+ *
+ * The function is called just before the driver is unloaded
+ * Unregister the driver as ice client and port mapper client
+ */
+static void __exit irdma_exit_module(void)
+{
+ irdma_unregister_notifiers();
+ platform_driver_unregister(&irdma_pdriver);
+}
+
+module_init(irdma_init_module);
+module_exit(irdma_exit_module);
diff --git a/drivers/infiniband/hw/irdma/main.h b/drivers/infiniband/hw/irdma/main.h
new file mode 100644
index 000000000000..6e67594b4d9d
--- /dev/null
+++ b/drivers/infiniband/hw/irdma/main.h
@@ -0,0 +1,639 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2019, Intel Corporation. */
+
+#ifndef IRDMA_MAIN_H
+#define IRDMA_MAIN_H
+
+#include <linux/ip.h>
+#include <linux/tcp.h>
+#include <linux/if_vlan.h>
+#include <net/addrconf.h>
+#include <net/netevent.h>
+#include <net/devlink.h>
+#include <linux/netdevice.h>
+#include <linux/inetdevice.h>
+#include <linux/spinlock.h>
+#include <linux/kernel.h>
+#include <linux/delay.h>
+#include <linux/pci.h>
+#include <linux/dma-mapping.h>
+#include <linux/workqueue.h>
+#include <linux/slab.h>
+#include <linux/io.h>
+#include <linux/crc32c.h>
+#include <linux/kthread.h>
+#include <linux/platform_device.h>
+#include <rdma/ib_smi.h>
+#include <rdma/ib_verbs.h>
+#include <rdma/ib_pack.h>
+#include <rdma/rdma_cm.h>
+#include <rdma/iw_cm.h>
+#include <crypto/hash.h>
+#include "status.h"
+#include "osdep.h"
+#include "defs.h"
+#include "hmc.h"
+#include "type.h"
+#include "protos.h"
+#include "pble.h"
+#include "verbs.h"
+#include "cm.h"
+#include "user.h"
+#include "puda.h"
+#include <rdma/irdma-abi.h>
+
+extern struct list_head irdma_handlers;
+extern spinlock_t irdma_handler_lock;
+
+#define IRDMA_FW_VER_DEFAULT 2
+#define IRDMA_HW_VER 2
+
+#define IRDMA_ARP_ADD 1
+#define IRDMA_ARP_DELETE 2
+#define IRDMA_ARP_RESOLVE 3
+
+#define IRDMA_MACIP_ADD 1
+#define IRDMA_MACIP_DELETE 2
+
+#define IW_CCQ_SIZE (IRDMA_CQP_SW_SQSIZE_2048 + 1)
+#define IW_CEQ_SIZE 2048
+#define IW_AEQ_SIZE 2048
+
+#define RX_BUF_SIZE (1536 + 8)
+#define IW_REG0_SIZE (4 * 1024)
+#define IW_TX_TIMEOUT (6 * HZ)
+#define IW_FIRST_QPN 1
+
+#define IW_SW_CONTEXT_ALIGN 1024
+
+#define MAX_DPC_ITERATIONS 128
+
+#define IRDMA_EVENT_TIMEOUT 100000
+#define IRDMA_VCHNL_EVENT_TIMEOUT 100000
+
+#define IRDMA_NO_QSET 0xffff
+
+#define IW_CFG_FPM_QP_COUNT 32768
+#define IRDMA_MAX_PAGES_PER_FMR 512
+#define IRDMA_MIN_PAGES_PER_FMR 1
+#define IRDMA_CQP_COMPL_RQ_WQE_FLUSHED 2
+#define IRDMA_CQP_COMPL_SQ_WQE_FLUSHED 3
+
+#define IRDMA_Q_TYPE_PE_AEQ 0x80
+#define IRDMA_Q_INVALID_IDX 0xffff
+#define IRDMA_REM_ENDPOINT_TRK_QPID 3
+
+#define IRDMA_DRV_OPT_ENA_MPA_VER_0 0x00000001
+#define IRDMA_DRV_OPT_DISABLE_MPA_CRC 0x00000002
+#define IRDMA_DRV_OPT_DISABLE_FIRST_WRITE 0x00000004
+#define IRDMA_DRV_OPT_DISABLE_INTF 0x00000008
+#define IRDMA_DRV_OPT_ENA_MSI 0x00000010
+#define IRDMA_DRV_OPT_DUAL_LOGICAL_PORT 0x00000020
+#define IRDMA_DRV_OPT_NO_INLINE_DATA 0x00000080
+#define IRDMA_DRV_OPT_DISABLE_INT_MOD 0x00000100
+#define IRDMA_DRV_OPT_DISABLE_VIRT_WQ 0x00000200
+#define IRDMA_DRV_OPT_ENA_PAU 0x00000400
+#define IRDMA_DRV_OPT_MCAST_LOGPORT_MAP 0x00000800
+
+#define IW_HMC_OBJ_TYPE_NUM ARRAY_SIZE(iw_hmc_obj_types)
+
+enum init_completion_state {
+ INVALID_STATE = 0,
+ INITIAL_STATE,
+ CQP_CREATED,
+ HMC_OBJS_CREATED,
+ CCQ_CREATED,
+ AEQ_CREATED,
+ CEQ0_CREATED, /* Last state of probe */
+ CEQS_CREATED,
+ ILQ_CREATED,
+ IEQ_CREATED,
+ PBLE_CHUNK_MEM,
+ IP_ADDR_REGISTERED,
+ RDMA_DEV_REGISTERED, /* Last state of open */
+};
+
+struct irdma_rsrc_limits {
+ u32 qplimit;
+ u32 mrlimit;
+ u32 cqlimit;
+};
+
+struct irdma_cqp_compl_info {
+ u32 op_ret_val;
+ u16 maj_err_code;
+ u16 min_err_code;
+ bool error;
+ u8 op_code;
+};
+
+struct irdma_cqp_request {
+ struct cqp_cmds_info info;
+ wait_queue_head_t waitq;
+ struct list_head list;
+ atomic_t refcount;
+ void (*callback_fcn)(struct irdma_cqp_request *cqp_request);
+ void *param;
+ struct irdma_cqp_compl_info compl_info;
+ bool waiting;
+ bool request_done;
+ bool dynamic;
+};
+
+struct irdma_cqp {
+ struct irdma_sc_cqp sc_cqp;
+ spinlock_t req_lock; /* protect CQP request list */
+ spinlock_t compl_lock; /* protect CQP completion processing */
+ wait_queue_head_t waitq;
+ wait_queue_head_t remove_wq;
+ struct irdma_dma_mem sq;
+ struct irdma_dma_mem host_ctx;
+ u64 *scratch_array;
+ struct irdma_cqp_request *cqp_requests;
+ struct list_head cqp_avail_reqs;
+ struct list_head cqp_pending_reqs;
+ struct task_struct *cqp_compl_thread;
+ struct semaphore cqp_compl_sem;
+};
+
+struct irdma_ccq {
+ struct irdma_sc_cq sc_cq;
+ struct irdma_dma_mem mem_cq;
+ struct irdma_dma_mem shadow_area;
+};
+
+struct irdma_ceq {
+ struct irdma_sc_ceq sc_ceq;
+ struct irdma_dma_mem mem;
+ u32 irq;
+ u32 msix_idx;
+ struct irdma_pci_f *rf;
+ struct tasklet_struct dpc_tasklet;
+};
+
+struct irdma_aeq {
+ struct irdma_sc_aeq sc_aeq;
+ struct irdma_dma_mem mem;
+};
+
+struct irdma_arp_entry {
+ u32 ip_addr[4];
+ u8 mac_addr[ETH_ALEN];
+};
+
+struct irdma_msix_vector {
+ u32 idx;
+ u32 irq;
+ u32 cpu_affinity;
+ u32 ceq_id;
+ cpumask_t mask;
+};
+
+struct l2params_work {
+ struct work_struct work;
+ struct irdma_device *iwdev;
+ struct irdma_l2params l2params;
+};
+
+struct virtchnl_work {
+ struct work_struct work;
+ union {
+ struct irdma_cqp_request *cqp_request;
+ struct irdma_virtchnl_work_info work_info;
+ };
+};
+
+struct irdma_mc_table_info {
+ bool ipv4_valid;
+ u32 mgn;
+ u32 dest_ip[4];
+ bool lan_fwd;
+};
+
+struct mc_table_list {
+ struct list_head list;
+ struct irdma_mc_table_info mc_info;
+ struct irdma_mcast_grp_info mc_grp_ctx;
+};
+
+struct irdma_qv_info {
+ u32 v_idx; /* msix_vector */
+ u16 ceq_idx;
+ u16 aeq_idx;
+ u8 itr_idx;
+};
+
+struct irdma_qvlist_info {
+ u32 num_vectors;
+ struct irdma_qv_info qv_info[1];
+};
+
+struct irdma_priv_ldev {
+ unsigned int fn_num;
+ bool ftype;
+ u16 pf_vsi_num;
+ u16 msix_count;
+ struct msix_entry *msix_entries;
+ void *if_client;
+ void *if_ldev;
+};
+
+struct irdma_dl_priv {
+ struct platform_device *pdev;
+ bool roce_ena;
+ u8 limits_sel;
+};
+
+struct irdma_pci_f {
+ bool ooo;
+ bool reset;
+ bool rsrc_created;
+ bool stop_cqp_thread;
+ bool msix_shared;
+ u8 rsrc_profile;
+ u8 max_rdma_vfs;
+ u8 max_ena_vfs;
+ u8 *hmc_info_mem;
+ u8 *mem_rsrc;
+ u8 rdma_ver;
+ enum irdma_protocol_used protocol_used;
+ u32 sd_type;
+ u32 msix_count;
+ u32 max_mr;
+ u32 max_qp;
+ u32 max_cq;
+ u32 max_ah;
+ u32 next_ah;
+ u32 max_mcg;
+ u32 next_mcg;
+ u32 max_pd;
+ u32 next_qp;
+ u32 next_cq;
+ u32 next_pd;
+ u32 max_mr_size;
+ u32 max_cqe;
+ u32 mr_stagmask;
+ u32 used_pds;
+ u32 used_cqs;
+ u32 used_mrs;
+ u32 used_qps;
+ u32 arp_table_size;
+ u32 next_arp_index;
+ u32 ceqs_count;
+ u32 next_ws_node_id;
+ u32 max_ws_node_id;
+ u32 limits_sel;
+ unsigned long *allocated_ws_nodes;
+ unsigned long *allocated_qps;
+ unsigned long *allocated_cqs;
+ unsigned long *allocated_mrs;
+ unsigned long *allocated_pds;
+ unsigned long *allocated_mcgs;
+ unsigned long *allocated_ahs;
+ unsigned long *allocated_arps;
+ enum init_completion_state init_state;
+ struct irdma_sc_dev sc_dev;
+ struct irdma_priv_ldev ldev;
+ struct irdma_handler *hdl;
+ struct pci_dev *pdev;
+ struct net_device *netdev;
+ struct irdma_hw hw;
+ struct irdma_cqp cqp;
+ struct irdma_ccq ccq;
+ struct irdma_aeq aeq;
+ struct irdma_ceq *ceqlist;
+ struct irdma_hmc_pble_rsrc *pble_rsrc;
+ struct irdma_arp_entry *arp_table;
+ spinlock_t arp_lock; /*protect ARP table access*/
+ spinlock_t rsrc_lock; /* protect HW resource array access */
+ spinlock_t qptable_lock; /*protect QP table access*/
+ struct irdma_qp **qp_table;
+ spinlock_t qh_list_lock; /* protect mc_qht_list */
+ struct mc_table_list mc_qht_list;
+ struct irdma_msix_vector *iw_msixtbl;
+ struct irdma_qvlist_info *iw_qvlist;
+ struct tasklet_struct dpc_tasklet;
+ struct irdma_dma_mem obj_mem;
+ struct irdma_dma_mem obj_next;
+ atomic_t vchnl_msgs;
+ wait_queue_head_t vchnl_waitq;
+ struct workqueue_struct *free_qp_wq;
+ struct workqueue_struct *free_cqbuf_wq;
+ struct virtchnl_work virtchnl_w[IRDMA_MAX_PE_ENA_VF_COUNT];
+ struct irdma_sc_vsi default_vsi;
+ void *back_fcn;
+ void (*init_hw)(struct irdma_sc_dev *dev);
+};
+
+struct irdma_device {
+ struct irdma_ib_device *iwibdev;
+ struct irdma_pci_f *rf;
+ struct irdma_priv_ldev *ldev;
+ struct net_device *netdev;
+ struct irdma_handler *hdl;
+ struct irdma_sc_vsi vsi;
+ struct irdma_cm_core cm_core;
+ bool roce_mode;
+ u32 vendor_id;
+ u32 vendor_part_id;
+ u32 device_cap_flags;
+ u32 push_mode;
+ u32 rcv_wnd;
+ u16 mac_ip_table_idx;
+ u8 rcv_wscale;
+ u16 vsi_num;
+ bool create_ilq;
+ u8 iw_status;
+ struct tasklet_struct dpc_tasklet;
+ enum init_completion_state init_state;
+ bool dcb;
+ bool reset;
+ wait_queue_head_t suspend_wq;
+ struct workqueue_struct *param_wq;
+ atomic_t params_busy;
+};
+
+struct irdma_ib_device {
+ struct ib_device ibdev;
+ struct irdma_device *iwdev;
+};
+
+struct irdma_handler {
+ struct list_head list;
+ struct irdma_pci_f rf;
+ struct irdma_priv_ldev *ldev;
+ struct platform_device *platform_dev;
+ bool shared_res_created;
+};
+
+/***********************************************************/
+/**
+ * to_iwdev - get device
+ * @ibdev: ib device
+ **/
+static inline struct irdma_device *to_iwdev(struct ib_device *ibdev)
+{
+ return container_of(ibdev, struct irdma_ib_device, ibdev)->iwdev;
+}
+
+/**
+ * to_ucontext - get user context
+ * @ibucontext: ib user context
+ **/
+static inline struct irdma_ucontext *to_ucontext(struct ib_ucontext *ibucontext)
+{
+ return container_of(ibucontext, struct irdma_ucontext, ibucontext);
+}
+
+/**
+ * to_iwpd - get protection domain
+ * @ibpd: ib pd
+ **/
+static inline struct irdma_pd *to_iwpd(struct ib_pd *ibpd)
+{
+ return container_of(ibpd, struct irdma_pd, ibpd);
+}
+
+/**
+ * to_iwah - get device ah
+ * @ibdev: ib ah
+ **/
+static inline struct irdma_ah *to_iwah(struct ib_ah *ibah)
+{
+ return container_of(ibah, struct irdma_ah, ibah);
+}
+
+/**
+ * to_iwmr - get device memory region
+ * @ibdev: ib memory region
+ **/
+static inline struct irdma_mr *to_iwmr(struct ib_mr *ibmr)
+{
+ return container_of(ibmr, struct irdma_mr, ibmr);
+}
+
+/**
+ * to_iwmr_from_ibfmr - get device memory region
+ * @ibfmr: ib fmr
+ **/
+static inline struct irdma_mr *to_iwmr_from_ibfmr(struct ib_fmr *ibfmr)
+{
+ return container_of(ibfmr, struct irdma_mr, ibfmr);
+}
+
+/**
+ * to_iwmw - get device memory window
+ * @ibmw: ib memory window
+ **/
+static inline struct irdma_mr *to_iwmw(struct ib_mw *ibmw)
+{
+ return container_of(ibmw, struct irdma_mr, ibmw);
+}
+
+/**
+ * to_iwcq - get completion queue
+ * @ibcq: ib cqdevice
+ **/
+static inline struct irdma_cq *to_iwcq(struct ib_cq *ibcq)
+{
+ return container_of(ibcq, struct irdma_cq, ibcq);
+}
+
+/**
+ * to_iwqp - get device qp
+ * @ibqp: ib qp
+ **/
+static inline struct irdma_qp *to_iwqp(struct ib_qp *ibqp)
+{
+ return container_of(ibqp, struct irdma_qp, ibqp);
+}
+
+/**
+ * irdma_alloc_resource - allocate a resource
+ * @iwdev: device pointer
+ * @resource_array: resource bit array:
+ * @max_resources: maximum resource number
+ * @req_resources_num: Allocated resource number
+ * @next: next free id
+ **/
+static inline int irdma_alloc_rsrc(struct irdma_pci_f *rf,
+ unsigned long *rsrc_array, u32 max_rsrc,
+ u32 *req_rsrc_num, u32 *next)
+{
+ u32 rsrc_num;
+ unsigned long flags;
+
+ spin_lock_irqsave(&rf->rsrc_lock, flags);
+ rsrc_num = find_next_zero_bit(rsrc_array, max_rsrc, *next);
+ if (rsrc_num >= max_rsrc) {
+ rsrc_num = find_first_zero_bit(rsrc_array, max_rsrc);
+ if (rsrc_num >= max_rsrc) {
+ spin_unlock_irqrestore(&rf->rsrc_lock, flags);
+ dev_dbg(rfdev_to_dev(&rf->sc_dev),
+ "ERR: resource [%d] allocation failed\n",
+ rsrc_num);
+ return -EOVERFLOW;
+ }
+ }
+ set_bit(rsrc_num, rsrc_array);
+ *next = rsrc_num + 1;
+ if (*next == max_rsrc)
+ *next = 0;
+ *req_rsrc_num = rsrc_num;
+ spin_unlock_irqrestore(&rf->rsrc_lock, flags);
+
+ return 0;
+}
+
+/**
+ * irdma_is_resource_allocated - detrmine if resource is
+ * allocated
+ * @iwdev: device pointer
+ * @resource_array: resource array for the resource_num
+ * @resource_num: resource number to check
+ **/
+static inline bool irdma_is_rsrc_allocated(struct irdma_pci_f *rf,
+ unsigned long *rsrc_array,
+ u32 rsrc_num)
+{
+ bool bit_is_set;
+ unsigned long flags;
+
+ spin_lock_irqsave(&rf->rsrc_lock, flags);
+
+ bit_is_set = test_bit(rsrc_num, rsrc_array);
+ spin_unlock_irqrestore(&rf->rsrc_lock, flags);
+
+ return bit_is_set;
+}
+
+/**
+ * irdma_free_resource - free a resource
+ * @iwdev: device pointer
+ * @resource_array: resource array for the resource_num
+ * @resource_num: resource number to free
+ **/
+static inline void irdma_free_rsrc(struct irdma_pci_f *rf,
+ unsigned long *rsrc_array, u32 rsrc_num)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&rf->rsrc_lock, flags);
+ clear_bit(rsrc_num, rsrc_array);
+ spin_unlock_irqrestore(&rf->rsrc_lock, flags);
+}
+
+void irdma_init_rf_config_params(struct irdma_pci_f *rf);
+enum irdma_status_code irdma_ctrl_init_hw(struct irdma_pci_f *rf);
+void irdma_deinit_ctrl_hw(struct irdma_pci_f *rf);
+enum irdma_status_code irdma_rt_init_hw(struct irdma_pci_f *rf,
+ struct irdma_device *iwdev,
+ struct irdma_l2params *l2params);
+void irdma_deinit_rt_device(struct irdma_device *iwdev);
+void irdma_add_ref(struct ib_qp *ibqp);
+void irdma_rem_ref(struct ib_qp *ibqp);
+struct ib_qp *irdma_get_qp(struct ib_device *ibdev, int qpn);
+void irdma_flush_wqes(struct irdma_pci_f *rf, struct irdma_qp *qp);
+void irdma_manage_arp_cache(struct irdma_pci_f *rf, unsigned char *mac_addr,
+ u32 *ip_addr, bool ipv4, u32 action);
+int irdma_manage_apbvt(struct irdma_device *iwdev, u16 accel_local_port,
+ bool add_port);
+struct irdma_cqp_request *irdma_get_cqp_request(struct irdma_cqp *cqp,
+ bool wait);
+void irdma_free_cqp_request(struct irdma_cqp *cqp,
+ struct irdma_cqp_request *cqp_request);
+void irdma_put_cqp_request(struct irdma_cqp *cqp,
+ struct irdma_cqp_request *cqp_request);
+struct irdma_device *irdma_get_device(struct net_device *netdev);
+void irdma_put_device(struct irdma_device *iwdev);
+struct irdma_handler *irdma_find_handler(struct pci_dev *pdev);
+struct irdma_device *irdma_find_iwdev(const char *name);
+void irdma_add_handler(struct irdma_handler *hdl);
+void irdma_del_handler(struct irdma_handler *hdl);
+void irdma_add_ip(struct irdma_device *iwdev);
+int irdma_alloc_local_mac_entry(struct irdma_pci_f *rf, u16 *mac_tbl_idx);
+int irdma_add_local_mac_entry(struct irdma_pci_f *rf, u8 *mac_addr, u16 idx);
+void irdma_del_local_mac_entry(struct irdma_pci_f *rf, u16 idx);
+
+u32 irdma_initialize_hw_rsrc(struct irdma_pci_f *rf);
+int irdma_register_rdma_device(struct irdma_device *iwdev);
+void irdma_port_ibevent(struct irdma_device *iwdev);
+void irdma_cm_disconn(struct irdma_qp *qp);
+
+enum irdma_status_code
+irdma_handle_cqp_op(struct irdma_pci_f *rf,
+ struct irdma_cqp_request *cqp_request);
+
+int irdma_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, int attr_mask,
+ struct ib_udata *udata);
+int irdma_modify_qp_roce(struct ib_qp *ibqp, struct ib_qp_attr *attr,
+ int attr_mask, struct ib_udata *udata);
+void irdma_cq_wq_destroy(struct irdma_pci_f *rf, struct irdma_sc_cq *cq);
+
+void irdma_cleanup_pending_cqp_op(struct irdma_pci_f *rf);
+/* TODO: remove once VMWare implements or if not needed */
+enum irdma_status_code irdma_hw_modify_qp(struct irdma_device *iwdev,
+ struct irdma_qp *iwqp,
+ struct irdma_modify_qp_info *info,
+ bool wait);
+enum irdma_status_code irdma_qp_suspend_resume(struct irdma_sc_qp *qp,
+ bool suspend);
+enum irdma_status_code
+irdma_manage_qhash(struct irdma_device *iwdev, struct irdma_cm_info *cminfo,
+ enum irdma_quad_entry_type etype,
+ enum irdma_quad_hash_manage_type mtype, void *cmnode,
+ bool wait);
+void irdma_receive_ilq(struct irdma_sc_vsi *vsi, struct irdma_puda_buf *rbuf);
+void irdma_free_sqbuf(struct irdma_sc_vsi *vsi, void *bufp);
+void irdma_free_qp_rsrc(struct irdma_device *iwdev, struct irdma_qp *iwqp,
+ u32 qp_num);
+void irdma_request_reset(struct irdma_pci_f *rf);
+void irdma_destroy_rdma_device(struct irdma_ib_device *iwibdev);
+enum irdma_status_code irdma_setup_cm_core(struct irdma_device *iwdev, u8 ver);
+void irdma_cleanup_cm_core(struct irdma_cm_core *cm_core);
+void irdma_next_iw_state(struct irdma_qp *iwqp, u8 state, u8 del_hash, u8 term,
+ u8 term_len);
+int irdma_send_syn(struct irdma_cm_node *cm_node, u32 sendack);
+int irdma_send_reset(struct irdma_cm_node *cm_node);
+struct irdma_cm_node *irdma_find_node(struct irdma_cm_core *cm_core,
+ u16 rem_port, u32 *rem_addr, u16 loc_port,
+ u32 *loc_addr, bool add_refcnt,
+ bool accelerated_list);
+enum irdma_status_code irdma_hw_flush_wqes(struct irdma_pci_f *rf,
+ struct irdma_sc_qp *qp,
+ struct irdma_qp_flush_info *info,
+ bool wait);
+void irdma_gen_ae(struct irdma_pci_f *rf, struct irdma_sc_qp *qp,
+ struct irdma_gen_ae_info *info, bool wait);
+void irdma_copy_ip_ntohl(u32 *dst, __be32 *src);
+void irdma_copy_ip_htonl(__be32 *dst, u32 *src);
+u16 irdma_get_vlan_ipv4(u32 *addr);
+struct net_device *irdma_netdev_vlan_ipv6(u32 *addr, u16 *vlan_id, u8 *mac);
+struct ib_mr *irdma_reg_phys_mr(struct ib_pd *ib_pd, u64 addr, u64 size,
+ int acc, u64 *iova_start);
+int cqp_compl_thread(void *context);
+int irdma_inetaddr_event(struct notifier_block *notifier, unsigned long event,
+ void *ptr);
+int irdma_inet6addr_event(struct notifier_block *notifier, unsigned long event,
+ void *ptr);
+int irdma_net_event(struct notifier_block *notifier, unsigned long event,
+ void *ptr);
+int irdma_netdevice_event(struct notifier_block *notifier, unsigned long event,
+ void *ptr);
+int irdma_probe(struct platform_device *pdev);
+int irdma_remove(struct platform_device *pdev);
+int i40iw_probe(struct platform_device *pdev);
+void i40iw_remove(struct platform_device *pdev);
+void i40iw_request_reset(struct irdma_pci_f *rf);
+void icrdma_request_reset(struct irdma_pci_f *rf);
+void irdma_register_notifiers(void);
+void irdma_unregister_notifiers(void);
+void irdma_cqp_ce_handler(struct irdma_pci_f *rf, struct irdma_sc_cq *cq);
+int irdma_ah_cqp_op(struct irdma_pci_f *rf, struct irdma_sc_ah *sc_ah, u8 cmd,
+ bool wait,
+ void (*callback_fcn)(struct irdma_cqp_request *cqp_request),
+ void *cb_param);
+void irdma_gsi_ud_qp_ah_cb(struct irdma_cqp_request *cqp_request);
+int irdma_configfs_init(void);
+void irdma_configfs_exit(void);
+#endif /* IRDMA_MAIN_H */
--
2.21.0
^ permalink raw reply related
* [rdma 00/16] Intel RDMA Diver Updates 2019-07-03
From: Jeff Kirsher @ 2019-07-04 2:12 UTC (permalink / raw)
To: dledford, jgg, davem
Cc: Jeff Kirsher, linux-rdma, netdev, nhorman, sassmann, poswald,
mustafa.ismail, shiraz.saleem, david.m.ertman
This patchset adds a unified Intel Ethernet RDMA driver that
supports a new network device E810 (iWARP and RoCEv2 capable)
and the existing X722 iWARP device. The driver architecture
provides the extensibility for future generations of Intel HW
supporting RDMA.
This driver obsoletes legacy X722 driver i40iw which is marked for
deprecation and extends the ABI already defined for i40iw. It is
backward compatible with legacy X722 rdma-core provider (libi40iw).
This series was built against the rdma for-next branch. These changes
are dependent upon changes to i40e and ice driver that are being
submitted to netdev for David Miller's net-next since the LAN drivers in
the RDMA tree are not up-to-date, patch against those drivers would
cause large conflicts with what is currently in Dave's net-next. See
the git tree URL reference for the 3 driver patches which are required
for these changes.
RFC --> v0:
* Rehashed the design to unify RDMA driver. irdma is registered as a
platform driver capable of supporting RDMA capable devices added to
the virtual platform bus by their individual netdev drivers i40e/ice.
Listening to netdev notifiers or running netdev lists are no longer
needed for attachment. There is no load order dependencies between
netdev drivers and irdma in the new model.
MFD architecture was also considered, and we selected the simpler
platform model. Supporting a MFD architecture would require an
additional MFD core driver, individual platform netdev, RDMA function
drivers, and stripping a large portion of the netdev drivers into
MFD core. The sub-devices registered by MFD core for function
drivers are indeed platform devices.
*Use netdev_to_ibdev API to reliably get iwdev in notifiers.
Remove VSI dev list tracking as a result.
*Fixed build make W=1 issues, sparse endianness warnings, 0-day
32-bit compile warnings.
*Test for userspaceness with udata and remove uobject references.
*Remove abstractions for memory allocators, dev_* and pr_* prints.
*Remove redundant castings in the driver.
*Relax barriers to a dma_wmb()/dma_rmb() since we are using coherent
mappings.
*Clang-format run on various portions of the driver.
*Remove internal verb objects tracking from driver as its already done
in IB core.
*Report correct values for max_send_wr and max_recv_wr in irdma_query_qp()
*Check and fail the call for invalid input values on irdma_create_qp().
*ABI fixups - __aligned_u64 on all u64s. Fix travis hit and removed
irdma_hw_attrs struct out of ABI.
*Use IRDMA_RING_MOVE_HEAD_NO_CHECK on cq_ring
*Sort call tables, Kconfig, Makefiles
*Add CQ resize feature and few fixes since RFC was published.
*Use same DRIVER_ID enum for irdma as i40iw.
*Updated ib_copy_from_udata/ib_copy_to_udata calls to do a safe copy()
*Adapt to core handling verb object allocations and other core API changes
*devlink is used to switch between RoCE and iWARP on a per function basis
The following are changes since commit 10dcc7448e9ea49488a38bca7551de1a9da06ad9:
RDMA/hns: fix spelling mistake "attatch" -> "attach"
and are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/rdma for-next
!!! These commits are dependent upon changes to the i40e and ice driver,
which have been submitted to netdev. They are available at:
git://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/next-queue 100GbE
Michael J. Ruhl (1):
RDMA/irdma: Add dynamic tracing for CM
Mustafa Ismail (13):
RDMA/irdma: Add driver framework definitions
RDMA/irdma: Implement device initialization definitions
RDMA/irdma: Implement HW Admin Queue OPs
RDMA/irdma: Add HMC backing store setup functions
RDMA/irdma: Add privileged UDA queue implementation
RDMA/irdma: Add QoS definitions
RDMA/irdma: Add connection manager
RDMA/irdma: Add PBLE resource manager
RDMA/irdma: Implement device supported verb APIs
RDMA/irdma: Add RoCEv2 UD OP support
RDMA/irdma: Add user/kernel shared libraries
RDMA/irdma: Add miscellaneous utility definitions
RDMA/irdma: Add ABI definitions
Shiraz Saleem (2):
RDMA/irdma: Update MAINTAINERS file
RDMA/irdma: Add Kconfig and Makefile
MAINTAINERS | 10 +-
drivers/infiniband/Kconfig | 1 +
drivers/infiniband/hw/Makefile | 1 +
drivers/infiniband/hw/i40iw/Kconfig | 4 +-
drivers/infiniband/hw/irdma/Kconfig | 11 +
drivers/infiniband/hw/irdma/Makefile | 31 +
drivers/infiniband/hw/irdma/cm.c | 4514 ++++++++++++++++
drivers/infiniband/hw/irdma/cm.h | 415 ++
drivers/infiniband/hw/irdma/ctrl.c | 5958 ++++++++++++++++++++++
drivers/infiniband/hw/irdma/defs.h | 2126 ++++++++
drivers/infiniband/hw/irdma/hmc.c | 706 +++
drivers/infiniband/hw/irdma/hmc.h | 219 +
drivers/infiniband/hw/irdma/hw.c | 2563 ++++++++++
drivers/infiniband/hw/irdma/i40iw_hw.c | 210 +
drivers/infiniband/hw/irdma/i40iw_hw.h | 163 +
drivers/infiniband/hw/irdma/i40iw_if.c | 256 +
drivers/infiniband/hw/irdma/icrdma_hw.c | 75 +
drivers/infiniband/hw/irdma/icrdma_hw.h | 63 +
drivers/infiniband/hw/irdma/irdma.h | 191 +
drivers/infiniband/hw/irdma/irdma_if.c | 426 ++
drivers/infiniband/hw/irdma/main.c | 531 ++
drivers/infiniband/hw/irdma/main.h | 639 +++
drivers/infiniband/hw/irdma/osdep.h | 108 +
drivers/infiniband/hw/irdma/pble.c | 511 ++
drivers/infiniband/hw/irdma/pble.h | 136 +
drivers/infiniband/hw/irdma/protos.h | 96 +
drivers/infiniband/hw/irdma/puda.c | 1693 ++++++
drivers/infiniband/hw/irdma/puda.h | 187 +
drivers/infiniband/hw/irdma/status.h | 70 +
drivers/infiniband/hw/irdma/trace.c | 113 +
drivers/infiniband/hw/irdma/trace.h | 4 +
drivers/infiniband/hw/irdma/trace_cm.h | 459 ++
drivers/infiniband/hw/irdma/type.h | 1701 ++++++
drivers/infiniband/hw/irdma/uda.c | 391 ++
drivers/infiniband/hw/irdma/uda.h | 65 +
drivers/infiniband/hw/irdma/uda_d.h | 383 ++
drivers/infiniband/hw/irdma/uk.c | 1739 +++++++
drivers/infiniband/hw/irdma/user.h | 451 ++
drivers/infiniband/hw/irdma/utils.c | 2333 +++++++++
drivers/infiniband/hw/irdma/verbs.c | 4347 ++++++++++++++++
drivers/infiniband/hw/irdma/verbs.h | 199 +
drivers/infiniband/hw/irdma/ws.c | 396 ++
drivers/infiniband/hw/irdma/ws.h | 40 +
include/uapi/rdma/irdma-abi.h | 130 +
include/uapi/rdma/rdma_user_ioctl_cmds.h | 1 +
45 files changed, 34664 insertions(+), 2 deletions(-)
create mode 100644 drivers/infiniband/hw/irdma/Kconfig
create mode 100644 drivers/infiniband/hw/irdma/Makefile
create mode 100644 drivers/infiniband/hw/irdma/cm.c
create mode 100644 drivers/infiniband/hw/irdma/cm.h
create mode 100644 drivers/infiniband/hw/irdma/ctrl.c
create mode 100644 drivers/infiniband/hw/irdma/defs.h
create mode 100644 drivers/infiniband/hw/irdma/hmc.c
create mode 100644 drivers/infiniband/hw/irdma/hmc.h
create mode 100644 drivers/infiniband/hw/irdma/hw.c
create mode 100644 drivers/infiniband/hw/irdma/i40iw_hw.c
create mode 100644 drivers/infiniband/hw/irdma/i40iw_hw.h
create mode 100644 drivers/infiniband/hw/irdma/i40iw_if.c
create mode 100644 drivers/infiniband/hw/irdma/icrdma_hw.c
create mode 100644 drivers/infiniband/hw/irdma/icrdma_hw.h
create mode 100644 drivers/infiniband/hw/irdma/irdma.h
create mode 100644 drivers/infiniband/hw/irdma/irdma_if.c
create mode 100644 drivers/infiniband/hw/irdma/main.c
create mode 100644 drivers/infiniband/hw/irdma/main.h
create mode 100644 drivers/infiniband/hw/irdma/osdep.h
create mode 100644 drivers/infiniband/hw/irdma/pble.c
create mode 100644 drivers/infiniband/hw/irdma/pble.h
create mode 100644 drivers/infiniband/hw/irdma/protos.h
create mode 100644 drivers/infiniband/hw/irdma/puda.c
create mode 100644 drivers/infiniband/hw/irdma/puda.h
create mode 100644 drivers/infiniband/hw/irdma/status.h
create mode 100644 drivers/infiniband/hw/irdma/trace.c
create mode 100644 drivers/infiniband/hw/irdma/trace.h
create mode 100644 drivers/infiniband/hw/irdma/trace_cm.h
create mode 100644 drivers/infiniband/hw/irdma/type.h
create mode 100644 drivers/infiniband/hw/irdma/uda.c
create mode 100644 drivers/infiniband/hw/irdma/uda.h
create mode 100644 drivers/infiniband/hw/irdma/uda_d.h
create mode 100644 drivers/infiniband/hw/irdma/uk.c
create mode 100644 drivers/infiniband/hw/irdma/user.h
create mode 100644 drivers/infiniband/hw/irdma/utils.c
create mode 100644 drivers/infiniband/hw/irdma/verbs.c
create mode 100644 drivers/infiniband/hw/irdma/verbs.h
create mode 100644 drivers/infiniband/hw/irdma/ws.c
create mode 100644 drivers/infiniband/hw/irdma/ws.h
create mode 100644 include/uapi/rdma/irdma-abi.h
--
2.21.0
^ permalink raw reply
* [rdma 1/1] RDMA/irdma: Add Kconfig and Makefile
From: Jeff Kirsher @ 2019-07-04 2:12 UTC (permalink / raw)
To: dledford, jgg, davem
Cc: Shiraz Saleem, linux-rdma, netdev, nhorman, sassmann, poswald,
david.m.ertman, mustafa.ismail
In-Reply-To: <20190704021259.15489-1-jeffrey.t.kirsher@intel.com>
From: Shiraz Saleem <shiraz.saleem@intel.com>
Add Kconfig and Makefile to build irdma driver and mark i40iw
deprecated/obsolete, since the irdma driver is replacing it and supports
x722 devices.
Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
---
drivers/infiniband/Kconfig | 1 +
drivers/infiniband/hw/Makefile | 1 +
drivers/infiniband/hw/i40iw/Kconfig | 4 +++-
drivers/infiniband/hw/irdma/Kconfig | 11 ++++++++++
drivers/infiniband/hw/irdma/Makefile | 31 ++++++++++++++++++++++++++++
5 files changed, 47 insertions(+), 1 deletion(-)
create mode 100644 drivers/infiniband/hw/irdma/Kconfig
create mode 100644 drivers/infiniband/hw/irdma/Makefile
diff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig
index 0fe6f76e8fdc..40b032a764f5 100644
--- a/drivers/infiniband/Kconfig
+++ b/drivers/infiniband/Kconfig
@@ -84,6 +84,7 @@ source "drivers/infiniband/hw/cxgb3/Kconfig"
source "drivers/infiniband/hw/cxgb4/Kconfig"
source "drivers/infiniband/hw/efa/Kconfig"
source "drivers/infiniband/hw/i40iw/Kconfig"
+source "drivers/infiniband/hw/irdma/Kconfig"
source "drivers/infiniband/hw/mlx4/Kconfig"
source "drivers/infiniband/hw/mlx5/Kconfig"
source "drivers/infiniband/hw/ocrdma/Kconfig"
diff --git a/drivers/infiniband/hw/Makefile b/drivers/infiniband/hw/Makefile
index 433fca59febd..d61d690ec0d4 100644
--- a/drivers/infiniband/hw/Makefile
+++ b/drivers/infiniband/hw/Makefile
@@ -5,6 +5,7 @@ obj-$(CONFIG_INFINIBAND_CXGB3) += cxgb3/
obj-$(CONFIG_INFINIBAND_CXGB4) += cxgb4/
obj-$(CONFIG_INFINIBAND_EFA) += efa/
obj-$(CONFIG_INFINIBAND_I40IW) += i40iw/
+obj-$(CONFIG_INFINIBAND_IRDMA) += irdma/
obj-$(CONFIG_MLX4_INFINIBAND) += mlx4/
obj-$(CONFIG_MLX5_INFINIBAND) += mlx5/
obj-$(CONFIG_INFINIBAND_OCRDMA) += ocrdma/
diff --git a/drivers/infiniband/hw/i40iw/Kconfig b/drivers/infiniband/hw/i40iw/Kconfig
index d867ef1ac72a..7454b84b74be 100644
--- a/drivers/infiniband/hw/i40iw/Kconfig
+++ b/drivers/infiniband/hw/i40iw/Kconfig
@@ -1,8 +1,10 @@
config INFINIBAND_I40IW
- tristate "Intel(R) Ethernet X722 iWARP Driver"
+ tristate "Intel(R) Ethernet X722 iWARP Driver (DEPRECATED)"
depends on INET && I40E
depends on IPV6 || !IPV6
depends on PCI
+ depends on !(INFINBAND_IRDMA=y || INFINIBAND_IRDMA=m)
select GENERIC_ALLOCATOR
---help---
Intel(R) Ethernet X722 iWARP Driver
+ This driver is being replaced by irdma.
diff --git a/drivers/infiniband/hw/irdma/Kconfig b/drivers/infiniband/hw/irdma/Kconfig
new file mode 100644
index 000000000000..652f5f978ce2
--- /dev/null
+++ b/drivers/infiniband/hw/irdma/Kconfig
@@ -0,0 +1,11 @@
+config INFINIBAND_IRDMA
+ tristate "Intel(R) Ethernet Connection RDMA Driver"
+ depends on INET && (I40E || ICE)
+ depends on IPV6 || !IPV6
+ depends on PCI
+ select GENERIC_ALLOCATOR
+ ---help---
+ This is an Ethernet RDMA driver that supports E810 (iWARP/RoCE)
+ and X722 (iWARP) network devices.
+ To compile this driver as a module, choose M here. The module
+ will be called irdma.
diff --git a/drivers/infiniband/hw/irdma/Makefile b/drivers/infiniband/hw/irdma/Makefile
new file mode 100644
index 000000000000..455940d7cc69
--- /dev/null
+++ b/drivers/infiniband/hw/irdma/Makefile
@@ -0,0 +1,31 @@
+# SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+# Copyright (c) 2019, Intel Corporation.
+
+#
+# Makefile for the Intel(R) Ethernet Connection RDMA Linux Driver
+#
+
+ccflags-y := -I $(srctree)/drivers/net/ethernet/intel/i40e
+ccflags-y += -I $(srctree)/drivers/net/ethernet/intel/ice
+
+obj-$(CONFIG_INFINIBAND_IRDMA) += irdma.o
+
+irdma-objs := cm.o \
+ ctrl.o \
+ hmc.o \
+ hw.o \
+ i40iw_hw.o \
+ i40iw_if.o \
+ icrdma_hw.o \
+ irdma_if.o \
+ main.o \
+ pble.o \
+ puda.o \
+ trace.o \
+ uda.o \
+ uk.o \
+ utils.o \
+ verbs.o \
+ ws.o \
+
+CFLAGS_trace.o = -I$(src)
--
2.21.0
^ permalink raw reply related
* [rdma 14/16] RDMA/irdma: Add ABI definitions
From: Jeff Kirsher @ 2019-07-04 2:12 UTC (permalink / raw)
To: dledford, jgg, davem
Cc: Mustafa Ismail, linux-rdma, netdev, nhorman, sassmann, poswald,
david.m.ertman, Shiraz Saleem
In-Reply-To: <20190704021259.15489-1-jeffrey.t.kirsher@intel.com>
From: Mustafa Ismail <mustafa.ismail@intel.com>
Add ABI definitions for irdma.
Signed-off-by: Mustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
---
include/uapi/rdma/irdma-abi.h | 130 ++++++++++++++++++++++++++++++++++
1 file changed, 130 insertions(+)
create mode 100644 include/uapi/rdma/irdma-abi.h
diff --git a/include/uapi/rdma/irdma-abi.h b/include/uapi/rdma/irdma-abi.h
new file mode 100644
index 000000000000..bdfbda4c829e
--- /dev/null
+++ b/include/uapi/rdma/irdma-abi.h
@@ -0,0 +1,130 @@
+/* SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause */
+/* Copyright (c) 2006 - 2019 Intel Corporation. All rights reserved.
+ * Copyright (c) 2005 Topspin Communications. All rights reserved.
+ * Copyright (c) 2005 Cisco Systems. All rights reserved.
+ * Copyright (c) 2005 Open Grid Computing, Inc. All rights reserved.
+ */
+
+#ifndef IRDMA_ABI_H
+#define IRDMA_ABI_H
+
+#include <linux/types.h>
+
+/* irdma must support legacy GEN_1 i40iw kernel
+ * and user-space whose last ABI ver is 5
+ */
+#define IRDMA_ABI_VER 6
+
+enum irdma_memreg_type {
+ IW_MEMREG_TYPE_MEM = 0,
+ IW_MEMREG_TYPE_QP = 1,
+ IW_MEMREG_TYPE_CQ = 2,
+ IW_MEMREG_TYPE_RSVD = 3,
+ IW_MEMREG_TYPE_MW = 4,
+};
+
+struct irdma_alloc_ucontext_req {
+ __u32 rsvd32;
+ __u8 userspace_ver;
+ __u8 rsvd8[3];
+};
+
+struct i40iw_alloc_ucontext_req {
+ __u32 rsvd32;
+ __u8 userspace_ver;
+ __u8 rsvd8[3];
+};
+
+struct irdma_alloc_ucontext_resp {
+ __aligned_u64 feature_flags;
+ __u32 max_hw_wq_frags;
+ __u32 max_hw_read_sges;
+ __u32 max_hw_inline;
+ __u32 max_hw_rq_quanta;
+ __u32 max_hw_wq_quanta;
+ __u32 min_hw_cq_size;
+ __u32 max_hw_cq_size;
+ __u32 rsvd1[7];
+ __u16 max_hw_sq_chunk;
+ __u16 rsvd2[11];
+ __u8 kernel_ver;
+ __u8 hw_rev;
+ __u8 rsvd3[6];
+};
+
+struct i40iw_alloc_ucontext_resp {
+ __u32 max_pds;
+ __u32 max_qps;
+ __u32 wq_size; /* size of the WQs (SQ+RQ) in the mmaped area */
+ __u8 kernel_ver;
+ __u8 rsvd[3];
+};
+
+struct irdma_alloc_pd_resp {
+ __u32 pd_id;
+ __u8 rsvd[4];
+};
+
+struct irdma_resize_cq_req {
+ __aligned_u64 user_cq_buffer;
+};
+
+struct irdma_create_cq_req {
+ __aligned_u64 user_cq_buf;
+ __aligned_u64 user_shadow_area;
+};
+
+struct irdma_create_qp_req {
+ __aligned_u64 user_wqe_bufs;
+ __aligned_u64 user_compl_ctx;
+};
+
+struct i40iw_create_qp_req {
+ __aligned_u64 user_wqe_bufs;
+ __aligned_u64 user_compl_ctx;
+};
+
+struct irdma_mem_reg_req {
+ __u16 reg_type; /* Memory, QP or CQ */
+ __u16 cq_pages;
+ __u16 rq_pages;
+ __u16 sq_pages;
+};
+
+struct irdma_create_cq_resp {
+ __u32 cq_id;
+ __u32 cq_size;
+};
+
+struct irdma_create_qp_resp {
+ __u32 qp_id;
+ __u32 actual_sq_size;
+ __u32 actual_rq_size;
+ __u32 irdma_drv_opt;
+ __u32 qp_caps;
+ __u16 rsvd1;
+ __u8 lsmm;
+ __u8 rsvd2;
+};
+
+struct i40iw_create_qp_resp {
+ __u32 qp_id;
+ __u32 actual_sq_size;
+ __u32 actual_rq_size;
+ __u32 i40iw_drv_opt;
+ __u16 push_idx;
+ __u8 lsmm;
+ __u8 rsvd;
+};
+
+struct irdma_modify_qp_resp {
+ __u16 push_idx;
+ __u16 push_offset;
+ __u8 rsvd[4];
+};
+
+struct irdma_create_ah_resp {
+ __u32 ah_id;
+ __u8 rsvd[4];
+};
+#endif /* IRDMA_ABI_H */
--
2.21.0
^ permalink raw reply related
* [rdma 08/16] RDMA/irdma: Add PBLE resource manager
From: Jeff Kirsher @ 2019-07-04 2:12 UTC (permalink / raw)
To: dledford, jgg, davem
Cc: Mustafa Ismail, linux-rdma, netdev, nhorman, sassmann, poswald,
david.m.ertman, Shiraz Saleem
In-Reply-To: <20190704021259.15489-1-jeffrey.t.kirsher@intel.com>
From: Mustafa Ismail <mustafa.ismail@intel.com>
Implement a Physical Buffer List Entry (PBLE) resource manager
to manage a pool of PBLE HMC resource objects.
Signed-off-by: Mustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
---
drivers/infiniband/hw/irdma/pble.c | 511 +++++++++++++++++++++++++++++
drivers/infiniband/hw/irdma/pble.h | 136 ++++++++
2 files changed, 647 insertions(+)
create mode 100644 drivers/infiniband/hw/irdma/pble.c
create mode 100644 drivers/infiniband/hw/irdma/pble.h
diff --git a/drivers/infiniband/hw/irdma/pble.c b/drivers/infiniband/hw/irdma/pble.c
new file mode 100644
index 000000000000..cbf3a01d9c63
--- /dev/null
+++ b/drivers/infiniband/hw/irdma/pble.c
@@ -0,0 +1,511 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019, Intel Corporation. */
+
+#include "osdep.h"
+#include "status.h"
+#include "hmc.h"
+#include "defs.h"
+#include "type.h"
+#include "protos.h"
+#include "pble.h"
+
+static enum irdma_status_code
+add_pble_prm(struct irdma_hmc_pble_rsrc *pble_rsrc);
+
+/**
+ * irdma_destroy_pble_prm - destroy prm during module unload
+ * @pble_rsrc: pble resources
+ */
+void irdma_destroy_pble_prm(struct irdma_hmc_pble_rsrc *pble_rsrc)
+{
+ struct irdma_chunk *chunk;
+ struct irdma_pble_prm *pinfo = &pble_rsrc->pinfo;
+
+ while (!list_empty(&pinfo->clist)) {
+ chunk = (struct irdma_chunk *) pinfo->clist.next;
+ list_del(&chunk->list);
+ if (chunk->type == PBLE_SD_PAGED)
+ irdma_pble_free_paged_mem(chunk);
+ if (chunk->bitmapbuf)
+ kfree(chunk->bitmapmem.va);
+ kfree(chunk->chunkmem.va);
+ }
+}
+
+/**
+ * irdma_hmc_init_pble - Initialize pble resources during module load
+ * @dev: irdma_sc_dev struct
+ * @pble_rsrc: pble resources
+ */
+enum irdma_status_code
+irdma_hmc_init_pble(struct irdma_sc_dev *dev,
+ struct irdma_hmc_pble_rsrc *pble_rsrc)
+{
+ struct irdma_hmc_info *hmc_info;
+ u32 fpm_idx = 0;
+ enum irdma_status_code status = 0;
+
+ hmc_info = dev->hmc_info;
+ pble_rsrc->dev = dev;
+ pble_rsrc->fpm_base_addr = hmc_info->hmc_obj[IRDMA_HMC_IW_PBLE].base;
+ /* Start pble' on 4k boundary */
+ if (pble_rsrc->fpm_base_addr & 0xfff)
+ fpm_idx = (4096 - (pble_rsrc->fpm_base_addr & 0xfff)) >> 3;
+ pble_rsrc->unallocated_pble =
+ hmc_info->hmc_obj[IRDMA_HMC_IW_PBLE].cnt - fpm_idx;
+ pble_rsrc->next_fpm_addr = pble_rsrc->fpm_base_addr + (fpm_idx << 3);
+ pble_rsrc->pinfo.pble_shift = PBLE_SHIFT;
+
+ spin_lock_init(&pble_rsrc->pinfo.prm_lock);
+ INIT_LIST_HEAD(&pble_rsrc->pinfo.clist);
+ if (add_pble_prm(pble_rsrc)) {
+ irdma_destroy_pble_prm(pble_rsrc);
+ status = IRDMA_ERR_NO_MEMORY;
+ }
+
+ return status;
+}
+
+/**
+ * get_sd_pd_idx - Returns sd index, pd index and rel_pd_idx from fpm address
+ * @pble_rsrc: structure containing fpm address
+ * @idx: where to return indexes
+ */
+static void get_sd_pd_idx(struct irdma_hmc_pble_rsrc *pble_rsrc,
+ struct sd_pd_idx *idx)
+{
+ idx->sd_idx = (u32)pble_rsrc->next_fpm_addr / IRDMA_HMC_DIRECT_BP_SIZE;
+ idx->pd_idx = (u32)(pble_rsrc->next_fpm_addr / IRDMA_HMC_PAGED_BP_SIZE);
+ idx->rel_pd_idx = (idx->pd_idx % IRDMA_HMC_PD_CNT_IN_SD);
+}
+
+/**
+ * add_sd_direct - add sd direct for pble
+ * @pble_rsrc: pble resource ptr
+ * @info: page info for sd
+ */
+static enum irdma_status_code
+add_sd_direct(struct irdma_hmc_pble_rsrc *pble_rsrc,
+ struct irdma_add_page_info *info)
+{
+ struct irdma_sc_dev *dev = pble_rsrc->dev;
+ enum irdma_status_code ret_code = 0;
+ struct sd_pd_idx *idx = &info->idx;
+ struct irdma_chunk *chunk = info->chunk;
+ struct irdma_hmc_info *hmc_info = info->hmc_info;
+ struct irdma_hmc_sd_entry *sd_entry = info->sd_entry;
+ u32 offset = 0;
+
+ if (!sd_entry->valid) {
+ ret_code = irdma_add_sd_table_entry(dev->hw, hmc_info,
+ info->idx.sd_idx,
+ IRDMA_SD_TYPE_DIRECT,
+ IRDMA_HMC_DIRECT_BP_SIZE);
+ if (ret_code)
+ return ret_code;
+
+ chunk->type = PBLE_SD_CONTIGOUS;
+ }
+
+ offset = idx->rel_pd_idx << HMC_PAGED_BP_SHIFT;
+ chunk->size = info->pages << HMC_PAGED_BP_SHIFT;
+ chunk->vaddr = (uintptr_t)(sd_entry->u.bp.addr.va + offset);
+ chunk->fpm_addr = pble_rsrc->next_fpm_addr;
+ dev_dbg(rfdev_to_dev(dev),
+ "PBLE: chunk_size[%lld] = 0x%llx vaddr=0x%llx fpm_addr = %llx\n",
+ chunk->size, chunk->size, chunk->vaddr, chunk->fpm_addr);
+
+ return 0;
+}
+
+/**
+ * fpm_to_idx - given fpm address, get pble index
+ * @pble_rsrc: pble resource management
+ * @addr: fpm address for index
+ */
+static u32 fpm_to_idx(struct irdma_hmc_pble_rsrc *pble_rsrc, u64 addr)
+{
+ u64 idx;
+
+ idx = (addr - (pble_rsrc->fpm_base_addr)) >> 3;
+
+ return (u32)idx;
+}
+
+/**
+ * add_bp_pages - add backing pages for sd
+ * @pble_rsrc: pble resource management
+ * @info: page info for sd
+ */
+static enum irdma_status_code
+add_bp_pages(struct irdma_hmc_pble_rsrc *pble_rsrc,
+ struct irdma_add_page_info *info)
+{
+ struct irdma_sc_dev *dev = pble_rsrc->dev;
+ u8 *addr;
+ struct irdma_dma_mem mem;
+ struct irdma_hmc_pd_entry *pd_entry;
+ struct irdma_hmc_sd_entry *sd_entry = info->sd_entry;
+ struct irdma_hmc_info *hmc_info = info->hmc_info;
+ struct irdma_chunk *chunk = info->chunk;
+ enum irdma_status_code status = 0;
+ u32 rel_pd_idx = info->idx.rel_pd_idx;
+ u32 pd_idx = info->idx.pd_idx;
+ u32 i;
+
+ if (irdma_pble_get_paged_mem(chunk, info->pages))
+ return IRDMA_ERR_NO_MEMORY;
+
+ status = irdma_add_sd_table_entry(dev->hw, hmc_info, info->idx.sd_idx,
+ IRDMA_SD_TYPE_PAGED,
+ IRDMA_HMC_DIRECT_BP_SIZE);
+
+ if (status)
+ goto error;
+
+ addr = (u8 *)(uintptr_t)chunk->vaddr;
+ for (i = 0; i < info->pages; i++) {
+ mem.pa = (u64)chunk->dmainfo.dmaaddrs[i];
+ mem.size = 4096;
+ mem.va = addr;
+ pd_entry = &sd_entry->u.pd_table.pd_entry[rel_pd_idx++];
+ if (!pd_entry->valid) {
+ status = irdma_add_pd_table_entry(dev, hmc_info,
+ pd_idx++, &mem);
+ if (status)
+ goto error;
+
+ addr += 4096;
+ }
+ }
+
+ chunk->fpm_addr = pble_rsrc->next_fpm_addr;
+ return 0;
+
+error:
+ irdma_pble_free_paged_mem(chunk);
+
+ return status;
+}
+
+/**
+ * add_pble_prm - add a sd entry for pble resoure
+ * @pble_rsrc: pble resource management
+ */
+static enum irdma_status_code
+add_pble_prm(struct irdma_hmc_pble_rsrc *pble_rsrc)
+{
+ struct irdma_sc_dev *dev = pble_rsrc->dev;
+ struct irdma_hmc_sd_entry *sd_entry;
+ struct irdma_hmc_info *hmc_info;
+ struct irdma_chunk *chunk;
+ struct irdma_add_page_info info;
+ struct sd_pd_idx *idx = &info.idx;
+ enum irdma_status_code ret_code = 0;
+ enum irdma_sd_entry_type sd_entry_type;
+ u64 sd_reg_val = 0;
+ struct irdma_virt_mem chunkmem;
+ u32 pages;
+
+ if (pble_rsrc->unallocated_pble < PBLE_PER_PAGE)
+ return IRDMA_ERR_NO_MEMORY;
+
+ if (pble_rsrc->next_fpm_addr & 0xfff)
+ return IRDMA_ERR_INVALID_PAGE_DESC_INDEX;
+
+ chunkmem.size = sizeof(*chunk);
+ chunkmem.va = kzalloc(chunkmem.size, GFP_ATOMIC);
+ if (!chunkmem.va)
+ return IRDMA_ERR_NO_MEMORY;
+
+ chunk = chunkmem.va;
+ chunk->chunkmem = chunkmem;
+ hmc_info = dev->hmc_info;
+ chunk->dev = dev;
+ chunk->fpm_addr = pble_rsrc->next_fpm_addr;
+ get_sd_pd_idx(pble_rsrc, idx);
+ sd_entry = &hmc_info->sd_table.sd_entry[idx->sd_idx];
+ pages = (idx->rel_pd_idx) ? (IRDMA_HMC_PD_CNT_IN_SD - idx->rel_pd_idx) :
+ IRDMA_HMC_PD_CNT_IN_SD;
+ pages = min(pages, pble_rsrc->unallocated_pble >> PBLE_512_SHIFT);
+ info.chunk = chunk;
+ info.hmc_info = hmc_info;
+ info.pages = pages;
+ info.sd_entry = sd_entry;
+ if (!sd_entry->valid)
+ sd_entry_type = (!idx->rel_pd_idx &&
+ (pages == IRDMA_HMC_PD_CNT_IN_SD) &&
+ dev->is_pf) ?
+ IRDMA_SD_TYPE_DIRECT : IRDMA_SD_TYPE_PAGED;
+ else
+ sd_entry_type = sd_entry->entry_type;
+
+ dev_dbg(rfdev_to_dev(dev),
+ "PBLE: pages = %d, unallocated_pble[%d] current_fpm_addr = %llx\n",
+ pages, pble_rsrc->unallocated_pble, pble_rsrc->next_fpm_addr);
+ dev_dbg(rfdev_to_dev(dev), "PBLE: sd_entry_type = %d\n",
+ sd_entry_type);
+ if (sd_entry_type == IRDMA_SD_TYPE_DIRECT)
+ ret_code = add_sd_direct(pble_rsrc, &info);
+
+ if (ret_code)
+ sd_entry_type = IRDMA_SD_TYPE_PAGED;
+ else
+ pble_rsrc->stats_direct_sds++;
+
+ if (sd_entry_type == IRDMA_SD_TYPE_PAGED) {
+ ret_code = add_bp_pages(pble_rsrc, &info);
+ if (ret_code)
+ goto error;
+ else
+ pble_rsrc->stats_paged_sds++;
+ }
+
+ ret_code = irdma_prm_add_pble_mem(&pble_rsrc->pinfo, chunk);
+ if (ret_code)
+ goto error;
+
+ pble_rsrc->next_fpm_addr += chunk->size;
+ dev_dbg(rfdev_to_dev(dev),
+ "PBLE: next_fpm_addr = %llx chunk_size[%llu] = 0x%llx\n",
+ pble_rsrc->next_fpm_addr, chunk->size, chunk->size);
+ pble_rsrc->unallocated_pble -= (u32)(chunk->size >> 3);
+ list_add(&chunk->list, &pble_rsrc->pinfo.clist);
+ sd_reg_val = (sd_entry_type == IRDMA_SD_TYPE_PAGED) ?
+ sd_entry->u.pd_table.pd_page_addr.pa :
+ sd_entry->u.bp.addr.pa;
+ if (sd_entry->valid)
+ return 0;
+
+ if (dev->is_pf) {
+ ret_code = irdma_hmc_sd_one(dev, hmc_info->hmc_fn_id,
+ sd_reg_val, idx->sd_idx,
+ sd_entry->entry_type, true);
+ if (ret_code)
+ goto error;
+ }
+
+ sd_entry->valid = true;
+ return 0;
+
+error:
+ if (chunk->bitmapbuf)
+ kfree(chunk->bitmapmem.va);
+
+ kfree(chunk->chunkmem.va);
+
+ return ret_code;
+}
+
+/**
+ * free_lvl2 - fee level 2 pble
+ * @pble_rsrc: pble resource management
+ * @palloc: level 2 pble allocation
+ */
+static void free_lvl2(struct irdma_hmc_pble_rsrc *pble_rsrc,
+ struct irdma_pble_alloc *palloc)
+{
+ u32 i;
+ struct irdma_pble_level2 *lvl2 = &palloc->level2;
+ struct irdma_pble_info *root = &lvl2->root;
+ struct irdma_pble_info *leaf = lvl2->leaf;
+
+ for (i = 0; i < lvl2->leaf_cnt; i++, leaf++) {
+ if (leaf->addr)
+ irdma_prm_return_pbles(&pble_rsrc->pinfo,
+ &leaf->chunkinfo);
+ else
+ break;
+ }
+
+ if (root->addr)
+ irdma_prm_return_pbles(&pble_rsrc->pinfo, &root->chunkinfo);
+
+ kfree(lvl2->leafmem.va);
+ lvl2->leaf = NULL;
+}
+
+/**
+ * get_lvl2_pble - get level 2 pble resource
+ * @pble_rsrc: pble resource management
+ * @palloc: level 2 pble allocation
+ */
+static enum irdma_status_code
+get_lvl2_pble(struct irdma_hmc_pble_rsrc *pble_rsrc,
+ struct irdma_pble_alloc *palloc)
+{
+ u32 lf4k, lflast, total, i;
+ u32 pblcnt = PBLE_PER_PAGE;
+ u64 *addr;
+ struct irdma_pble_level2 *lvl2 = &palloc->level2;
+ struct irdma_pble_info *root = &lvl2->root;
+ struct irdma_pble_info *leaf;
+ enum irdma_status_code ret_code;
+ u64 fpm_addr;
+
+ /* number of full 512 (4K) leafs) */
+ lf4k = palloc->total_cnt >> 9;
+ lflast = palloc->total_cnt % PBLE_PER_PAGE;
+ total = (lflast == 0) ? lf4k : lf4k + 1;
+ lvl2->leaf_cnt = total;
+
+ lvl2->leafmem.size = (sizeof(*leaf) * total);
+ lvl2->leafmem.va = kzalloc(lvl2->leafmem.size, GFP_ATOMIC);
+ if (!lvl2->leafmem.va)
+ return IRDMA_ERR_NO_MEMORY;
+
+ lvl2->leaf = lvl2->leafmem.va;
+ leaf = lvl2->leaf;
+ ret_code = irdma_prm_get_pbles(&pble_rsrc->pinfo, &root->chunkinfo,
+ total << 3, &root->addr, &fpm_addr);
+ if (ret_code) {
+ kfree(lvl2->leafmem.va);
+ lvl2->leaf = NULL;
+ return IRDMA_ERR_NO_MEMORY;
+ }
+
+ root->idx = fpm_to_idx(pble_rsrc, fpm_addr);
+ root->cnt = total;
+ addr = (u64 *)(uintptr_t)root->addr;
+ for (i = 0; i < total; i++, leaf++) {
+ pblcnt = (lflast && ((i + 1) == total)) ?
+ lflast : PBLE_PER_PAGE;
+ ret_code = irdma_prm_get_pbles(&pble_rsrc->pinfo,
+ &leaf->chunkinfo, pblcnt << 3,
+ &leaf->addr, &fpm_addr);
+ if (ret_code)
+ goto error;
+
+ leaf->idx = fpm_to_idx(pble_rsrc, fpm_addr);
+
+ leaf->cnt = pblcnt;
+ *addr = (u64)leaf->idx;
+ addr++;
+ }
+
+ palloc->level = PBLE_LEVEL_2;
+ pble_rsrc->stats_lvl2++;
+ return 0;
+
+error:
+ free_lvl2(pble_rsrc, palloc);
+
+ return IRDMA_ERR_NO_MEMORY;
+}
+
+/**
+ * get_lvl1_pble - get level 1 pble resource
+ * @pble_rsrc: pble resource management
+ * @palloc: level 1 pble allocation
+ */
+static enum irdma_status_code
+get_lvl1_pble(struct irdma_hmc_pble_rsrc *pble_rsrc,
+ struct irdma_pble_alloc *palloc)
+{
+ enum irdma_status_code ret_code;
+ u64 fpm_addr, vaddr;
+ struct irdma_pble_info *lvl1 = &palloc->level1;
+
+ ret_code = irdma_prm_get_pbles(&pble_rsrc->pinfo, &lvl1->chunkinfo,
+ palloc->total_cnt << 3, &vaddr,
+ &fpm_addr);
+ if (ret_code)
+ return IRDMA_ERR_NO_MEMORY;
+
+ lvl1->addr = vaddr;
+ palloc->level = PBLE_LEVEL_1;
+ lvl1->idx = fpm_to_idx(pble_rsrc, fpm_addr);
+ lvl1->cnt = palloc->total_cnt;
+ pble_rsrc->stats_lvl1++;
+
+ return 0;
+}
+
+/**
+ * get_lvl1_lvl2_pble - calls get_lvl1 and get_lvl2 pble routine
+ * @pble_rsrc: pble resources
+ * @palloc: contains all inforamtion regarding pble (idx + pble addr)
+ * @level1_only: flag for a level 1 PBLE
+ */
+static enum irdma_status_code
+get_lvl1_lvl2_pble(struct irdma_hmc_pble_rsrc *pble_rsrc,
+ struct irdma_pble_alloc *palloc, bool level1_only)
+{
+ enum irdma_status_code status = 0;
+
+ status = get_lvl1_pble(pble_rsrc, palloc);
+ if (!status || level1_only || palloc->total_cnt <= PBLE_PER_PAGE)
+ return status;
+
+ status = get_lvl2_pble(pble_rsrc, palloc);
+
+ return status;
+}
+
+/**
+ * irdma_get_pble - allocate pbles from the prm
+ * @pble_rsrc: pble resources
+ * @palloc: contains all inforamtion regarding pble (idx + pble addr)
+ * @pble_cnt: #of pbles requested
+ * @level1_only: true if only pble level 1 to acquire
+ */
+enum irdma_status_code irdma_get_pble(struct irdma_hmc_pble_rsrc *pble_rsrc,
+ struct irdma_pble_alloc *palloc,
+ u32 pble_cnt, bool level1_only)
+{
+ enum irdma_status_code status = 0;
+ unsigned long flags;
+ int max_sds = 0;
+ int i;
+
+ palloc->total_cnt = pble_cnt;
+ palloc->level = PBLE_LEVEL_0;
+ spin_lock_irqsave(&pble_rsrc->pble_lock, flags);
+ /*check first to see if we can get pble's without acquiring
+ * additional sd's
+ */
+ status = get_lvl1_lvl2_pble(pble_rsrc, palloc, level1_only);
+ if (!status)
+ goto exit;
+
+ max_sds = (palloc->total_cnt >> 18) + 1;
+ for (i = 0; i < max_sds; i++) {
+ status = add_pble_prm(pble_rsrc);
+ if (status)
+ break;
+
+ status = get_lvl1_lvl2_pble(pble_rsrc, palloc, level1_only);
+ /* if level1_only, only go through it once */
+ if (!status || level1_only)
+ break;
+ }
+
+exit:
+ if (!status) {
+ pble_rsrc->allocdpbles += pble_cnt;
+ pble_rsrc->stats_alloc_ok++;
+ } else {
+ pble_rsrc->stats_alloc_fail++;
+ }
+ spin_unlock_irqrestore(&pble_rsrc->pble_lock, flags);
+
+ return status;
+}
+
+/**
+ * irdma_free_pble - put pbles back into prm
+ * @pble_rsrc: pble resources
+ * @palloc: contains all information regarding pble resource being freed
+ */
+void irdma_free_pble(struct irdma_hmc_pble_rsrc *pble_rsrc,
+ struct irdma_pble_alloc *palloc)
+{
+ pble_rsrc->freedpbles += palloc->total_cnt;
+
+ if (palloc->level == PBLE_LEVEL_2)
+ free_lvl2(pble_rsrc, palloc);
+ else
+ irdma_prm_return_pbles(&pble_rsrc->pinfo,
+ &palloc->level1.chunkinfo);
+ pble_rsrc->stats_alloc_freed++;
+}
diff --git a/drivers/infiniband/hw/irdma/pble.h b/drivers/infiniband/hw/irdma/pble.h
new file mode 100644
index 000000000000..a6041e9748d0
--- /dev/null
+++ b/drivers/infiniband/hw/irdma/pble.h
@@ -0,0 +1,136 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2019, Intel Corporation. */
+
+#ifndef IRDMA_PBLE_H
+#define IRDMA_PBLE_H
+
+#define PBLE_SHIFT 6
+#define PBLE_PER_PAGE 512
+#define HMC_PAGED_BP_SHIFT 12
+#define PBLE_512_SHIFT 9
+#define PBLE_INVALID_IDX 0xffffffff
+
+enum irdma_pble_level {
+ PBLE_LEVEL_0 = 0,
+ PBLE_LEVEL_1 = 1,
+ PBLE_LEVEL_2 = 2,
+};
+
+enum irdma_alloc_type {
+ PBLE_NO_ALLOC = 0,
+ PBLE_SD_CONTIGOUS = 1,
+ PBLE_SD_PAGED = 2,
+};
+
+struct irdma_chunk;
+
+struct irdma_pble_chunkinfo {
+ struct irdma_chunk *pchunk;
+ u64 bit_idx;
+ u64 bits_used;
+};
+
+struct irdma_pble_info {
+ u64 addr;
+ u32 idx;
+ u32 cnt;
+ struct irdma_pble_chunkinfo chunkinfo;
+};
+
+struct irdma_pble_level2 {
+ struct irdma_pble_info root;
+ struct irdma_pble_info *leaf;
+ struct irdma_virt_mem leafmem;
+ u32 leaf_cnt;
+};
+
+struct irdma_pble_alloc {
+ u32 total_cnt;
+ enum irdma_pble_level level;
+ union {
+ struct irdma_pble_info level1;
+ struct irdma_pble_level2 level2;
+ };
+};
+
+struct sd_pd_idx {
+ u32 sd_idx;
+ u32 pd_idx;
+ u32 rel_pd_idx;
+};
+
+struct irdma_add_page_info {
+ struct irdma_chunk *chunk;
+ struct irdma_hmc_sd_entry *sd_entry;
+ struct irdma_hmc_info *hmc_info;
+ struct sd_pd_idx idx;
+ u32 pages;
+};
+
+struct irdma_chunk {
+ struct list_head list;
+ struct irdma_dma_info dmainfo;
+ void *bitmapbuf;
+
+ u32 sizeofbitmap;
+ u64 size;
+ u64 vaddr;
+ u64 fpm_addr;
+ u32 pg_cnt;
+ enum irdma_alloc_type type;
+ struct irdma_sc_dev *dev;
+ struct irdma_virt_mem bitmapmem;
+ struct irdma_virt_mem chunkmem;
+};
+
+struct irdma_pble_prm {
+ struct list_head clist;
+ spinlock_t prm_lock; /* protect prm bitmap */
+ u64 total_pble_alloc;
+ u64 free_pble_cnt;
+ u8 pble_shift;
+};
+
+struct irdma_hmc_pble_rsrc {
+ u32 unallocated_pble;
+ spinlock_t pble_lock; /* to serialize PBLE resource acquisition */
+ struct irdma_sc_dev *dev;
+ u64 fpm_base_addr;
+ u64 next_fpm_addr;
+ struct irdma_pble_prm pinfo;
+ u64 allocdpbles;
+ u64 freedpbles;
+ u32 stats_direct_sds;
+ u32 stats_paged_sds;
+ u64 stats_alloc_ok;
+ u64 stats_alloc_fail;
+ u64 stats_alloc_freed;
+ u64 stats_lvl1;
+ u64 stats_lvl2;
+};
+
+void irdma_destroy_pble_prm(struct irdma_hmc_pble_rsrc *pble_rsrc);
+enum irdma_status_code
+irdma_hmc_init_pble(struct irdma_sc_dev *dev,
+ struct irdma_hmc_pble_rsrc *pble_rsrc);
+void irdma_free_pble(struct irdma_hmc_pble_rsrc *pble_rsrc,
+ struct irdma_pble_alloc *palloc);
+enum irdma_status_code irdma_get_pble(struct irdma_hmc_pble_rsrc *pble_rsrc,
+ struct irdma_pble_alloc *palloc,
+ u32 pble_cnt, bool level1_only);
+enum irdma_status_code irdma_prm_add_pble_mem(struct irdma_pble_prm *pprm,
+ struct irdma_chunk *pchunk);
+enum irdma_status_code
+irdma_prm_get_pbles(struct irdma_pble_prm *pprm,
+ struct irdma_pble_chunkinfo *chunkinfo, u32 mem_size,
+ u64 *vaddr, u64 *fpm_addr);
+void irdma_prm_return_pbles(struct irdma_pble_prm *pprm,
+ struct irdma_pble_chunkinfo *chunkinfo);
+void irdma_pble_acquire_lock(struct irdma_hmc_pble_rsrc *pble_rsrc,
+ unsigned long *flags);
+void irdma_pble_release_lock(struct irdma_hmc_pble_rsrc *pble_rsrc,
+ unsigned long *flags);
+void irdma_pble_free_paged_mem(struct irdma_chunk *chunk);
+enum irdma_status_code irdma_pble_get_paged_mem(struct irdma_chunk *chunk,
+ int pg_cnt);
+#endif /* IRDMA_PBLE_H */
--
2.21.0
^ permalink raw reply related
* [rdma 15/16] RDMA/irdma: Update MAINTAINERS file
From: Jeff Kirsher @ 2019-07-04 2:12 UTC (permalink / raw)
To: dledford, jgg, davem
Cc: Shiraz Saleem, linux-rdma, netdev, nhorman, sassmann, poswald,
david.m.ertman, Mustafa Ismail
In-Reply-To: <20190704021259.15489-1-jeffrey.t.kirsher@intel.com>
From: Shiraz Saleem <shiraz.saleem@intel.com>
Add maintainer entry for irdma driver.
Signed-off-by: Mustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
---
MAINTAINERS | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/MAINTAINERS b/MAINTAINERS
index 9ac03f3e3bd5..b552bf583c97 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8093,7 +8093,15 @@ L: linux-pm@vger.kernel.org
S: Supported
F: drivers/cpufreq/intel_pstate.c
-INTEL RDMA RNIC DRIVER
+INTEL ETHERNET RDMA DRIVER
+M: Mustafa Ismail <mustafa.ismail@intel.com>
+M: Shiraz Saleem <shiraz.saleem@intel.com>
+L: linux-rdma@vger.kernel.org
+S: Supported
+F: drivers/infiniband/hw/irdma/
+F: include/uapi/rdma/irdma-abi.h
+
+INTEL X722 RDMA RNIC DRIVER
M: Faisal Latif <faisal.latif@intel.com>
M: Shiraz Saleem <shiraz.saleem@intel.com>
L: linux-rdma@vger.kernel.org
--
2.21.0
^ permalink raw reply related
* [rdma 07/16] RDMA/irdma: Add connection manager
From: Jeff Kirsher @ 2019-07-04 2:12 UTC (permalink / raw)
To: dledford, jgg, davem
Cc: Mustafa Ismail, linux-rdma, netdev, nhorman, sassmann, poswald,
david.m.ertman, Shiraz Saleem
In-Reply-To: <20190704021259.15489-1-jeffrey.t.kirsher@intel.com>
From: Mustafa Ismail <mustafa.ismail@intel.com>
Add connection management (CM) implementation for
iWARP including accept, reject, connect, create_listen,
destroy_listen and CM utility functions
Signed-off-by: Mustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com>
---
drivers/infiniband/hw/irdma/cm.c | 4514 ++++++++++++++++++++++++++++++
drivers/infiniband/hw/irdma/cm.h | 415 +++
2 files changed, 4929 insertions(+)
create mode 100644 drivers/infiniband/hw/irdma/cm.c
create mode 100644 drivers/infiniband/hw/irdma/cm.h
diff --git a/drivers/infiniband/hw/irdma/cm.c b/drivers/infiniband/hw/irdma/cm.c
new file mode 100644
index 000000000000..c6b96faf7818
--- /dev/null
+++ b/drivers/infiniband/hw/irdma/cm.c
@@ -0,0 +1,4514 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/* Copyright (c) 2019, Intel Corporation. */
+
+#include <linux/highmem.h>
+#include <net/addrconf.h>
+#include <net/ip6_route.h>
+#include <net/flow.h>
+#include <net/secure_seq.h>
+
+#include "main.h"
+#include "trace.h"
+
+static void irdma_rem_ref_cm_node(struct irdma_cm_node *);
+static void irdma_cm_post_event(struct irdma_cm_event *event);
+static void irdma_disconnect_worker(struct work_struct *work);
+
+/**
+ * irdma_free_sqbuf - put back puda buffer if refcount is 0
+ * @vsi: The VSI structure of the device
+ * @bufp: puda buffer to free
+ */
+void irdma_free_sqbuf(struct irdma_sc_vsi *vsi, void *bufp)
+{
+ struct irdma_puda_buf *buf = bufp;
+ struct irdma_puda_rsrc *ilq = vsi->ilq;
+
+ if (!atomic_dec_return(&buf->refcount))
+ irdma_puda_ret_bufpool(ilq, buf);
+}
+
+/**
+ * irdma_derive_hw_ird_setting - Calculate IRD
+ * @cm_ird: IRD of connection's node
+ *
+ * The ird from the connection is rounded to a supported HW
+ * setting (2,8,32,64,128) and then encoded for ird_size field
+ * of qp_ctx
+ */
+u8 irdma_derive_hw_ird_setting(u16 cm_ird)
+{
+ /* ird_size field is encoded in qp_ctx */
+ switch (cm_ird ? roundup_pow_of_two(cm_ird) : 0) {
+ case IRDMA_HW_IRD_SETTING_128:
+ return 4;
+ case IRDMA_HW_IRD_SETTING_64:
+ return 3;
+ case IRDMA_HW_IRD_SETTING_32:
+ case IRDMA_HW_IRD_SETTING_16:
+ return 2;
+ case IRDMA_HW_IRD_SETTING_8:
+ case IRDMA_HW_IRD_SETTING_4:
+ return 1;
+ case IRDMA_HW_IRD_SETTING_2:
+ default:
+ break;
+ }
+
+ return 0;
+}
+
+/**
+ * irdma_record_ird_ord - Record IRD/ORD passed in
+ * @cm_node: connection's node
+ * @conn_ird: connection IRD
+ * @conn_ord: connection ORD
+ */
+static void irdma_record_ird_ord(struct irdma_cm_node *cm_node, u32 conn_ird,
+ u32 conn_ord)
+{
+ if (conn_ird > cm_node->dev->hw_attrs.max_hw_ird)
+ conn_ird = cm_node->dev->hw_attrs.max_hw_ird;
+
+ if (conn_ord > cm_node->dev->hw_attrs.max_hw_ord)
+ conn_ord = cm_node->dev->hw_attrs.max_hw_ord;
+ else if (!conn_ord && cm_node->send_rdma0_op == SEND_RDMA_READ_ZERO)
+ conn_ord = 1;
+ cm_node->ird_size = conn_ird;
+ cm_node->ord_size = conn_ord;
+}
+
+/**
+ * irdma_copy_ip_ntohl - copy IP address from network to host
+ * @dst: IP address in host order
+ * @src: IP address in network order (big endian)
+ */
+void irdma_copy_ip_ntohl(u32 *dst, __be32 *src)
+{
+ *dst++ = ntohl(*src++);
+ *dst++ = ntohl(*src++);
+ *dst++ = ntohl(*src++);
+ *dst = ntohl(*src);
+}
+
+/**
+ * irdma_copy_ip_htonl - copy IP address from host to network order
+ * @dst: IP address in network order (big endian)
+ * @src: IP address in host order
+ */
+void irdma_copy_ip_htonl(__be32 *dst, u32 *src)
+{
+ *dst++ = htonl(*src++);
+ *dst++ = htonl(*src++);
+ *dst++ = htonl(*src++);
+ *dst = htonl(*src);
+}
+
+/**
+ * irdma_get_addr_info
+ * @cm_node: contains ip/tcp info
+ * @cm_info: to get a copy of the cm_node ip/tcp info
+ */
+static void irdma_get_addr_info(struct irdma_cm_node *cm_node,
+ struct irdma_cm_info *cm_info)
+{
+ memset(cm_info, 0, sizeof(*cm_info));
+ cm_info->ipv4 = cm_node->ipv4;
+ cm_info->vlan_id = cm_node->vlan_id;
+ memcpy(cm_info->loc_addr, cm_node->loc_addr, sizeof(cm_info->loc_addr));
+ memcpy(cm_info->rem_addr, cm_node->rem_addr, sizeof(cm_info->rem_addr));
+ cm_info->loc_port = cm_node->loc_port;
+ cm_info->rem_port = cm_node->rem_port;
+}
+
+/**
+ * irdma_fill_sockaddr4 - fill in addr info for IPv4 connection
+ * @cm_node: connection's node
+ * @event: upper layer's cm event
+ */
+static inline void irdma_fill_sockaddr4(struct irdma_cm_node *cm_node,
+ struct iw_cm_event *event)
+{
+ struct sockaddr_in *laddr = (struct sockaddr_in *)&event->local_addr;
+ struct sockaddr_in *raddr = (struct sockaddr_in *)&event->remote_addr;
+
+ laddr->sin_family = AF_INET;
+ raddr->sin_family = AF_INET;
+
+ laddr->sin_port = htons(cm_node->loc_port);
+ raddr->sin_port = htons(cm_node->rem_port);
+
+ laddr->sin_addr.s_addr = htonl(cm_node->loc_addr[0]);
+ raddr->sin_addr.s_addr = htonl(cm_node->rem_addr[0]);
+}
+
+/**
+ * irdma_fill_sockaddr6 - fill in addr info for IPv6 connection
+ * @cm_node: connection's node
+ * @event: upper layer's cm event
+ */
+static inline void irdma_fill_sockaddr6(struct irdma_cm_node *cm_node,
+ struct iw_cm_event *event)
+{
+ struct sockaddr_in6 *laddr6 = (struct sockaddr_in6 *)&event->local_addr;
+ struct sockaddr_in6 *raddr6 = (struct sockaddr_in6 *)&event->remote_addr;
+
+ laddr6->sin6_family = AF_INET6;
+ raddr6->sin6_family = AF_INET6;
+
+ laddr6->sin6_port = htons(cm_node->loc_port);
+ raddr6->sin6_port = htons(cm_node->rem_port);
+
+ irdma_copy_ip_htonl(laddr6->sin6_addr.in6_u.u6_addr32,
+ cm_node->loc_addr);
+ irdma_copy_ip_htonl(raddr6->sin6_addr.in6_u.u6_addr32,
+ cm_node->rem_addr);
+}
+
+/**
+ * irdma_get_cmevent_info - for cm event upcall
+ * @cm_node: connection's node
+ * @cm_id: upper layers cm struct for the event
+ * @event: upper layer's cm event
+ */
+static inline void irdma_get_cmevent_info(struct irdma_cm_node *cm_node,
+ struct iw_cm_id *cm_id,
+ struct iw_cm_event *event)
+{
+ memcpy(&event->local_addr, &cm_id->m_local_addr,
+ sizeof(event->local_addr));
+ memcpy(&event->remote_addr, &cm_id->m_remote_addr,
+ sizeof(event->remote_addr));
+ if (cm_node) {
+ event->private_data = (void *)cm_node->pdata_buf;
+ event->private_data_len = (u8)cm_node->pdata.size;
+ event->ird = cm_node->ird_size;
+ event->ord = cm_node->ord_size;
+ }
+}
+
+/**
+ * irdma_send_cm_event - upcall cm's event handler
+ * @cm_node: connection's node
+ * @cm_id: upper layer's cm info struct
+ * @type: Event type to indicate
+ * @status: status for the event type
+ */
+static int irdma_send_cm_event(struct irdma_cm_node *cm_node,
+ struct iw_cm_id *cm_id,
+ enum iw_cm_event_type type, int status)
+{
+ struct iw_cm_event event = {};
+
+ event.event = type;
+ event.status = status;
+ switch (type) {
+ case IW_CM_EVENT_CONNECT_REQUEST:
+ trace_irdma_send_cm_event(cm_node, cm_id, type, status,
+ __builtin_return_address(0));
+ if (cm_node->ipv4)
+ irdma_fill_sockaddr4(cm_node, &event);
+ else
+ irdma_fill_sockaddr6(cm_node, &event);
+ event.provider_data = cm_node;
+ event.private_data = cm_node->pdata_buf;
+ event.private_data_len = (u8)cm_node->pdata.size;
+ event.ird = cm_node->ird_size;
+ break;
+ case IW_CM_EVENT_CONNECT_REPLY:
+ trace_irdma_send_cm_event(cm_node, cm_id, type, status,
+ __builtin_return_address(0));
+ irdma_get_cmevent_info(cm_node, cm_id, &event);
+ break;
+ case IW_CM_EVENT_ESTABLISHED:
+ trace_irdma_send_cm_event(cm_node, cm_id, type, status,
+ __builtin_return_address(0));
+ event.ird = cm_node->ird_size;
+ event.ord = cm_node->ord_size;
+ break;
+ case IW_CM_EVENT_DISCONNECT:
+ trace_irdma_send_cm_event_no_node(cm_id, type, status,
+ __builtin_return_address(0));
+ break;
+ case IW_CM_EVENT_CLOSE:
+ trace_irdma_send_cm_event_no_node(cm_id, type, status,
+ __builtin_return_address(0));
+ break;
+ default:
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: Unsupported event type received type = %d\n",
+ type);
+ return -1;
+ }
+
+ return cm_id->event_handler(cm_id, &event);
+}
+
+/**
+ * irdma_create_event - create cm event
+ * @cm_node: connection's node
+ * @type: Event type to generate
+ */
+static struct irdma_cm_event *irdma_create_event(struct irdma_cm_node *cm_node,
+ enum irdma_cm_event_type type)
+{
+ struct irdma_cm_event *event;
+
+ if (!cm_node->cm_id)
+ return NULL;
+
+ event = kzalloc(sizeof(*event), GFP_ATOMIC);
+
+ if (!event)
+ return NULL;
+
+ event->type = type;
+ event->cm_node = cm_node;
+ memcpy(event->cm_info.rem_addr, cm_node->rem_addr,
+ sizeof(event->cm_info.rem_addr));
+ memcpy(event->cm_info.loc_addr, cm_node->loc_addr,
+ sizeof(event->cm_info.loc_addr));
+ event->cm_info.rem_port = cm_node->rem_port;
+ event->cm_info.loc_port = cm_node->loc_port;
+ event->cm_info.cm_id = cm_node->cm_id;
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: node=%p event=%p type=%u dst=%pI4 src=%pI4\n", cm_node,
+ event, type, event->cm_info.loc_addr,
+ event->cm_info.rem_addr);
+ trace_irdma_create_event(cm_node, type, __builtin_return_address(0));
+ irdma_cm_post_event(event);
+
+ return event;
+}
+
+/**
+ * irdma_free_retrans_entry - free send entry
+ * @cm_node: connection's node
+ */
+static void irdma_free_retrans_entry(struct irdma_cm_node *cm_node)
+{
+ struct irdma_device *iwdev = cm_node->iwdev;
+ struct irdma_timer_entry *send_entry;
+
+ send_entry = cm_node->send_entry;
+ if (!send_entry)
+ return;
+
+ cm_node->send_entry = NULL;
+ irdma_free_sqbuf(&iwdev->vsi, send_entry->sqbuf);
+ kfree(send_entry);
+ atomic_dec(&cm_node->ref_count);
+}
+
+/**
+ * irdma_cleanup_retrans_entry - free send entry with lock
+ * @cm_node: connection's node
+ */
+static void irdma_cleanup_retrans_entry(struct irdma_cm_node *cm_node)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&cm_node->retrans_list_lock, flags);
+ irdma_free_retrans_entry(cm_node);
+ spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags);
+}
+
+/**
+ * irdma_form_ah_cm_frame - get a free packet and build frame with address handle
+ * @cm_node: connection's node ionfo to use in frame
+ * @options: pointer to options info
+ * @hdr: pointer mpa header
+ * @pdata: pointer to private data
+ * @flags: indicates FIN or ACK
+ */
+static struct irdma_puda_buf *irdma_form_ah_cm_frame(struct irdma_cm_node *cm_node,
+ struct irdma_kmem_info *options,
+ struct irdma_kmem_info *hdr,
+ struct irdma_kmem_info *pdata,
+ u8 flags)
+{
+ struct irdma_puda_buf *sqbuf;
+ struct irdma_sc_vsi *vsi = &cm_node->iwdev->vsi;
+ u8 *buf;
+ struct tcphdr *tcph;
+ u16 pktsize;
+ u32 opts_len = 0;
+ u32 pd_len = 0;
+ u32 hdr_len = 0;
+
+ if (!cm_node->ah || !cm_node->ah->ah_info.ah_valid) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev), "CM: AH invalid\n");
+ return NULL;
+ }
+
+ sqbuf = irdma_puda_get_bufpool(vsi->ilq);
+ if (!sqbuf) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev), "CM: SQ buf NULL\n");
+ return NULL;
+ }
+
+ sqbuf->ah_id = cm_node->ah->ah_info.ah_idx;
+ buf = sqbuf->mem.va;
+ if (options)
+ opts_len = (u32)options->size;
+
+ if (hdr)
+ hdr_len = hdr->size;
+
+ if (pdata)
+ pd_len = pdata->size;
+
+ pktsize = sizeof(*tcph) + opts_len + hdr_len + pd_len;
+
+ memset(buf, 0, pktsize);
+
+ sqbuf->totallen = pktsize;
+ sqbuf->tcphlen = sizeof(*tcph) + opts_len;
+ sqbuf->scratch = (void *)cm_node;
+
+ tcph = (struct tcphdr *)buf;
+ buf += sizeof(*tcph);
+
+ tcph->source = htons(cm_node->loc_port);
+ tcph->dest = htons(cm_node->rem_port);
+ tcph->seq = htonl(cm_node->tcp_cntxt.loc_seq_num);
+
+ if (flags & SET_ACK) {
+ cm_node->tcp_cntxt.loc_ack_num = cm_node->tcp_cntxt.rcv_nxt;
+ tcph->ack_seq = htonl(cm_node->tcp_cntxt.loc_ack_num);
+ tcph->ack = 1;
+ } else {
+ tcph->ack_seq = 0;
+ }
+
+ if (flags & SET_SYN) {
+ cm_node->tcp_cntxt.loc_seq_num++;
+ tcph->syn = 1;
+ } else {
+ cm_node->tcp_cntxt.loc_seq_num += hdr_len + pd_len;
+ }
+
+ if (flags & SET_FIN) {
+ cm_node->tcp_cntxt.loc_seq_num++;
+ tcph->fin = 1;
+ }
+
+ if (flags & SET_RST)
+ tcph->rst = 1;
+
+ tcph->doff = (u16)((sizeof(*tcph) + opts_len + 3) >> 2);
+ sqbuf->tcphlen = tcph->doff << 2;
+ tcph->window = htons(cm_node->tcp_cntxt.rcv_wnd);
+ tcph->urg_ptr = 0;
+
+ if (opts_len) {
+ memcpy(buf, options->addr, opts_len);
+ buf += opts_len;
+ }
+
+ if (hdr_len) {
+ memcpy(buf, hdr->addr, hdr_len);
+ buf += hdr_len;
+ }
+
+ if (pdata && pdata->addr)
+ memcpy(buf, pdata->addr, pdata->size);
+
+ atomic_set(&sqbuf->refcount, 1);
+
+ irdma_debug_buf(vsi->dev, IRDMA_DEBUG_ILQ, "TRANSMIT ILQ BUFFER",
+ sqbuf->mem.va, sqbuf->totallen);
+
+ return sqbuf;
+}
+
+/**
+ * irdma_form_uda_cm_frame - get a free packet and build frame full tcpip packet
+ * @cm_node: connection's node ionfo to use in frame
+ * @options: pointer to options info
+ * @hdr: pointer mpa header
+ * @pdata: pointer to private data
+ * @flags: indicates FIN or ACK
+ */
+static struct irdma_puda_buf *irdma_form_uda_cm_frame(struct irdma_cm_node *cm_node,
+ struct irdma_kmem_info *options,
+ struct irdma_kmem_info *hdr,
+ struct irdma_kmem_info *pdata,
+ u8 flags)
+{
+ struct irdma_puda_buf *sqbuf;
+ struct irdma_sc_vsi *vsi = &cm_node->iwdev->vsi;
+ u8 *buf;
+
+ struct tcphdr *tcph;
+ struct iphdr *iph;
+ struct ipv6hdr *ip6h;
+ struct ethhdr *ethh;
+ u16 pktsize;
+ u16 eth_hlen = ETH_HLEN;
+ u32 opts_len = 0;
+ u32 pd_len = 0;
+ u32 hdr_len = 0;
+
+ u16 vtag;
+
+ sqbuf = irdma_puda_get_bufpool(vsi->ilq);
+ if (!sqbuf)
+ return NULL;
+
+ buf = sqbuf->mem.va;
+
+ if (options)
+ opts_len = (u32)options->size;
+
+ if (hdr)
+ hdr_len = hdr->size;
+
+ if (pdata)
+ pd_len = pdata->size;
+
+ if (cm_node->vlan_id < VLAN_N_VID)
+ eth_hlen += 4;
+
+ if (cm_node->ipv4)
+ pktsize = sizeof(*iph) + sizeof(*tcph);
+ else
+ pktsize = sizeof(*ip6h) + sizeof(*tcph);
+ pktsize += opts_len + hdr_len + pd_len;
+
+ memset(buf, 0, eth_hlen + pktsize);
+
+ sqbuf->totallen = pktsize + eth_hlen;
+ sqbuf->maclen = eth_hlen;
+ sqbuf->tcphlen = sizeof(*tcph) + opts_len;
+ sqbuf->scratch = (void *)cm_node;
+
+ ethh = (struct ethhdr *)buf;
+ buf += eth_hlen;
+
+ if (cm_node->do_lpb)
+ sqbuf->do_lpb = true;
+
+ if (cm_node->ipv4) {
+ sqbuf->ipv4 = true;
+
+ iph = (struct iphdr *)buf;
+ buf += sizeof(*iph);
+ tcph = (struct tcphdr *)buf;
+ buf += sizeof(*tcph);
+
+ ether_addr_copy(ethh->h_dest, cm_node->rem_mac);
+ ether_addr_copy(ethh->h_source, cm_node->loc_mac);
+ if (cm_node->vlan_id < VLAN_N_VID) {
+ ((struct vlan_ethhdr *)ethh)->h_vlan_proto =
+ htons(ETH_P_8021Q);
+ vtag = (cm_node->user_pri << VLAN_PRIO_SHIFT) |
+ cm_node->vlan_id;
+ ((struct vlan_ethhdr *)ethh)->h_vlan_TCI = htons(vtag);
+
+ ((struct vlan_ethhdr *)ethh)->h_vlan_encapsulated_proto =
+ htons(ETH_P_IP);
+ } else {
+ ethh->h_proto = htons(ETH_P_IP);
+ }
+
+ iph->version = IPVERSION;
+ iph->ihl = 5; /* 5 * 4Byte words, IP headr len */
+ iph->tos = cm_node->tos;
+ iph->tot_len = htons(pktsize);
+ iph->id = htons(++cm_node->tcp_cntxt.loc_id);
+
+ iph->frag_off = htons(0x4000);
+ iph->ttl = 0x40;
+ iph->protocol = IPPROTO_TCP;
+ iph->saddr = htonl(cm_node->loc_addr[0]);
+ iph->daddr = htonl(cm_node->rem_addr[0]);
+ } else {
+ sqbuf->ipv4 = false;
+ ip6h = (struct ipv6hdr *)buf;
+ buf += sizeof(*ip6h);
+ tcph = (struct tcphdr *)buf;
+ buf += sizeof(*tcph);
+
+ ether_addr_copy(ethh->h_dest, cm_node->rem_mac);
+ ether_addr_copy(ethh->h_source, cm_node->loc_mac);
+ if (cm_node->vlan_id < VLAN_N_VID) {
+ ((struct vlan_ethhdr *)ethh)->h_vlan_proto =
+ htons(ETH_P_8021Q);
+ vtag = (cm_node->user_pri << VLAN_PRIO_SHIFT) |
+ cm_node->vlan_id;
+ ((struct vlan_ethhdr *)ethh)->h_vlan_TCI = htons(vtag);
+ ((struct vlan_ethhdr *)ethh)->h_vlan_encapsulated_proto =
+ htons(ETH_P_IPV6);
+ } else {
+ ethh->h_proto = htons(ETH_P_IPV6);
+ }
+ ip6h->version = 6;
+ ip6h->priority = cm_node->tos >> 4;
+ ip6h->flow_lbl[0] = cm_node->tos << 4;
+ ip6h->flow_lbl[1] = 0;
+ ip6h->flow_lbl[2] = 0;
+ ip6h->payload_len = htons(pktsize - sizeof(*ip6h));
+ ip6h->nexthdr = 6;
+ ip6h->hop_limit = 128;
+ irdma_copy_ip_htonl(ip6h->saddr.in6_u.u6_addr32,
+ cm_node->loc_addr);
+ irdma_copy_ip_htonl(ip6h->daddr.in6_u.u6_addr32,
+ cm_node->rem_addr);
+ }
+
+ tcph->source = htons(cm_node->loc_port);
+ tcph->dest = htons(cm_node->rem_port);
+ tcph->seq = htonl(cm_node->tcp_cntxt.loc_seq_num);
+
+ if (flags & SET_ACK) {
+ cm_node->tcp_cntxt.loc_ack_num = cm_node->tcp_cntxt.rcv_nxt;
+ tcph->ack_seq = htonl(cm_node->tcp_cntxt.loc_ack_num);
+ tcph->ack = 1;
+ } else {
+ tcph->ack_seq = 0;
+ }
+
+ if (flags & SET_SYN) {
+ cm_node->tcp_cntxt.loc_seq_num++;
+ tcph->syn = 1;
+ } else {
+ cm_node->tcp_cntxt.loc_seq_num += hdr_len + pd_len;
+ }
+
+ if (flags & SET_FIN) {
+ cm_node->tcp_cntxt.loc_seq_num++;
+ tcph->fin = 1;
+ }
+
+ if (flags & SET_RST)
+ tcph->rst = 1;
+
+ tcph->doff = (u16)((sizeof(*tcph) + opts_len + 3) >> 2);
+ sqbuf->tcphlen = tcph->doff << 2;
+ tcph->window = htons(cm_node->tcp_cntxt.rcv_wnd);
+ tcph->urg_ptr = 0;
+
+ if (opts_len) {
+ memcpy(buf, options->addr, opts_len);
+ buf += opts_len;
+ }
+
+ if (hdr_len) {
+ memcpy(buf, hdr->addr, hdr_len);
+ buf += hdr_len;
+ }
+
+ if (pdata && pdata->addr)
+ memcpy(buf, pdata->addr, pdata->size);
+
+ atomic_set(&sqbuf->refcount, 1);
+
+ irdma_debug_buf(vsi->dev, IRDMA_DEBUG_ILQ, "TRANSMIT ILQ BUFFER",
+ sqbuf->mem.va, sqbuf->totallen);
+
+ return sqbuf;
+}
+
+/**
+ * irdma_send_reset - Send RST packet
+ * @cm_node: connection's node
+ */
+int irdma_send_reset(struct irdma_cm_node *cm_node)
+{
+ struct irdma_puda_buf *sqbuf;
+ int flags = SET_RST | SET_ACK;
+
+ trace_irdma_send_reset(cm_node, 0, __builtin_return_address(0));
+ sqbuf = cm_node->cm_core->form_cm_frame(cm_node, NULL, NULL, NULL,
+ flags);
+ if (!sqbuf)
+ return -1;
+
+ return irdma_schedule_cm_timer(cm_node, sqbuf, IRDMA_TIMER_TYPE_SEND, 0,
+ 1);
+}
+
+/**
+ * irdma_active_open_err - send event for active side cm error
+ * @cm_node: connection's node
+ * @reset: Flag to send reset or not
+ */
+static void irdma_active_open_err(struct irdma_cm_node *cm_node, bool reset)
+{
+ trace_irdma_active_open_err(cm_node, reset,
+ __builtin_return_address(0));
+ irdma_cleanup_retrans_entry(cm_node);
+ cm_node->cm_core->stats_connect_errs++;
+ if (reset) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: cm_node=%p state=%d\n", cm_node,
+ cm_node->state);
+ atomic_inc(&cm_node->ref_count);
+ irdma_send_reset(cm_node);
+ }
+
+ cm_node->state = IRDMA_CM_STATE_CLOSED;
+ irdma_create_event(cm_node, IRDMA_CM_EVENT_ABORTED);
+}
+
+/**
+ * irdma_passive_open_err - handle passive side cm error
+ * @cm_node: connection's node
+ * @reset: send reset or just free cm_node
+ */
+static void irdma_passive_open_err(struct irdma_cm_node *cm_node, bool reset)
+{
+ irdma_cleanup_retrans_entry(cm_node);
+ cm_node->cm_core->stats_passive_errs++;
+ cm_node->state = IRDMA_CM_STATE_CLOSED;
+ ibdev_dbg(to_ibdev(cm_node->iwdev), "CM: cm_node=%p state =%d\n",
+ cm_node, cm_node->state);
+ trace_irdma_passive_open_err(cm_node, reset,
+ __builtin_return_address(0));
+ if (reset)
+ irdma_send_reset(cm_node);
+ else
+ irdma_rem_ref_cm_node(cm_node);
+}
+
+/**
+ * irdma_event_connect_error - to create connect error event
+ * @event: cm information for connect event
+ */
+static void irdma_event_connect_error(struct irdma_cm_event *event)
+{
+ struct irdma_qp *iwqp;
+ struct iw_cm_id *cm_id;
+
+ cm_id = event->cm_node->cm_id;
+ if (!cm_id)
+ return;
+
+ iwqp = cm_id->provider_data;
+
+ if (!iwqp || !iwqp->iwdev)
+ return;
+
+ iwqp->cm_id = NULL;
+ cm_id->provider_data = NULL;
+ irdma_send_cm_event(event->cm_node, cm_id, IW_CM_EVENT_CONNECT_REPLY,
+ -ECONNRESET);
+ cm_id->rem_ref(cm_id);
+ irdma_rem_ref_cm_node(event->cm_node);
+}
+
+/**
+ * irdma_process_options - process options from TCP header
+ * @cm_node: connection's node
+ * @optionsloc: point to start of options
+ * @optionsize: size of all options
+ * @syn_pkt: flag if syn packet
+ */
+static int irdma_process_options(struct irdma_cm_node *cm_node, u8 *optionsloc,
+ u32 optionsize, u32 syn_pkt)
+{
+ u32 tmp;
+ u32 offset = 0;
+ union all_known_options *all_options;
+ char got_mss_option = 0;
+
+ while (offset < optionsize) {
+ all_options = (union all_known_options *)(optionsloc + offset);
+ switch (all_options->base.optionnum) {
+ case OPTION_NUM_EOL:
+ offset = optionsize;
+ break;
+ case OPTION_NUM_NONE:
+ offset += 1;
+ continue;
+ case OPTION_NUM_MSS:
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: MSS Length: %d Offset: %d Size: %d\n",
+ all_options->mss.len, offset, optionsize);
+ got_mss_option = 1;
+ if (all_options->mss.len != 4)
+ return -EINVAL;
+ tmp = ntohs(all_options->mss.mss);
+ if ((cm_node->ipv4 &&
+ (tmp + IRDMA_MTU_TO_MSS_IPV4) < IRDMA_MIN_MTU_IPV4) ||
+ (!cm_node->ipv4 &&
+ (tmp + IRDMA_MTU_TO_MSS_IPV6) < IRDMA_MIN_MTU_IPV6))
+ return -EINVAL;
+ if (tmp < cm_node->tcp_cntxt.mss)
+ cm_node->tcp_cntxt.mss = tmp;
+ break;
+ case OPTION_NUM_WINDOW_SCALE:
+ cm_node->tcp_cntxt.snd_wscale =
+ all_options->windowscale.shiftcount;
+ break;
+ default:
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: Unsupported TCP Option: %x\n",
+ all_options->base.optionnum);
+ break;
+ }
+ offset += all_options->base.len;
+ }
+ if (!got_mss_option && syn_pkt)
+ cm_node->tcp_cntxt.mss = IRDMA_CM_DEFAULT_MSS;
+
+ return 0;
+}
+
+/**
+ * irdma_handle_tcp_options - setup TCP context info after parsing TCP options
+ * @cm_node: connection's node
+ * @tcph: pointer tcp header
+ * @optionsize: size of options rcvd
+ * @passive: active or passive flag
+ */
+static int irdma_handle_tcp_options(struct irdma_cm_node *cm_node,
+ struct tcphdr *tcph, int optionsize,
+ int passive)
+{
+ u8 *optionsloc = (u8 *)&tcph[1];
+
+ if (optionsize) {
+ if (irdma_process_options(cm_node, optionsloc, optionsize,
+ (u32)tcph->syn)) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: Node %p, Sending Reset\n", cm_node);
+ if (passive)
+ irdma_passive_open_err(cm_node, true);
+ else
+ irdma_active_open_err(cm_node, true);
+ return -1;
+ }
+ }
+
+ cm_node->tcp_cntxt.snd_wnd = ntohs(tcph->window)
+ << cm_node->tcp_cntxt.snd_wscale;
+
+ if (cm_node->tcp_cntxt.snd_wnd > cm_node->tcp_cntxt.max_snd_wnd)
+ cm_node->tcp_cntxt.max_snd_wnd = cm_node->tcp_cntxt.snd_wnd;
+
+ return 0;
+}
+
+/**
+ * irdma_build_mpa_v1 - build a MPA V1 frame
+ * @cm_node: connection's node
+ * @start_addr: address where to build frame
+ * @mpa_key: to do read0 or write0
+ */
+static void irdma_build_mpa_v1(struct irdma_cm_node *cm_node, void *start_addr,
+ u8 mpa_key)
+{
+ struct ietf_mpa_v1 *mpa_frame = start_addr;
+
+ switch (mpa_key) {
+ case MPA_KEY_REQUEST:
+ memcpy(mpa_frame->key, IEFT_MPA_KEY_REQ, IETF_MPA_KEY_SIZE);
+ break;
+ case MPA_KEY_REPLY:
+ memcpy(mpa_frame->key, IEFT_MPA_KEY_REP, IETF_MPA_KEY_SIZE);
+ break;
+ default:
+ break;
+ }
+ mpa_frame->flags = IETF_MPA_FLAGS_CRC;
+ mpa_frame->rev = cm_node->mpa_frame_rev;
+ mpa_frame->priv_data_len = htons(cm_node->pdata.size);
+}
+
+/**
+ * irdma_build_mpa_v2 - build a MPA V2 frame
+ * @cm_node: connection's node
+ * @start_addr: buffer start address
+ * @mpa_key: to do read0 or write0
+ */
+static void irdma_build_mpa_v2(struct irdma_cm_node *cm_node, void *start_addr,
+ u8 mpa_key)
+{
+ struct ietf_mpa_v2 *mpa_frame = start_addr;
+ struct ietf_rtr_msg *rtr_msg = &mpa_frame->rtr_msg;
+ u16 ctrl_ird, ctrl_ord;
+
+ /* initialize the upper 5 bytes of the frame */
+ irdma_build_mpa_v1(cm_node, start_addr, mpa_key);
+ mpa_frame->flags |= IETF_MPA_V2_FLAG;
+ mpa_frame->priv_data_len = cpu_to_be16(be16_to_cpu(mpa_frame->priv_data_len) +
+ IETF_RTR_MSG_SIZE);
+
+ /* initialize RTR msg */
+ if (cm_node->mpav2_ird_ord == IETF_NO_IRD_ORD) {
+ ctrl_ird = IETF_NO_IRD_ORD;
+ ctrl_ord = IETF_NO_IRD_ORD;
+ } else {
+ ctrl_ird = (cm_node->ird_size > IETF_NO_IRD_ORD) ?
+ IETF_NO_IRD_ORD :
+ cm_node->ird_size;
+ ctrl_ord = (cm_node->ord_size > IETF_NO_IRD_ORD) ?
+ IETF_NO_IRD_ORD :
+ cm_node->ord_size;
+ }
+ ctrl_ird |= IETF_PEER_TO_PEER;
+
+ switch (mpa_key) {
+ case MPA_KEY_REQUEST:
+ ctrl_ord |= IETF_RDMA0_WRITE;
+ ctrl_ord |= IETF_RDMA0_READ;
+ break;
+ case MPA_KEY_REPLY:
+ switch (cm_node->send_rdma0_op) {
+ case SEND_RDMA_WRITE_ZERO:
+ ctrl_ord |= IETF_RDMA0_WRITE;
+ break;
+ case SEND_RDMA_READ_ZERO:
+ ctrl_ord |= IETF_RDMA0_READ;
+ break;
+ }
+ break;
+ default:
+ break;
+ }
+ rtr_msg->ctrl_ird = htons(ctrl_ird);
+ rtr_msg->ctrl_ord = htons(ctrl_ord);
+}
+
+/**
+ * irdma_cm_build_mpa_frame - build mpa frame for mpa version 1 or version 2
+ * @cm_node: connection's node
+ * @mpa: mpa: data buffer
+ * @mpa_key: to do read0 or write0
+ */
+static int irdma_cm_build_mpa_frame(struct irdma_cm_node *cm_node,
+ struct irdma_kmem_info *mpa, u8 mpa_key)
+{
+ int hdr_len = 0;
+
+ switch (cm_node->mpa_frame_rev) {
+ case IETF_MPA_V1:
+ hdr_len = sizeof(struct ietf_mpa_v1);
+ irdma_build_mpa_v1(cm_node, mpa->addr, mpa_key);
+ break;
+ case IETF_MPA_V2:
+ hdr_len = sizeof(struct ietf_mpa_v2);
+ irdma_build_mpa_v2(cm_node, mpa->addr, mpa_key);
+ break;
+ default:
+ break;
+ }
+
+ return hdr_len;
+}
+
+/**
+ * irdma_send_mpa_request - active node send mpa request to passive node
+ * @cm_node: connection's node
+ */
+static int irdma_send_mpa_request(struct irdma_cm_node *cm_node)
+{
+ struct irdma_puda_buf *sqbuf;
+
+ if (!cm_node) {
+ pr_err("cm_node == NULL\n");
+ return -1;
+ }
+
+ cm_node->mpa_hdr.addr = &cm_node->mpa_frame;
+ cm_node->mpa_hdr.size = irdma_cm_build_mpa_frame(cm_node,
+ &cm_node->mpa_hdr,
+ MPA_KEY_REQUEST);
+ if (!cm_node->mpa_hdr.size) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev), "CM: mpa size = %d\n",
+ cm_node->mpa_hdr.size);
+ return -1;
+ }
+
+ sqbuf = cm_node->cm_core->form_cm_frame(cm_node, NULL,
+ &cm_node->mpa_hdr,
+ &cm_node->pdata, SET_ACK);
+ if (!sqbuf)
+ return -1;
+
+ return irdma_schedule_cm_timer(cm_node, sqbuf, IRDMA_TIMER_TYPE_SEND, 1,
+ 0);
+}
+
+/**
+ * irdma_send_mpa_reject -
+ * @cm_node: connection's node
+ * @pdata: reject data for connection
+ * @plen: length of reject data
+ */
+static int irdma_send_mpa_reject(struct irdma_cm_node *cm_node,
+ const void *pdata, u8 plen)
+{
+ struct irdma_puda_buf *sqbuf;
+ struct irdma_kmem_info priv_info;
+
+ cm_node->mpa_hdr.addr = &cm_node->mpa_frame;
+ cm_node->mpa_hdr.size = irdma_cm_build_mpa_frame(cm_node,
+ &cm_node->mpa_hdr,
+ MPA_KEY_REPLY);
+
+ cm_node->mpa_frame.flags |= IETF_MPA_FLAGS_REJECT;
+ priv_info.addr = (void *)pdata;
+ priv_info.size = plen;
+
+ sqbuf = cm_node->cm_core->form_cm_frame(cm_node, NULL,
+ &cm_node->mpa_hdr, &priv_info,
+ SET_ACK | SET_FIN);
+ if (!sqbuf)
+ return -ENOMEM;
+
+ cm_node->state = IRDMA_CM_STATE_FIN_WAIT1;
+
+ return irdma_schedule_cm_timer(cm_node, sqbuf, IRDMA_TIMER_TYPE_SEND, 1,
+ 0);
+}
+
+/**
+ * irdma_mpav2_negotiate_ird_ord - negotiate MPAv2 IRD/ORD
+ * @cm_node: connection's node
+ * @buf: Data pointer
+ */
+static int irdma_negotiate_mpa_v2_ird_ord(struct irdma_cm_node *cm_node,
+ u8 *buf)
+{
+ struct ietf_mpa_v2 *mpa_v2_frame;
+ struct ietf_rtr_msg *rtr_msg;
+ u16 ird_size;
+ u16 ord_size;
+ u16 ctrl_ord;
+ u16 ctrl_ird;
+
+ mpa_v2_frame = (struct ietf_mpa_v2 *)buf;
+ rtr_msg = &mpa_v2_frame->rtr_msg;
+
+ /* parse rtr message */
+ ctrl_ord = ntohs(rtr_msg->ctrl_ord);
+ ctrl_ird = ntohs(rtr_msg->ctrl_ird);
+ ird_size = ctrl_ird & IETF_NO_IRD_ORD;
+ ord_size = ctrl_ord & IETF_NO_IRD_ORD;
+
+ if (!(ctrl_ird & IETF_PEER_TO_PEER))
+ return -1;
+
+ if (ird_size == IETF_NO_IRD_ORD || ord_size == IETF_NO_IRD_ORD) {
+ cm_node->mpav2_ird_ord = IETF_NO_IRD_ORD;
+ goto negotiate_done;
+ }
+
+ if (cm_node->state != IRDMA_CM_STATE_MPAREQ_SENT) {
+ /* responder */
+ if (!ord_size && (ctrl_ord & IETF_RDMA0_READ))
+ cm_node->ird_size = 1;
+ if (cm_node->ord_size > ird_size)
+ cm_node->ord_size = ird_size;
+ } else {
+ /* initiator */
+ if (!ird_size && (ctrl_ord & IETF_RDMA0_READ))
+ /* Remote peer doesn't support RDMA0_READ */
+ return -1;
+
+ if (cm_node->ord_size > ird_size)
+ cm_node->ord_size = ird_size;
+
+ if (cm_node->ird_size < ord_size)
+ /* no resources available */
+ return -1;
+ }
+
+negotiate_done:
+ if (ctrl_ord & IETF_RDMA0_READ)
+ cm_node->send_rdma0_op = SEND_RDMA_READ_ZERO;
+ else if (ctrl_ord & IETF_RDMA0_WRITE)
+ cm_node->send_rdma0_op = SEND_RDMA_WRITE_ZERO;
+ else
+ /* Not supported RDMA0 operation */
+ return -1;
+
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: MPAV2 Negotiated ORD: %d, IRD: %d\n",
+ cm_node->ord_size, cm_node->ird_size);
+ trace_irdma_negotiate_mpa_v2(cm_node);
+ return 0;
+}
+
+/**
+ * recv_mpa - process an IETF MPA frame
+ * @cm_node: connection's node
+ * @buf: Data pointer
+ * @type: to return accept or reject
+ * @len: Len of mpa buffer
+ */
+static int irdma_parse_mpa(struct irdma_cm_node *cm_node, u8 *buf, u32 *type,
+ u32 len)
+{
+ struct ietf_mpa_v1 *mpa_frame;
+ int mpa_hdr_len;
+ int priv_data_len;
+
+ *type = IRDMA_MPA_REQUEST_ACCEPT;
+
+ if (len < sizeof(struct ietf_mpa_v1)) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: ietf buffer small (%x)\n", len);
+ return -1;
+ }
+
+ mpa_frame = (struct ietf_mpa_v1 *)buf;
+ mpa_hdr_len = sizeof(struct ietf_mpa_v1);
+ priv_data_len = ntohs(mpa_frame->priv_data_len);
+
+ if (priv_data_len > IETF_MAX_PRIV_DATA_LEN) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev), "CM: large pri_data %d\n",
+ priv_data_len);
+ return -1;
+ }
+
+ if (mpa_frame->rev != IETF_MPA_V1 && mpa_frame->rev != IETF_MPA_V2) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: unsupported mpa rev = %d\n", mpa_frame->rev);
+ return -1;
+ }
+
+ if (mpa_frame->rev > cm_node->mpa_frame_rev) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev), "CM: rev %d\n",
+ mpa_frame->rev);
+ return -1;
+ }
+
+ cm_node->mpa_frame_rev = mpa_frame->rev;
+ if (cm_node->state != IRDMA_CM_STATE_MPAREQ_SENT) {
+ if (memcmp(mpa_frame->key, IEFT_MPA_KEY_REQ,
+ IETF_MPA_KEY_SIZE)) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: Unexpected MPA Key received\n");
+ return -1;
+ }
+ } else {
+ if (memcmp(mpa_frame->key, IEFT_MPA_KEY_REP,
+ IETF_MPA_KEY_SIZE)) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: Unexpected MPA Key received\n");
+ return -1;
+ }
+ }
+
+ if (priv_data_len + mpa_hdr_len > len) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: ietf buffer len(%x + %x != %x)\n",
+ priv_data_len, mpa_hdr_len, len);
+ return -1;
+ }
+
+ if (len > IRDMA_MAX_CM_BUF) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: ietf buffer large len = %d\n", len);
+ return -1;
+ }
+
+ switch (mpa_frame->rev) {
+ case IETF_MPA_V2:
+ mpa_hdr_len += IETF_RTR_MSG_SIZE;
+ if (irdma_negotiate_mpa_v2_ird_ord(cm_node, buf))
+ return -1;
+ break;
+ case IETF_MPA_V1:
+ default:
+ break;
+ }
+
+ memcpy(cm_node->pdata_buf, buf + mpa_hdr_len, priv_data_len);
+ cm_node->pdata.size = priv_data_len;
+
+ if (mpa_frame->flags & IETF_MPA_FLAGS_REJECT)
+ *type = IRDMA_MPA_REQUEST_REJECT;
+
+ if (mpa_frame->flags & IETF_MPA_FLAGS_MARKERS)
+ cm_node->snd_mark_en = true;
+
+ return 0;
+}
+
+/**
+ * irdma_schedule_cm_timer
+ * @cm_node: connection's node
+ * @sqbuf: buffer to send
+ * @type: if it is send or close
+ * @send_retrans: if rexmits to be done
+ * @close_when_complete: is cm_node to be removed
+ *
+ * note - cm_node needs to be protected before calling this. Encase in:
+ * irdma_rem_ref_cm_node(cm_core, cm_node);
+ * irdma_schedule_cm_timer(...)
+ * atomic_inc(&cm_node->ref_count);
+ */
+int irdma_schedule_cm_timer(struct irdma_cm_node *cm_node,
+ struct irdma_puda_buf *sqbuf,
+ enum irdma_timer_type type, int send_retrans,
+ int close_when_complete)
+{
+ struct irdma_sc_vsi *vsi = &cm_node->iwdev->vsi;
+ struct irdma_cm_core *cm_core = cm_node->cm_core;
+ struct irdma_timer_entry *new_send;
+ u32 was_timer_set;
+ unsigned long flags;
+
+ new_send = kzalloc(sizeof(*new_send), GFP_ATOMIC);
+ if (!new_send) {
+ if (type != IRDMA_TIMER_TYPE_CLOSE)
+ irdma_free_sqbuf(vsi, (void *)sqbuf);
+ return -ENOMEM;
+ }
+
+ new_send->retrycount = IRDMA_DEFAULT_RETRYS;
+ new_send->retranscount = IRDMA_DEFAULT_RETRANS;
+ new_send->sqbuf = sqbuf;
+ new_send->timetosend = jiffies;
+ new_send->type = type;
+ new_send->send_retrans = send_retrans;
+ new_send->close_when_complete = close_when_complete;
+
+ if (type == IRDMA_TIMER_TYPE_CLOSE) {
+ new_send->timetosend += (HZ / 10);
+ if (cm_node->close_entry) {
+ kfree(new_send);
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: already close entry\n");
+ return -EINVAL;
+ }
+
+ cm_node->close_entry = new_send;
+ } else { /* type == IRDMA_TIMER_TYPE_SEND */
+ spin_lock_irqsave(&cm_node->retrans_list_lock, flags);
+ cm_node->send_entry = new_send;
+ atomic_inc(&cm_node->ref_count);
+ spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags);
+ new_send->timetosend = jiffies + IRDMA_RETRY_TIMEOUT;
+
+ atomic_inc(&sqbuf->refcount);
+ irdma_puda_send_buf(vsi->ilq, sqbuf);
+ if (!send_retrans) {
+ irdma_cleanup_retrans_entry(cm_node);
+ if (close_when_complete)
+ irdma_rem_ref_cm_node(cm_node);
+ return 0;
+ }
+ }
+
+ spin_lock_irqsave(&cm_core->ht_lock, flags);
+ was_timer_set = timer_pending(&cm_core->tcp_timer);
+
+ if (!was_timer_set) {
+ cm_core->tcp_timer.expires = new_send->timetosend;
+ add_timer(&cm_core->tcp_timer);
+ }
+ spin_unlock_irqrestore(&cm_core->ht_lock, flags);
+
+ return 0;
+}
+
+/**
+ * irdma_retrans_expired - Could not rexmit the packet
+ * @cm_node: connection's node
+ */
+static void irdma_retrans_expired(struct irdma_cm_node *cm_node)
+{
+ struct iw_cm_id *cm_id = cm_node->cm_id;
+ enum irdma_cm_node_state state = cm_node->state;
+
+ cm_node->state = IRDMA_CM_STATE_CLOSED;
+ switch (state) {
+ case IRDMA_CM_STATE_SYN_RCVD:
+ case IRDMA_CM_STATE_CLOSING:
+ irdma_rem_ref_cm_node(cm_node);
+ break;
+ case IRDMA_CM_STATE_FIN_WAIT1:
+ case IRDMA_CM_STATE_LAST_ACK:
+ if (cm_node->cm_id)
+ cm_id->rem_ref(cm_id);
+ irdma_send_reset(cm_node);
+ break;
+ default:
+ atomic_inc(&cm_node->ref_count);
+ irdma_send_reset(cm_node);
+ irdma_create_event(cm_node, IRDMA_CM_EVENT_ABORTED);
+ break;
+ }
+}
+
+/**
+ * irdma_handle_close_entry - for handling retry/timeouts
+ * @cm_node: connection's node
+ * @rem_node: flag for remove cm_node
+ */
+static void irdma_handle_close_entry(struct irdma_cm_node *cm_node,
+ u32 rem_node)
+{
+ struct irdma_timer_entry *close_entry = cm_node->close_entry;
+ struct iw_cm_id *cm_id = cm_node->cm_id;
+ struct irdma_qp *iwqp;
+ unsigned long flags;
+
+ if (!close_entry)
+ return;
+ iwqp = (struct irdma_qp *)close_entry->sqbuf;
+ if (iwqp) {
+ spin_lock_irqsave(&iwqp->lock, flags);
+ if (iwqp->cm_id) {
+ iwqp->hw_tcp_state = IRDMA_TCP_STATE_CLOSED;
+ iwqp->hw_iwarp_state = IRDMA_QP_STATE_ERROR;
+ iwqp->last_aeq = IRDMA_AE_RESET_SENT;
+ iwqp->ibqp_state = IB_QPS_ERR;
+ spin_unlock_irqrestore(&iwqp->lock, flags);
+ irdma_cm_disconn(iwqp);
+ } else {
+ spin_unlock_irqrestore(&iwqp->lock, flags);
+ }
+ } else if (rem_node) {
+ /* TIME_WAIT state */
+ irdma_rem_ref_cm_node(cm_node);
+ }
+ if (cm_id)
+ cm_id->rem_ref(cm_id);
+ kfree(close_entry);
+ cm_node->close_entry = NULL;
+}
+
+/**
+ * irdma_build_timer_list - Add cm_nodes to timer list
+ * @timer_list: ptr to timer list
+ * @hte: ptr to accelerated or non-accelerated list
+ */
+static void irdma_build_timer_list(struct list_head *timer_list,
+ struct list_head *hte)
+{
+ struct irdma_cm_node *cm_node;
+ struct list_head *list_core_temp, *list_node;
+
+ list_for_each_safe(list_node, list_core_temp, hte) {
+ cm_node = container_of(list_node, struct irdma_cm_node, list);
+ if (cm_node->close_entry || cm_node->send_entry) {
+ atomic_inc(&cm_node->ref_count);
+ list_add(&cm_node->timer_entry, timer_list);
+ }
+ }
+}
+
+/**
+ * irdma_cm_timer_tick - system's timer expired callback
+ * @t: Pointer to timer_list
+ */
+static void irdma_cm_timer_tick(struct timer_list *t)
+{
+ unsigned long nexttimeout = jiffies + IRDMA_LONG_TIME;
+ struct irdma_cm_node *cm_node;
+ struct irdma_timer_entry *send_entry, *close_entry;
+ struct list_head *list_core_temp;
+ struct list_head *list_node;
+ struct irdma_cm_core *cm_core = from_timer(cm_core, t, tcp_timer);
+ struct irdma_sc_vsi *vsi;
+ u32 settimer = 0;
+ unsigned long timetosend;
+ unsigned long flags;
+ struct list_head timer_list;
+
+ INIT_LIST_HEAD(&timer_list);
+
+ spin_lock_irqsave(&cm_core->ht_lock, flags);
+ irdma_build_timer_list(&timer_list, &cm_core->non_accelerated_list);
+ irdma_build_timer_list(&timer_list, &cm_core->accelerated_list);
+ spin_unlock_irqrestore(&cm_core->ht_lock, flags);
+
+ list_for_each_safe(list_node, list_core_temp, &timer_list) {
+ cm_node = container_of(list_node, struct irdma_cm_node,
+ timer_entry);
+ close_entry = cm_node->close_entry;
+
+ if (close_entry) {
+ if (time_after(close_entry->timetosend, jiffies)) {
+ if (nexttimeout > close_entry->timetosend ||
+ !settimer) {
+ nexttimeout = close_entry->timetosend;
+ settimer = 1;
+ }
+ } else {
+ irdma_handle_close_entry(cm_node, 1);
+ }
+ }
+
+ spin_lock_irqsave(&cm_node->retrans_list_lock, flags);
+
+ send_entry = cm_node->send_entry;
+ if (!send_entry)
+ goto done;
+ if (time_after(send_entry->timetosend, jiffies)) {
+ if (cm_node->state != IRDMA_CM_STATE_OFFLOADED) {
+ if (nexttimeout > send_entry->timetosend ||
+ !settimer) {
+ nexttimeout = send_entry->timetosend;
+ settimer = 1;
+ }
+ } else {
+ irdma_free_retrans_entry(cm_node);
+ }
+ goto done;
+ }
+
+ if (cm_node->state == IRDMA_CM_STATE_OFFLOADED ||
+ cm_node->state == IRDMA_CM_STATE_CLOSED) {
+ irdma_free_retrans_entry(cm_node);
+ goto done;
+ }
+
+ if (!send_entry->retranscount || !send_entry->retrycount) {
+ irdma_free_retrans_entry(cm_node);
+
+ spin_unlock_irqrestore(&cm_node->retrans_list_lock,
+ flags);
+ irdma_retrans_expired(cm_node);
+ cm_node->state = IRDMA_CM_STATE_CLOSED;
+ spin_lock_irqsave(&cm_node->retrans_list_lock, flags);
+ goto done;
+ }
+ spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags);
+
+ vsi = &cm_node->iwdev->vsi;
+ if (!cm_node->ack_rcvd) {
+ atomic_inc(&send_entry->sqbuf->refcount);
+ irdma_puda_send_buf(vsi->ilq, send_entry->sqbuf);
+ cm_node->cm_core->stats_pkt_retrans++;
+ }
+
+ spin_lock_irqsave(&cm_node->retrans_list_lock, flags);
+ if (send_entry->send_retrans) {
+ send_entry->retranscount--;
+ timetosend = (IRDMA_RETRY_TIMEOUT <<
+ (IRDMA_DEFAULT_RETRANS -
+ send_entry->retranscount));
+
+ send_entry->timetosend = jiffies +
+ min(timetosend, IRDMA_MAX_TIMEOUT);
+ if (nexttimeout > send_entry->timetosend || !settimer) {
+ nexttimeout = send_entry->timetosend;
+ settimer = 1;
+ }
+ } else {
+ int close_when_complete;
+
+ close_when_complete = send_entry->close_when_complete;
+ irdma_free_retrans_entry(cm_node);
+ if (close_when_complete)
+ irdma_rem_ref_cm_node(cm_node);
+ }
+done:
+ spin_unlock_irqrestore(&cm_node->retrans_list_lock, flags);
+ irdma_rem_ref_cm_node(cm_node);
+ }
+
+ if (settimer) {
+ spin_lock_irqsave(&cm_core->ht_lock, flags);
+ if (!timer_pending(&cm_core->tcp_timer)) {
+ cm_core->tcp_timer.expires = nexttimeout;
+ add_timer(&cm_core->tcp_timer);
+ }
+ spin_unlock_irqrestore(&cm_core->ht_lock, flags);
+ }
+}
+
+/**
+ * irdma_send_syn - send SYN packet
+ * @cm_node: connection's node
+ * @sendack: flag to set ACK bit or not
+ */
+int irdma_send_syn(struct irdma_cm_node *cm_node, u32 sendack)
+{
+ struct irdma_puda_buf *sqbuf;
+ int flags = SET_SYN;
+ char optionsbuf[sizeof(struct option_mss) +
+ sizeof(struct option_windowscale) +
+ sizeof(struct option_base) + TCP_OPTIONS_PADDING];
+ struct irdma_kmem_info opts;
+ int optionssize = 0;
+ /* Sending MSS option */
+ union all_known_options *options;
+
+ opts.addr = optionsbuf;
+ if (!cm_node)
+ return -EINVAL;
+
+ options = (union all_known_options *)&optionsbuf[optionssize];
+ options->mss.optionnum = OPTION_NUM_MSS;
+ options->mss.len = sizeof(struct option_mss);
+ options->mss.mss = htons(cm_node->tcp_cntxt.mss);
+ optionssize += sizeof(struct option_mss);
+
+ options = (union all_known_options *)&optionsbuf[optionssize];
+ options->windowscale.optionnum = OPTION_NUM_WINDOW_SCALE;
+ options->windowscale.len = sizeof(struct option_windowscale);
+ options->windowscale.shiftcount = cm_node->tcp_cntxt.rcv_wscale;
+ optionssize += sizeof(struct option_windowscale);
+ options = (union all_known_options *)&optionsbuf[optionssize];
+ options->eol = OPTION_NUM_EOL;
+ optionssize += 1;
+
+ if (sendack)
+ flags |= SET_ACK;
+
+ opts.size = optionssize;
+
+ sqbuf = cm_node->cm_core->form_cm_frame(cm_node, &opts, NULL, NULL,
+ flags);
+ if (!sqbuf)
+ return -ENOMEM;
+
+ return irdma_schedule_cm_timer(cm_node, sqbuf, IRDMA_TIMER_TYPE_SEND, 1,
+ 0);
+}
+
+/**
+ * irdma_send_ack - Send ACK packet
+ * @cm_node: connection's node
+ */
+void irdma_send_ack(struct irdma_cm_node *cm_node)
+{
+ struct irdma_puda_buf *sqbuf;
+ struct irdma_sc_vsi *vsi = &cm_node->iwdev->vsi;
+
+ sqbuf = cm_node->cm_core->form_cm_frame(cm_node, NULL, NULL, NULL,
+ SET_ACK);
+ if (sqbuf)
+ irdma_puda_send_buf(vsi->ilq, sqbuf);
+}
+
+/**
+ * irdma_send_fin - Send FIN pkt
+ * @cm_node: connection's node
+ */
+static int irdma_send_fin(struct irdma_cm_node *cm_node)
+{
+ struct irdma_puda_buf *sqbuf;
+
+ sqbuf = cm_node->cm_core->form_cm_frame(cm_node, NULL, NULL, NULL,
+ SET_ACK | SET_FIN);
+ if (!sqbuf)
+ return -1;
+
+ return irdma_schedule_cm_timer(cm_node, sqbuf, IRDMA_TIMER_TYPE_SEND, 1,
+ 0);
+}
+
+/**
+ * irdma_find_node - find a cm node that matches the reference
+ * cm node
+ * @cm_core: cm's core
+ * @rem_port: remote tcp port num
+ * @rem_addr: remote ip addr
+ * @loc_port: local tcp port num
+ * @loc_addr: loc ip addr
+ * @add_refcnt: flag to increment refcount of cm_node
+ * @accelerated_list: flag for accelerated vs non-accelerated list to search
+ */
+struct irdma_cm_node *irdma_find_node(struct irdma_cm_core *cm_core,
+ u16 rem_port, u32 *rem_addr, u16 loc_port,
+ u32 *loc_addr, bool add_refcnt,
+ bool accelerated_list)
+{
+ struct list_head *hte;
+ struct irdma_cm_node *cm_node;
+ unsigned long flags;
+
+ hte = accelerated_list ? &cm_core->accelerated_list :
+ &cm_core->non_accelerated_list;
+
+ spin_lock_irqsave(&cm_core->ht_lock, flags);
+ list_for_each_entry(cm_node, hte, list) {
+ if (!memcmp(cm_node->loc_addr, loc_addr, sizeof(cm_node->loc_addr)) &&
+ cm_node->loc_port == loc_port &&
+ !memcmp(cm_node->rem_addr, rem_addr, sizeof(cm_node->rem_addr)) &&
+ cm_node->rem_port == rem_port) {
+ if (add_refcnt)
+ atomic_inc(&cm_node->ref_count);
+ spin_unlock_irqrestore(&cm_core->ht_lock, flags);
+ trace_irdma_find_node(cm_node, 0, NULL);
+ return cm_node;
+ }
+ }
+ spin_unlock_irqrestore(&cm_core->ht_lock, flags);
+
+ /* no owner node */
+ return NULL;
+}
+
+/**
+ * irdma_find_listener - find a cm node listening on this addr-port pair
+ * @cm_core: cm's core
+ * @dst_addr: listener ip addr
+ * @dst_port: listener tcp port num
+ * @vlan_id: virtual LAN ID
+ * @listener_state: state to match with listen node's
+ */
+static struct irdma_cm_listener *
+irdma_find_listener(struct irdma_cm_core *cm_core, u32 *dst_addr, u16 dst_port,
+ u16 vlan_id, enum irdma_cm_listener_state listener_state)
+{
+ struct irdma_cm_listener *listen_node;
+ static const u32 ip_zero[4] = { 0, 0, 0, 0 };
+ u32 listen_addr[4];
+ u16 listen_port;
+ unsigned long flags;
+
+ /* walk list and find cm_node associated with this session ID */
+ spin_lock_irqsave(&cm_core->listen_list_lock, flags);
+ list_for_each_entry(listen_node, &cm_core->listen_nodes, list) {
+ memcpy(listen_addr, listen_node->loc_addr, sizeof(listen_addr));
+ listen_port = listen_node->loc_port;
+ /* compare node pair, return node handle if a match */
+ if ((!memcmp(listen_addr, dst_addr, sizeof(listen_addr)) ||
+ !memcmp(listen_addr, ip_zero, sizeof(listen_addr))) &&
+ listen_port == dst_port &&
+ vlan_id == listen_node->vlan_id &&
+ (listener_state & listen_node->listener_state)) {
+ atomic_inc(&listen_node->ref_count);
+ spin_unlock_irqrestore(&cm_core->listen_list_lock,
+ flags);
+ trace_irdma_find_listener(listen_node);
+ return listen_node;
+ }
+ }
+ spin_unlock_irqrestore(&cm_core->listen_list_lock, flags);
+
+ return NULL;
+}
+
+/**
+ * irdma_add_hte_node - add a cm node to the hash table
+ * @cm_core: cm's core
+ * @cm_node: connection's node
+ */
+static void irdma_add_hte_node(struct irdma_cm_core *cm_core,
+ struct irdma_cm_node *cm_node)
+{
+ unsigned long flags;
+
+ if (!cm_node || !cm_core)
+ return;
+
+ spin_lock_irqsave(&cm_core->ht_lock, flags);
+ list_add_tail(&cm_node->list, &cm_core->non_accelerated_list);
+ spin_unlock_irqrestore(&cm_core->ht_lock, flags);
+}
+
+/**
+ * irdma_find_port - find port that matches reference port
+ * @hte: ptr to accelerated or non-accelerated list
+ * @port: port number
+ */
+static bool irdma_find_port(struct list_head *hte, u16 port)
+{
+ struct irdma_cm_node *cm_node;
+
+ list_for_each_entry(cm_node, hte, list) {
+ if (cm_node->loc_port == port)
+ return true;
+ }
+
+ return false;
+}
+
+/**
+ * irdma_port_in_use - determine if port is in use
+ * @cm_core: cm's core
+ * @port: port number
+ */
+bool irdma_port_in_use(struct irdma_cm_core *cm_core, u16 port)
+{
+ struct irdma_cm_listener *listen_node;
+ unsigned long flags;
+
+ spin_lock_irqsave(&cm_core->ht_lock, flags);
+ if (irdma_find_port(&cm_core->accelerated_list, port) ||
+ irdma_find_port(&cm_core->non_accelerated_list, port)) {
+ spin_unlock_irqrestore(&cm_core->ht_lock, flags);
+ return true;
+ }
+ spin_unlock_irqrestore(&cm_core->ht_lock, flags);
+
+ spin_lock_irqsave(&cm_core->listen_list_lock, flags);
+ list_for_each_entry(listen_node, &cm_core->listen_nodes, list) {
+ if (listen_node->loc_port == port) {
+ spin_unlock_irqrestore(&cm_core->listen_list_lock, flags);
+ return true;
+ }
+ }
+ spin_unlock_irqrestore(&cm_core->listen_list_lock, flags);
+
+ return false;
+}
+
+/**
+ * irdma_del_multiple_qhash - Remove qhash and child listens
+ * @iwdev: iWarp device
+ * @cm_info: CM info for parent listen node
+ * @cm_parent_listen_node: The parent listen node
+ */
+static enum irdma_status_code
+irdma_del_multiple_qhash(struct irdma_device *iwdev,
+ struct irdma_cm_info *cm_info,
+ struct irdma_cm_listener *cm_parent_listen_node)
+{
+ struct irdma_cm_listener *child_listen_node;
+ enum irdma_status_code ret = IRDMA_ERR_CFG;
+ struct list_head *pos, *tpos;
+ unsigned long flags;
+
+ spin_lock_irqsave(&iwdev->cm_core.listen_list_lock, flags);
+ list_for_each_safe(pos, tpos,
+ &cm_parent_listen_node->child_listen_list) {
+ child_listen_node = list_entry(pos, struct irdma_cm_listener,
+ child_listen_list);
+ if (child_listen_node->ipv4)
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: removing child listen for IP=%pI4, port=%d, vlan=%d\n",
+ child_listen_node->loc_addr,
+ child_listen_node->loc_port,
+ child_listen_node->vlan_id);
+ else
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: removing child listen for IP=%pI6, port=%d, vlan=%d\n",
+ child_listen_node->loc_addr,
+ child_listen_node->loc_port,
+ child_listen_node->vlan_id);
+ trace_irdma_del_multiple_qhash(child_listen_node);
+ list_del(pos);
+ memcpy(cm_info->loc_addr, child_listen_node->loc_addr,
+ sizeof(cm_info->loc_addr));
+ cm_info->vlan_id = child_listen_node->vlan_id;
+ if (child_listen_node->qhash_set) {
+ ret = irdma_manage_qhash(iwdev, cm_info,
+ IRDMA_QHASH_TYPE_TCP_SYN,
+ IRDMA_QHASH_MANAGE_TYPE_DELETE,
+ NULL, false);
+ child_listen_node->qhash_set = false;
+ } else {
+ ret = 0;
+ }
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: Child listen node freed = %p\n",
+ child_listen_node);
+ kfree(child_listen_node);
+ cm_parent_listen_node->cm_core->stats_listen_nodes_destroyed++;
+ }
+ spin_unlock_irqrestore(&iwdev->cm_core.listen_list_lock, flags);
+
+ return ret;
+}
+
+/**
+ * irdma_netdev_vlan_ipv6 - Gets the netdev and mac
+ * @addr: local IPv6 address
+ * @vlan_id: vlan id for the given IPv6 address
+ * @mac: mac address for the given IPv6 address
+ *
+ * Returns the net_device of the IPv6 address and also sets the
+ * vlan id and mac for that address.
+ */
+struct net_device *irdma_netdev_vlan_ipv6(u32 *addr, u16 *vlan_id, u8 *mac)
+{
+ struct net_device *ip_dev = NULL;
+ struct in6_addr laddr6;
+
+ if (!IS_ENABLED(CONFIG_IPV6))
+ return NULL;
+
+ irdma_copy_ip_htonl(laddr6.in6_u.u6_addr32, addr);
+ if (vlan_id)
+ *vlan_id = 0xFFFF; /* Match rdma_vlan_dev_vlan_id() */
+ if (mac)
+ eth_zero_addr(mac);
+
+ rcu_read_lock();
+ for_each_netdev_rcu(&init_net, ip_dev) {
+ if (ipv6_chk_addr(&init_net, &laddr6, ip_dev, 1)) {
+ if (vlan_id)
+ *vlan_id = rdma_vlan_dev_vlan_id(ip_dev);
+ if (ip_dev->dev_addr && mac)
+ ether_addr_copy(mac, ip_dev->dev_addr);
+ break;
+ }
+ }
+ rcu_read_unlock();
+
+ return ip_dev;
+}
+
+/**
+ * irdma_get_vlan_ipv4 - Returns the vlan_id for IPv4 address
+ * @addr: local IPv4 address
+ */
+u16 irdma_get_vlan_ipv4(u32 *addr)
+{
+ struct net_device *netdev;
+ u16 vlan_id = 0xFFFF;
+
+ netdev = ip_dev_find(&init_net, htonl(addr[0]));
+ if (netdev) {
+ vlan_id = rdma_vlan_dev_vlan_id(netdev);
+ dev_put(netdev);
+ }
+
+ return vlan_id;
+}
+
+/**
+ * irdma_add_mqh_6 - Adds multiple qhashes for IPv6
+ * @iwdev: iWarp device
+ * @cm_info: CM info for parent listen node
+ * @cm_parent_listen_node: The parent listen node
+ *
+ * Adds a qhash and a child listen node for every IPv6 address
+ * on the adapter and adds the associated qhash filter
+ */
+static enum irdma_status_code
+irdma_add_mqh_6(struct irdma_device *iwdev, struct irdma_cm_info *cm_info,
+ struct irdma_cm_listener *cm_parent_listen_node)
+{
+ struct net_device *ip_dev;
+ struct inet6_dev *idev;
+ struct inet6_ifaddr *ifp, *tmp;
+ enum irdma_status_code ret = 0;
+ struct irdma_cm_listener *child_listen_node;
+ unsigned long flags;
+
+ rtnl_lock();
+ for_each_netdev(&init_net, ip_dev) {
+ if (!(ip_dev->flags & IFF_UP))
+ continue;
+
+ if (((rdma_vlan_dev_vlan_id(ip_dev) >= VLAN_N_VID) ||
+ (rdma_vlan_dev_real_dev(ip_dev) != iwdev->netdev)) &&
+ ip_dev != iwdev->netdev)
+ continue;
+
+ idev = __in6_dev_get(ip_dev);
+ if (!idev) {
+ ibdev_dbg(to_ibdev(iwdev), "CM: idev == NULL\n");
+ break;
+ }
+ list_for_each_entry_safe(ifp, tmp, &idev->addr_list, if_list) {
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: IP=%pI6, vlan_id=%d, MAC=%pM\n",
+ &ifp->addr, rdma_vlan_dev_vlan_id(ip_dev),
+ ip_dev->dev_addr);
+ child_listen_node = kzalloc(sizeof(*child_listen_node), GFP_KERNEL);
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: Allocating child listener %p\n",
+ child_listen_node);
+ if (!child_listen_node) {
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: listener memory allocation\n");
+ ret = IRDMA_ERR_NO_MEMORY;
+ goto exit;
+ }
+
+ cm_info->vlan_id = rdma_vlan_dev_vlan_id(ip_dev);
+ cm_parent_listen_node->vlan_id = cm_info->vlan_id;
+ memcpy(child_listen_node, cm_parent_listen_node,
+ sizeof(*child_listen_node));
+ irdma_copy_ip_ntohl(child_listen_node->loc_addr,
+ ifp->addr.in6_u.u6_addr32);
+ memcpy(cm_info->loc_addr, child_listen_node->loc_addr,
+ sizeof(cm_info->loc_addr));
+ ret = irdma_manage_qhash(iwdev, cm_info,
+ IRDMA_QHASH_TYPE_TCP_SYN,
+ IRDMA_QHASH_MANAGE_TYPE_ADD,
+ NULL, true);
+ if (ret) {
+ kfree(child_listen_node);
+ continue;
+ }
+
+ trace_irdma_add_mqh_6(iwdev, child_listen_node,
+ ip_dev->dev_addr);
+
+ child_listen_node->qhash_set = true;
+ spin_lock_irqsave(&iwdev->cm_core.listen_list_lock, flags);
+ list_add(&child_listen_node->child_listen_list,
+ &cm_parent_listen_node->child_listen_list);
+ spin_unlock_irqrestore(&iwdev->cm_core.listen_list_lock, flags);
+ cm_parent_listen_node->cm_core->stats_listen_nodes_created++;
+ }
+ }
+exit:
+ rtnl_unlock();
+
+ return ret;
+}
+
+/**
+ * irdma_add_mqh_4 - Adds multiple qhashes for IPv4
+ * @iwdev: iWarp device
+ * @cm_info: CM info for parent listen node
+ * @cm_parent_listen_node: The parent listen node
+ *
+ * Adds a qhash and a child listen node for every IPv4 address
+ * on the adapter and adds the associated qhash filter
+ */
+static enum irdma_status_code
+irdma_add_mqh_4(struct irdma_device *iwdev, struct irdma_cm_info *cm_info,
+ struct irdma_cm_listener *cm_parent_listen_node)
+{
+ struct net_device *dev;
+ struct in_device *idev;
+ struct irdma_cm_listener *child_listen_node;
+ enum irdma_status_code ret = 0;
+ unsigned long flags;
+
+ rtnl_lock();
+ for_each_netdev(&init_net, dev) {
+ if (!(dev->flags & IFF_UP))
+ continue;
+
+ if (((rdma_vlan_dev_vlan_id(dev) >= VLAN_N_VID) ||
+ (rdma_vlan_dev_real_dev(dev) != iwdev->netdev)) &&
+ dev != iwdev->netdev)
+ continue;
+
+ idev = in_dev_get(dev);
+ for_ifa(idev)
+ {
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: Allocating child CM Listener forIP=%pI4, vlan_id=%d, MAC=%pM\n",
+ &ifa->ifa_address,
+ rdma_vlan_dev_vlan_id(dev), dev->dev_addr);
+ child_listen_node = kzalloc(sizeof(*child_listen_node), GFP_KERNEL);
+ cm_parent_listen_node->cm_core->stats_listen_nodes_created++;
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: Allocating child listener %p\n",
+ child_listen_node);
+ if (!child_listen_node) {
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: listener memory allocation\n");
+ in_dev_put(idev);
+ ret = IRDMA_ERR_NO_MEMORY;
+ goto exit;
+ }
+
+ cm_info->vlan_id = rdma_vlan_dev_vlan_id(dev);
+ cm_parent_listen_node->vlan_id = cm_info->vlan_id;
+ memcpy(child_listen_node, cm_parent_listen_node,
+ sizeof(*child_listen_node));
+ child_listen_node->loc_addr[0] =
+ ntohl(ifa->ifa_address);
+ memcpy(cm_info->loc_addr, child_listen_node->loc_addr,
+ sizeof(cm_info->loc_addr));
+ ret = irdma_manage_qhash(iwdev, cm_info,
+ IRDMA_QHASH_TYPE_TCP_SYN,
+ IRDMA_QHASH_MANAGE_TYPE_ADD,
+ NULL, true);
+ if (ret) {
+ kfree(child_listen_node);
+ cm_parent_listen_node->cm_core
+ ->stats_listen_nodes_created--;
+ continue;
+ }
+
+ trace_irdma_add_mqh_4(iwdev, child_listen_node,
+ dev->dev_addr);
+
+ child_listen_node->qhash_set = true;
+ spin_lock_irqsave(&iwdev->cm_core.listen_list_lock,
+ flags);
+ list_add(&child_listen_node->child_listen_list,
+ &cm_parent_listen_node->child_listen_list);
+ spin_unlock_irqrestore(&iwdev->cm_core.listen_list_lock, flags);
+ }
+ endfor_ifa(idev);
+ in_dev_put(idev);
+ }
+exit:
+ rtnl_unlock();
+
+ return ret;
+}
+
+/**
+ * irdma_dec_refcnt_listen - delete listener and associated cm nodes
+ * @cm_core: cm's core
+ * @listener: pointer to listener node
+ * @free_hanging_nodes: to free associated cm_nodes
+ * @apbvt_del: flag to delete the apbvt
+ */
+static int irdma_dec_refcnt_listen(struct irdma_cm_core *cm_core,
+ struct irdma_cm_listener *listener,
+ int free_hanging_nodes, bool apbvt_del)
+{
+ int err;
+ struct list_head *list_pos;
+ struct list_head *list_temp;
+ struct irdma_cm_node *cm_node;
+ struct list_head reset_list;
+ struct irdma_cm_info nfo;
+ enum irdma_cm_node_state old_state;
+ unsigned long flags;
+
+ trace_irdma_dec_refcnt_listen(listener, __builtin_return_address(0));
+ /* free non-accelerated child nodes for this listener */
+ INIT_LIST_HEAD(&reset_list);
+ if (free_hanging_nodes) {
+ spin_lock_irqsave(&cm_core->ht_lock, flags);
+ list_for_each_safe(list_pos, list_temp,
+ &cm_core->non_accelerated_list) {
+ cm_node = container_of(list_pos, struct irdma_cm_node,
+ list);
+ if (cm_node->listener == listener &&
+ !cm_node->accelerated) {
+ atomic_inc(&cm_node->ref_count);
+ list_add(&cm_node->reset_entry, &reset_list);
+ }
+ }
+ spin_unlock_irqrestore(&cm_core->ht_lock, flags);
+ }
+
+ list_for_each_safe(list_pos, list_temp, &reset_list) {
+ cm_node = container_of(list_pos, struct irdma_cm_node,
+ reset_entry);
+ if (cm_node->state >= IRDMA_CM_STATE_FIN_WAIT1) {
+ irdma_rem_ref_cm_node(cm_node);
+ continue;
+ }
+
+ irdma_cleanup_retrans_entry(cm_node);
+ err = irdma_send_reset(cm_node);
+ if (err) {
+ cm_node->state = IRDMA_CM_STATE_CLOSED;
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: send reset failed\n");
+ } else {
+ old_state = cm_node->state;
+ cm_node->state = IRDMA_CM_STATE_LISTENER_DESTROYED;
+ if (old_state != IRDMA_CM_STATE_MPAREQ_RCVD)
+ irdma_rem_ref_cm_node(cm_node);
+ }
+ }
+
+ if (!atomic_dec_return(&listener->ref_count)) {
+ spin_lock_irqsave(&cm_core->listen_list_lock, flags);
+ list_del(&listener->list);
+ spin_unlock_irqrestore(&cm_core->listen_list_lock, flags);
+
+ if (listener->iwdev) {
+ if (apbvt_del)
+ irdma_manage_apbvt(listener->iwdev,
+ listener->loc_port,
+ IRDMA_MANAGE_APBVT_DEL);
+ memcpy(nfo.loc_addr, listener->loc_addr,
+ sizeof(nfo.loc_addr));
+ nfo.loc_port = listener->loc_port;
+ nfo.ipv4 = listener->ipv4;
+ nfo.vlan_id = listener->vlan_id;
+ nfo.user_pri = listener->user_pri;
+ nfo.qh_qpid = listener->iwdev->vsi.ilq->qp_id;
+
+ if (!list_empty(&listener->child_listen_list)) {
+ irdma_del_multiple_qhash(listener->iwdev, &nfo,
+ listener);
+ } else {
+ if (listener->qhash_set)
+ irdma_manage_qhash(listener->iwdev,
+ &nfo,
+ IRDMA_QHASH_TYPE_TCP_SYN,
+ IRDMA_QHASH_MANAGE_TYPE_DELETE,
+ NULL, false);
+ }
+ }
+
+ cm_core->stats_listen_destroyed++;
+ kfree(listener);
+ cm_core->stats_listen_nodes_destroyed++;
+ listener = NULL;
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
+/**
+ * irdma_cm_del_listen - delete a listener
+ * @cm_core: cm's core
+ * @listener: passive connection's listener
+ * @apbvt_del: flag to delete apbvt
+ */
+static int irdma_cm_del_listen(struct irdma_cm_core *cm_core,
+ struct irdma_cm_listener *listener,
+ bool apbvt_del)
+{
+ listener->listener_state = IRDMA_CM_LISTENER_PASSIVE_STATE;
+ listener->cm_id = NULL;
+
+ return irdma_dec_refcnt_listen(cm_core, listener, 1, apbvt_del);
+}
+
+/**
+ * irdma_addr_resolve_neigh - resolve neighbor address
+ * @iwdev: iwarp device structure
+ * @src_ip: local ip address
+ * @dst_ip: remote ip address
+ * @arpindex: if there is an arp entry
+ */
+static int irdma_addr_resolve_neigh(struct irdma_device *iwdev, u32 src_ip,
+ u32 dst_ip, int arpindex)
+{
+ struct rtable *rt;
+ struct neighbour *neigh;
+ int rc = arpindex;
+ __be32 dst_ipaddr = htonl(dst_ip);
+ __be32 src_ipaddr = htonl(src_ip);
+
+ rt = ip_route_output(&init_net, dst_ipaddr, src_ipaddr, 0, 0);
+ if (IS_ERR(rt)) {
+ ibdev_dbg(to_ibdev(iwdev), "CM: ip_route_output fail\n");
+ return -EINVAL;
+ }
+
+ neigh = dst_neigh_lookup(&rt->dst, &dst_ipaddr);
+
+ rcu_read_lock();
+ if (!neigh)
+ goto exit;
+
+ if (neigh->nud_state & NUD_VALID)
+ rc = irdma_add_arp(iwdev->rf, &dst_ip, true, neigh->ha);
+ else
+ neigh_event_send(neigh, NULL);
+exit:
+ rcu_read_unlock();
+
+ if (neigh)
+ neigh_release(neigh);
+
+ ip_rt_put(rt);
+
+ return rc;
+}
+
+/**
+ * irdma_get_dst_ipv6 - get destination cache entry via ipv6 lookup
+ * @src_addr: local ipv6 sock address
+ * @dst_addr: destination ipv6 sock address
+ */
+static struct dst_entry *irdma_get_dst_ipv6(struct sockaddr_in6 *src_addr,
+ struct sockaddr_in6 *dst_addr)
+{
+ struct dst_entry *dst = NULL;
+
+ if ((IS_ENABLED(CONFIG_IPV6))) {
+ struct flowi6 fl6 = {};
+
+ fl6.daddr = dst_addr->sin6_addr;
+ fl6.saddr = src_addr->sin6_addr;
+ if (ipv6_addr_type(&fl6.daddr) & IPV6_ADDR_LINKLOCAL)
+ fl6.flowi6_oif = dst_addr->sin6_scope_id;
+
+ dst = ip6_route_output(&init_net, NULL, &fl6);
+ }
+
+ return dst;
+}
+
+/**
+ * irdma_addr_resolve_neigh_ipv6 - resolve neighbor ipv6 address
+ * @iwdev: iwarp device structure
+ * @src: local ip address
+ * @dest: remote ip address
+ * @arpindex: if there is an arp entry
+ */
+static int irdma_addr_resolve_neigh_ipv6(struct irdma_device *iwdev, u32 *src,
+ u32 *dest, int arpindex)
+{
+ struct neighbour *neigh;
+ int rc = arpindex;
+ struct dst_entry *dst;
+ struct sockaddr_in6 dst_addr = {};
+ struct sockaddr_in6 src_addr = {};
+
+ dst_addr.sin6_family = AF_INET6;
+ irdma_copy_ip_htonl(dst_addr.sin6_addr.in6_u.u6_addr32, dest);
+ src_addr.sin6_family = AF_INET6;
+ irdma_copy_ip_htonl(src_addr.sin6_addr.in6_u.u6_addr32, src);
+ dst = irdma_get_dst_ipv6(&src_addr, &dst_addr);
+ if (!dst || dst->error) {
+ if (dst) {
+ dst_release(dst);
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: ip6_route_output returned dst->error = %d\n",
+ dst->error);
+ }
+ return -EINVAL;
+ }
+
+ neigh = dst_neigh_lookup(dst, dst_addr.sin6_addr.in6_u.u6_addr32);
+
+ rcu_read_lock();
+ if (!neigh)
+ goto exit;
+
+ ibdev_dbg(to_ibdev(iwdev), "CM: dst_neigh_lookup MAC=%pM\n",
+ neigh->ha);
+
+ trace_irdma_addr_resolve(iwdev, neigh->ha);
+
+ if (neigh->nud_state & NUD_VALID)
+ rc = irdma_add_arp(iwdev->rf, dest, false, neigh->ha);
+ else
+ neigh_event_send(neigh, NULL);
+exit:
+
+ rcu_read_unlock();
+ if (neigh)
+ neigh_release(neigh);
+ dst_release(dst);
+
+ return rc;
+}
+
+/**
+ * irdma_ipv4_is_lpb - check if loopback
+ * @loc_addr: local addr to compare
+ * @rem_addr: remote address
+ */
+bool irdma_ipv4_is_lpb(u32 loc_addr, u32 rem_addr)
+{
+ return ipv4_is_loopback(htonl(rem_addr)) || (loc_addr == rem_addr);
+}
+
+/**
+ * irdma_ipv6_is_loopback - check if loopback
+ * @loc_addr: local addr to compare
+ * @rem_addr: remote address
+ */
+bool irdma_ipv6_is_lpb(u32 *loc_addr, u32 *rem_addr)
+{
+ struct in6_addr raddr6;
+
+ irdma_copy_ip_htonl(raddr6.in6_u.u6_addr32, rem_addr);
+
+ return !memcmp(loc_addr, rem_addr, 16) || ipv6_addr_loopback(&raddr6);
+}
+
+/**
+ * irdma_cm_create_ah - create a cm address handle
+ * @cm_node: The connection manager node to create AH for
+ * @wait: Provides option to wait for ah creation or not
+ */
+static int irdma_cm_create_ah(struct irdma_cm_node *cm_node, bool wait)
+{
+ struct irdma_ah_info ah_info = {};
+ struct irdma_device *iwdev = cm_node->iwdev;
+
+ ether_addr_copy(ah_info.mac_addr, iwdev->netdev->dev_addr);
+
+ ah_info.hop_ttl = 0x40;
+ ah_info.tc_tos = cm_node->tos;
+ ah_info.vsi = &iwdev->vsi;
+
+ if (cm_node->ipv4) {
+ ah_info.ipv4_valid = true;
+ ah_info.dest_ip_addr[0] = cm_node->rem_addr[0];
+ ah_info.src_ip_addr[0] = cm_node->loc_addr[0];
+ ah_info.do_lpbk = irdma_ipv4_is_lpb(ah_info.src_ip_addr[0],
+ ah_info.dest_ip_addr[0]);
+ } else {
+ memcpy(ah_info.dest_ip_addr, cm_node->rem_addr,
+ sizeof(ah_info.dest_ip_addr));
+ memcpy(ah_info.src_ip_addr, cm_node->loc_addr,
+ sizeof(ah_info.src_ip_addr));
+ ah_info.do_lpbk = irdma_ipv6_is_lpb(ah_info.src_ip_addr,
+ ah_info.dest_ip_addr);
+ }
+
+ ah_info.vlan_tag = cm_node->vlan_id;
+ if (cm_node->vlan_id < VLAN_N_VID) {
+ ah_info.insert_vlan_tag = 1;
+ ah_info.vlan_tag |= cm_node->user_pri << VLAN_PRIO_SHIFT;
+ }
+
+ ah_info.dst_arpindex =
+ irdma_arp_table(iwdev->rf, ah_info.dest_ip_addr,
+ ah_info.ipv4_valid, NULL, IRDMA_ARP_RESOLVE);
+
+ if (irdma_puda_create_ah(&iwdev->rf->sc_dev, &ah_info, wait,
+ IRDMA_PUDA_RSRC_TYPE_ILQ, cm_node,
+ &cm_node->ah))
+ return -ENOMEM;
+
+ trace_irdma_create_ah(cm_node);
+ return 0;
+}
+
+/**
+ * irdma_cm_free_ah - free a cm address handle
+ * @cm_node: The connection manager node to create AH for
+ */
+static void irdma_cm_free_ah(struct irdma_cm_node *cm_node)
+{
+ struct irdma_device *iwdev = cm_node->iwdev;
+
+ trace_irdma_cm_free_ah(cm_node);
+ irdma_puda_free_ah(&iwdev->rf->sc_dev, cm_node->ah);
+ cm_node->ah = NULL;
+}
+
+/**
+ * irdma_make_cm_node - create a new instance of a cm node
+ * @cm_core: cm's core
+ * @iwdev: iwarp device structure
+ * @cm_info: quad info for connection
+ * @listener: passive connection's listener
+ */
+static struct irdma_cm_node *
+irdma_make_cm_node(struct irdma_cm_core *cm_core, struct irdma_device *iwdev,
+ struct irdma_cm_info *cm_info,
+ struct irdma_cm_listener *listener)
+{
+ struct irdma_cm_node *cm_node;
+ int oldarpindex;
+ int arpindex;
+ struct net_device *netdev = iwdev->netdev;
+
+ /* create an hte and cm_node for this instance */
+ cm_node = kzalloc(sizeof(*cm_node), GFP_ATOMIC);
+ if (!cm_node)
+ return NULL;
+
+ /* set our node specific transport info */
+ cm_node->ipv4 = cm_info->ipv4;
+ cm_node->vlan_id = cm_info->vlan_id;
+ if (cm_node->vlan_id >= VLAN_N_VID && iwdev->dcb)
+ cm_node->vlan_id = 0;
+ cm_node->tos = cm_info->tos;
+ cm_node->user_pri = cm_info->user_pri;
+ if (listener) {
+ if (listener->tos != cm_info->tos)
+ dev_warn(rfdev_to_dev(&iwdev->rf->sc_dev),
+ "application TOS[%d] and remote client TOS[%d] mismatch\n",
+ listener->tos, cm_info->tos);
+ cm_node->tos = max(listener->tos, cm_info->tos);
+ cm_node->user_pri = rt_tos2priority(cm_node->tos);
+ ibdev_dbg(to_ibdev(iwdev),
+ "DCB: listener: TOS:[%d] UP:[%d]\n", cm_node->tos,
+ cm_node->user_pri);
+ trace_irdma_listener_tos(iwdev, cm_node->tos,
+ cm_node->user_pri);
+ }
+ memcpy(cm_node->loc_addr, cm_info->loc_addr, sizeof(cm_node->loc_addr));
+ memcpy(cm_node->rem_addr, cm_info->rem_addr, sizeof(cm_node->rem_addr));
+ cm_node->loc_port = cm_info->loc_port;
+ cm_node->rem_port = cm_info->rem_port;
+
+ cm_node->mpa_frame_rev = IRDMA_CM_DEFAULT_MPA_VER;
+ cm_node->send_rdma0_op = SEND_RDMA_READ_ZERO;
+ cm_node->iwdev = iwdev;
+ cm_node->dev = &iwdev->rf->sc_dev;
+
+ cm_node->ird_size = cm_node->dev->hw_attrs.max_hw_ird;
+ cm_node->ord_size = cm_node->dev->hw_attrs.max_hw_ord;
+
+ cm_node->listener = listener;
+ cm_node->cm_id = cm_info->cm_id;
+ ether_addr_copy(cm_node->loc_mac, netdev->dev_addr);
+ spin_lock_init(&cm_node->retrans_list_lock);
+ cm_node->ack_rcvd = false;
+
+ atomic_set(&cm_node->ref_count, 1);
+ /* associate our parent CM core */
+ cm_node->cm_core = cm_core;
+ cm_node->tcp_cntxt.loc_id = IRDMA_CM_DEFAULT_LOCAL_ID;
+ cm_node->tcp_cntxt.rcv_wscale = iwdev->rcv_wscale;
+ cm_node->tcp_cntxt.rcv_wnd = iwdev->rcv_wnd >> cm_node->tcp_cntxt.rcv_wscale;
+ if (cm_node->ipv4) {
+ cm_node->tcp_cntxt.loc_seq_num = secure_tcp_seq(htonl(cm_node->loc_addr[0]),
+ htonl(cm_node->rem_addr[0]),
+ htons(cm_node->loc_port),
+ htons(cm_node->rem_port));
+ cm_node->tcp_cntxt.mss = iwdev->vsi.mtu - IRDMA_MTU_TO_MSS_IPV4;
+ } else if (IS_ENABLED(CONFIG_IPV6)) {
+ __be32 loc[4] = {
+ htonl(cm_node->loc_addr[0]), htonl(cm_node->loc_addr[1]),
+ htonl(cm_node->loc_addr[2]), htonl(cm_node->loc_addr[3])
+ };
+ __be32 rem[4] = {
+ htonl(cm_node->rem_addr[0]), htonl(cm_node->rem_addr[1]),
+ htonl(cm_node->rem_addr[2]), htonl(cm_node->rem_addr[3])
+ };
+ cm_node->tcp_cntxt.loc_seq_num = secure_tcpv6_seq(loc, rem,
+ htons(cm_node->loc_port),
+ htons(cm_node->rem_port));
+ cm_node->tcp_cntxt.mss = iwdev->vsi.mtu - IRDMA_MTU_TO_MSS_IPV6;
+ }
+
+ if ((cm_node->ipv4 &&
+ irdma_ipv4_is_lpb(cm_node->loc_addr[0], cm_node->rem_addr[0])) ||
+ (!cm_node->ipv4 &&
+ irdma_ipv6_is_lpb(cm_node->loc_addr, cm_node->rem_addr))) {
+ cm_node->do_lpb = true;
+ arpindex = irdma_arp_table(iwdev->rf, cm_node->rem_addr,
+ cm_node->ipv4, NULL,
+ IRDMA_ARP_RESOLVE);
+ } else {
+ oldarpindex = irdma_arp_table(iwdev->rf, cm_node->rem_addr,
+ cm_node->ipv4, NULL,
+ IRDMA_ARP_RESOLVE);
+ if (cm_node->ipv4)
+ arpindex = irdma_addr_resolve_neigh(iwdev,
+ cm_info->loc_addr[0],
+ cm_info->rem_addr[0],
+ oldarpindex);
+ else if (IS_ENABLED(CONFIG_IPV6))
+ arpindex = irdma_addr_resolve_neigh_ipv6(iwdev,
+ cm_info->loc_addr,
+ cm_info->rem_addr,
+ oldarpindex);
+ else
+ arpindex = -EINVAL;
+ }
+
+ if (arpindex < 0)
+ goto err;
+
+ ether_addr_copy(cm_node->rem_mac,
+ iwdev->rf->arp_table[arpindex].mac_addr);
+ irdma_add_hte_node(cm_core, cm_node);
+ cm_core->stats_nodes_created++;
+ return cm_node;
+
+err:
+ kfree(cm_node);
+
+ return NULL;
+}
+
+/**
+ * irdma_rem_ref_cm_node - destroy an instance of a cm node
+ * @cm_node: connection's node
+ */
+static void irdma_rem_ref_cm_node(struct irdma_cm_node *cm_node)
+{
+ struct irdma_cm_core *cm_core = cm_node->cm_core;
+ struct irdma_qp *iwqp;
+ struct irdma_cm_info nfo;
+ unsigned long flags;
+
+ trace_irdma_rem_ref_cm_node(cm_node, 0, __builtin_return_address(0));
+ spin_lock_irqsave(&cm_node->cm_core->ht_lock, flags);
+ if (atomic_dec_return(&cm_node->ref_count)) {
+ spin_unlock_irqrestore(&cm_node->cm_core->ht_lock, flags);
+ return;
+ }
+
+ list_del(&cm_node->list);
+ spin_unlock_irqrestore(&cm_node->cm_core->ht_lock, flags);
+
+ /* if the node is destroyed before connection was accelerated */
+ if (!cm_node->accelerated && cm_node->accept_pend) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: node destroyed before established\n");
+ atomic_dec(&cm_node->listener->pend_accepts_cnt);
+ }
+ if (cm_node->close_entry)
+ irdma_handle_close_entry(cm_node, 0);
+ if (cm_node->listener) {
+ irdma_dec_refcnt_listen(cm_core, cm_node->listener, 0, true);
+ } else {
+ if (cm_node->apbvt_set) {
+ irdma_manage_apbvt(cm_node->iwdev, cm_node->loc_port,
+ IRDMA_MANAGE_APBVT_DEL);
+ cm_node->apbvt_set = 0;
+ }
+ irdma_get_addr_info(cm_node, &nfo);
+ if (cm_node->qhash_set) {
+ nfo.qh_qpid = cm_node->iwdev->vsi.ilq->qp_id;
+ irdma_manage_qhash(cm_node->iwdev, &nfo,
+ IRDMA_QHASH_TYPE_TCP_ESTABLISHED,
+ IRDMA_QHASH_MANAGE_TYPE_DELETE, NULL,
+ false);
+ cm_node->qhash_set = 0;
+ }
+ }
+
+ iwqp = cm_node->iwqp;
+ if (iwqp) {
+ iwqp->cm_node = NULL;
+ irdma_rem_ref(&iwqp->ibqp);
+ cm_node->iwqp = NULL;
+ } else if (cm_node->qhash_set) {
+ irdma_get_addr_info(cm_node, &nfo);
+ nfo.qh_qpid = cm_node->iwdev->vsi.ilq->qp_id;
+ irdma_manage_qhash(cm_node->iwdev, &nfo,
+ IRDMA_QHASH_TYPE_TCP_ESTABLISHED,
+ IRDMA_QHASH_MANAGE_TYPE_DELETE, NULL, false);
+ cm_node->qhash_set = 0;
+ }
+ cm_core->cm_free_ah(cm_node);
+ cm_node->cm_core->stats_nodes_destroyed++;
+ kfree(cm_node);
+}
+
+/**
+ * irdma_handle_fin_pkt - FIN packet received
+ * @cm_node: connection's node
+ */
+static void irdma_handle_fin_pkt(struct irdma_cm_node *cm_node)
+{
+ switch (cm_node->state) {
+ case IRDMA_CM_STATE_SYN_RCVD:
+ case IRDMA_CM_STATE_SYN_SENT:
+ case IRDMA_CM_STATE_ESTABLISHED:
+ case IRDMA_CM_STATE_MPAREJ_RCVD:
+ cm_node->tcp_cntxt.rcv_nxt++;
+ irdma_cleanup_retrans_entry(cm_node);
+ cm_node->state = IRDMA_CM_STATE_LAST_ACK;
+ irdma_send_fin(cm_node);
+ break;
+ case IRDMA_CM_STATE_MPAREQ_SENT:
+ irdma_create_event(cm_node, IRDMA_CM_EVENT_ABORTED);
+ cm_node->tcp_cntxt.rcv_nxt++;
+ irdma_cleanup_retrans_entry(cm_node);
+ cm_node->state = IRDMA_CM_STATE_CLOSED;
+ atomic_inc(&cm_node->ref_count);
+ irdma_send_reset(cm_node);
+ break;
+ case IRDMA_CM_STATE_FIN_WAIT1:
+ cm_node->tcp_cntxt.rcv_nxt++;
+ irdma_cleanup_retrans_entry(cm_node);
+ cm_node->state = IRDMA_CM_STATE_CLOSING;
+ irdma_send_ack(cm_node);
+ /*
+ * Wait for ACK as this is simultaneous close.
+ * After we receive ACK, do not send anything.
+ * Just rm the node.
+ */
+ break;
+ case IRDMA_CM_STATE_FIN_WAIT2:
+ cm_node->tcp_cntxt.rcv_nxt++;
+ irdma_cleanup_retrans_entry(cm_node);
+ cm_node->state = IRDMA_CM_STATE_TIME_WAIT;
+ irdma_send_ack(cm_node);
+ irdma_schedule_cm_timer(cm_node, NULL, IRDMA_TIMER_TYPE_CLOSE,
+ 1, 0);
+ break;
+ case IRDMA_CM_STATE_TIME_WAIT:
+ cm_node->tcp_cntxt.rcv_nxt++;
+ irdma_cleanup_retrans_entry(cm_node);
+ cm_node->state = IRDMA_CM_STATE_CLOSED;
+ irdma_rem_ref_cm_node(cm_node);
+ break;
+ case IRDMA_CM_STATE_OFFLOADED:
+ default:
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: bad state node state = %d\n", cm_node->state);
+ break;
+ }
+}
+
+/**
+ * irdma_handle_rst_pkt - process received RST packet
+ * @cm_node: connection's node
+ * @rbuf: receive buffer
+ */
+static void irdma_handle_rst_pkt(struct irdma_cm_node *cm_node,
+ struct irdma_puda_buf *rbuf)
+{
+ irdma_cleanup_retrans_entry(cm_node);
+ switch (cm_node->state) {
+ case IRDMA_CM_STATE_SYN_SENT:
+ case IRDMA_CM_STATE_MPAREQ_SENT:
+ switch (cm_node->mpa_frame_rev) {
+ case IETF_MPA_V2:
+ /* Drop down to MPA_V1*/
+ cm_node->mpa_frame_rev = IETF_MPA_V1;
+ /* send a syn and goto syn sent state */
+ cm_node->state = IRDMA_CM_STATE_SYN_SENT;
+ if (irdma_send_syn(cm_node, 0))
+ irdma_active_open_err(cm_node, false);
+ break;
+ case IETF_MPA_V1:
+ default:
+ irdma_active_open_err(cm_node, false);
+ break;
+ }
+ break;
+ case IRDMA_CM_STATE_MPAREQ_RCVD:
+ atomic_add_return(1, &cm_node->passive_state);
+ break;
+ case IRDMA_CM_STATE_ESTABLISHED:
+ case IRDMA_CM_STATE_SYN_RCVD:
+ case IRDMA_CM_STATE_LISTENING:
+ irdma_passive_open_err(cm_node, false);
+ break;
+ case IRDMA_CM_STATE_OFFLOADED:
+ irdma_active_open_err(cm_node, false);
+ break;
+ case IRDMA_CM_STATE_CLOSED:
+ break;
+ case IRDMA_CM_STATE_FIN_WAIT2:
+ case IRDMA_CM_STATE_FIN_WAIT1:
+ case IRDMA_CM_STATE_LAST_ACK:
+ cm_node->cm_id->rem_ref(cm_node->cm_id);
+ /* fall through */
+ case IRDMA_CM_STATE_TIME_WAIT:
+ cm_node->state = IRDMA_CM_STATE_CLOSED;
+ irdma_rem_ref_cm_node(cm_node);
+ break;
+ default:
+ break;
+ }
+}
+
+/**
+ * irdma_handle_rcv_mpa - Process a recv'd mpa buffer
+ * @cm_node: connection's node
+ * @rbuf: receive buffer
+ */
+static void irdma_handle_rcv_mpa(struct irdma_cm_node *cm_node,
+ struct irdma_puda_buf *rbuf)
+{
+ int err;
+ int datasize = rbuf->datalen;
+ u8 *dataloc = rbuf->data;
+
+ enum irdma_cm_event_type type = IRDMA_CM_EVENT_UNKNOWN;
+ u32 res_type;
+
+ err = irdma_parse_mpa(cm_node, dataloc, &res_type, datasize);
+ if (err) {
+ if (cm_node->state == IRDMA_CM_STATE_MPAREQ_SENT)
+ irdma_active_open_err(cm_node, true);
+ else
+ irdma_passive_open_err(cm_node, true);
+ return;
+ }
+
+ switch (cm_node->state) {
+ case IRDMA_CM_STATE_ESTABLISHED:
+ if (res_type == IRDMA_MPA_REQUEST_REJECT)
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: state for reject\n");
+ cm_node->state = IRDMA_CM_STATE_MPAREQ_RCVD;
+ type = IRDMA_CM_EVENT_MPA_REQ;
+ irdma_send_ack(cm_node); /* ACK received MPA request */
+ atomic_set(&cm_node->passive_state,
+ IRDMA_PASSIVE_STATE_INDICATED);
+ break;
+ case IRDMA_CM_STATE_MPAREQ_SENT:
+ irdma_cleanup_retrans_entry(cm_node);
+ if (res_type == IRDMA_MPA_REQUEST_REJECT) {
+ type = IRDMA_CM_EVENT_MPA_REJECT;
+ cm_node->state = IRDMA_CM_STATE_MPAREJ_RCVD;
+ } else {
+ type = IRDMA_CM_EVENT_CONNECTED;
+ cm_node->state = IRDMA_CM_STATE_OFFLOADED;
+ }
+ irdma_send_ack(cm_node);
+ break;
+ default:
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: wrong cm_node state =%d\n", cm_node->state);
+ break;
+ }
+ irdma_create_event(cm_node, type);
+}
+
+/**
+ * irdma_check_syn - Check for error on received syn ack
+ * @cm_node: connection's node
+ * @tcph: pointer tcp header
+ */
+static int irdma_check_syn(struct irdma_cm_node *cm_node, struct tcphdr *tcph)
+{
+ if (ntohl(tcph->ack_seq) != cm_node->tcp_cntxt.loc_seq_num) {
+ irdma_active_open_err(cm_node, true);
+ return 1;
+ }
+
+ return 0;
+}
+
+/**
+ * irdma_check_seq - check seq numbers if OK
+ * @cm_node: connection's node
+ * @tcph: pointer tcp header
+ */
+static int irdma_check_seq(struct irdma_cm_node *cm_node, struct tcphdr *tcph)
+{
+ u32 seq;
+ u32 ack_seq;
+ u32 loc_seq_num = cm_node->tcp_cntxt.loc_seq_num;
+ u32 rcv_nxt = cm_node->tcp_cntxt.rcv_nxt;
+ u32 rcv_wnd;
+ int err = 0;
+
+ seq = ntohl(tcph->seq);
+ ack_seq = ntohl(tcph->ack_seq);
+ rcv_wnd = cm_node->tcp_cntxt.rcv_wnd;
+ if (ack_seq != loc_seq_num ||
+ !between(seq, rcv_nxt, (rcv_nxt + rcv_wnd)))
+ err = -1;
+ if (err)
+ ibdev_dbg(to_ibdev(cm_node->iwdev), "CM: seq number err\n");
+
+ return err;
+}
+
+/**
+ * irdma_handle_syn_pkt - is for Passive node
+ * @cm_node: connection's node
+ * @rbuf: receive buffer
+ */
+static void irdma_handle_syn_pkt(struct irdma_cm_node *cm_node,
+ struct irdma_puda_buf *rbuf)
+{
+ struct tcphdr *tcph = (struct tcphdr *)rbuf->tcph;
+ int err;
+ u32 inc_sequence;
+ int optionsize;
+ struct irdma_cm_info nfo;
+
+ optionsize = (tcph->doff << 2) - sizeof(struct tcphdr);
+ inc_sequence = ntohl(tcph->seq);
+
+ switch (cm_node->state) {
+ case IRDMA_CM_STATE_SYN_SENT:
+ case IRDMA_CM_STATE_MPAREQ_SENT:
+ /* Rcvd syn on active open connection */
+ irdma_active_open_err(cm_node, 1);
+ break;
+ case IRDMA_CM_STATE_LISTENING:
+ /* Passive OPEN */
+ if (atomic_read(&cm_node->listener->pend_accepts_cnt) >
+ cm_node->listener->backlog) {
+ cm_node->cm_core->stats_backlog_drops++;
+ irdma_passive_open_err(cm_node, false);
+ break;
+ }
+ err = irdma_handle_tcp_options(cm_node, tcph, optionsize, 1);
+ if (err) {
+ irdma_passive_open_err(cm_node, false);
+ /* drop pkt */
+ break;
+ }
+ err = cm_node->cm_core->cm_create_ah(cm_node, false);
+ if (err) {
+ irdma_passive_open_err(cm_node, false);
+ /* drop pkt */
+ break;
+ }
+ cm_node->tcp_cntxt.rcv_nxt = inc_sequence + 1;
+ cm_node->accept_pend = 1;
+ atomic_inc(&cm_node->listener->pend_accepts_cnt);
+
+ cm_node->state = IRDMA_CM_STATE_SYN_RCVD;
+ irdma_get_addr_info(cm_node, &nfo);
+ nfo.qh_qpid = cm_node->iwdev->vsi.ilq->qp_id;
+ err = irdma_manage_qhash(cm_node->iwdev, &nfo,
+ IRDMA_QHASH_TYPE_TCP_ESTABLISHED,
+ IRDMA_QHASH_MANAGE_TYPE_ADD,
+ cm_node, false);
+ cm_node->qhash_set = true;
+ break;
+ case IRDMA_CM_STATE_CLOSED:
+ irdma_cleanup_retrans_entry(cm_node);
+ atomic_inc(&cm_node->ref_count);
+ irdma_send_reset(cm_node);
+ break;
+ case IRDMA_CM_STATE_OFFLOADED:
+ case IRDMA_CM_STATE_ESTABLISHED:
+ case IRDMA_CM_STATE_FIN_WAIT1:
+ case IRDMA_CM_STATE_FIN_WAIT2:
+ case IRDMA_CM_STATE_MPAREQ_RCVD:
+ case IRDMA_CM_STATE_LAST_ACK:
+ case IRDMA_CM_STATE_CLOSING:
+ case IRDMA_CM_STATE_UNKNOWN:
+ default:
+ break;
+ }
+}
+
+/**
+ * irdma_handle_synack_pkt - Process SYN+ACK packet (active side)
+ * @cm_node: connection's node
+ * @rbuf: receive buffer
+ */
+static void irdma_handle_synack_pkt(struct irdma_cm_node *cm_node,
+ struct irdma_puda_buf *rbuf)
+{
+ struct tcphdr *tcph = (struct tcphdr *)rbuf->tcph;
+ int err;
+ u32 inc_sequence;
+ int optionsize;
+
+ optionsize = (tcph->doff << 2) - sizeof(struct tcphdr);
+ inc_sequence = ntohl(tcph->seq);
+ switch (cm_node->state) {
+ case IRDMA_CM_STATE_SYN_SENT:
+ irdma_cleanup_retrans_entry(cm_node);
+ /* active open */
+ if (irdma_check_syn(cm_node, tcph)) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: check syn fail\n");
+ return;
+ }
+ cm_node->tcp_cntxt.rem_ack_num = ntohl(tcph->ack_seq);
+ /* setup options */
+ err = irdma_handle_tcp_options(cm_node, tcph, optionsize, 0);
+ if (err) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: cm_node=%p tcp_options failed\n",
+ cm_node);
+ break;
+ }
+ irdma_cleanup_retrans_entry(cm_node);
+ cm_node->tcp_cntxt.rcv_nxt = inc_sequence + 1;
+ irdma_send_ack(cm_node); /* ACK for the syn_ack */
+ err = irdma_send_mpa_request(cm_node);
+ if (err) {
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: cm_node=%p irdma_send_mpa_request failed\n",
+ cm_node);
+ break;
+ }
+ cm_node->state = IRDMA_CM_STATE_MPAREQ_SENT;
+ break;
+ case IRDMA_CM_STATE_MPAREQ_RCVD:
+ irdma_passive_open_err(cm_node, true);
+ break;
+ case IRDMA_CM_STATE_LISTENING:
+ cm_node->tcp_cntxt.loc_seq_num = ntohl(tcph->ack_seq);
+ irdma_cleanup_retrans_entry(cm_node);
+ cm_node->state = IRDMA_CM_STATE_CLOSED;
+ irdma_send_reset(cm_node);
+ break;
+ case IRDMA_CM_STATE_CLOSED:
+ cm_node->tcp_cntxt.loc_seq_num = ntohl(tcph->ack_seq);
+ irdma_cleanup_retrans_entry(cm_node);
+ atomic_inc(&cm_node->ref_count);
+ irdma_send_reset(cm_node);
+ break;
+ case IRDMA_CM_STATE_ESTABLISHED:
+ case IRDMA_CM_STATE_FIN_WAIT1:
+ case IRDMA_CM_STATE_FIN_WAIT2:
+ case IRDMA_CM_STATE_LAST_ACK:
+ case IRDMA_CM_STATE_OFFLOADED:
+ case IRDMA_CM_STATE_CLOSING:
+ case IRDMA_CM_STATE_UNKNOWN:
+ case IRDMA_CM_STATE_MPAREQ_SENT:
+ default:
+ break;
+ }
+}
+
+/**
+ * irdma_handle_ack_pkt - process packet with ACK
+ * @cm_node: connection's node
+ * @rbuf: receive buffer
+ */
+static int irdma_handle_ack_pkt(struct irdma_cm_node *cm_node,
+ struct irdma_puda_buf *rbuf)
+{
+ struct tcphdr *tcph = (struct tcphdr *)rbuf->tcph;
+ u32 inc_sequence;
+ int ret;
+ int optionsize;
+ u32 datasize = rbuf->datalen;
+
+ optionsize = (tcph->doff << 2) - sizeof(struct tcphdr);
+
+ if (irdma_check_seq(cm_node, tcph))
+ return -EINVAL;
+
+ inc_sequence = ntohl(tcph->seq);
+ switch (cm_node->state) {
+ case IRDMA_CM_STATE_SYN_RCVD:
+ irdma_cleanup_retrans_entry(cm_node);
+ ret = irdma_handle_tcp_options(cm_node, tcph, optionsize, 1);
+ if (ret)
+ return ret;
+ cm_node->tcp_cntxt.rem_ack_num = ntohl(tcph->ack_seq);
+ cm_node->state = IRDMA_CM_STATE_ESTABLISHED;
+ if (datasize) {
+ cm_node->tcp_cntxt.rcv_nxt = inc_sequence + datasize;
+ irdma_handle_rcv_mpa(cm_node, rbuf);
+ }
+ break;
+ case IRDMA_CM_STATE_ESTABLISHED:
+ irdma_cleanup_retrans_entry(cm_node);
+ if (datasize) {
+ cm_node->tcp_cntxt.rcv_nxt = inc_sequence + datasize;
+ irdma_handle_rcv_mpa(cm_node, rbuf);
+ }
+ break;
+ case IRDMA_CM_STATE_MPAREQ_SENT:
+ cm_node->tcp_cntxt.rem_ack_num = ntohl(tcph->ack_seq);
+ if (datasize) {
+ cm_node->tcp_cntxt.rcv_nxt = inc_sequence + datasize;
+ cm_node->ack_rcvd = false;
+ irdma_handle_rcv_mpa(cm_node, rbuf);
+ } else {
+ cm_node->ack_rcvd = true;
+ }
+ break;
+ case IRDMA_CM_STATE_LISTENING:
+ irdma_cleanup_retrans_entry(cm_node);
+ cm_node->state = IRDMA_CM_STATE_CLOSED;
+ irdma_send_reset(cm_node);
+ break;
+ case IRDMA_CM_STATE_CLOSED:
+ irdma_cleanup_retrans_entry(cm_node);
+ atomic_inc(&cm_node->ref_count);
+ irdma_send_reset(cm_node);
+ break;
+ case IRDMA_CM_STATE_LAST_ACK:
+ case IRDMA_CM_STATE_CLOSING:
+ irdma_cleanup_retrans_entry(cm_node);
+ cm_node->state = IRDMA_CM_STATE_CLOSED;
+ if (!cm_node->accept_pend)
+ cm_node->cm_id->rem_ref(cm_node->cm_id);
+ irdma_rem_ref_cm_node(cm_node);
+ break;
+ case IRDMA_CM_STATE_FIN_WAIT1:
+ irdma_cleanup_retrans_entry(cm_node);
+ cm_node->state = IRDMA_CM_STATE_FIN_WAIT2;
+ break;
+ case IRDMA_CM_STATE_SYN_SENT:
+ case IRDMA_CM_STATE_FIN_WAIT2:
+ case IRDMA_CM_STATE_OFFLOADED:
+ case IRDMA_CM_STATE_MPAREQ_RCVD:
+ case IRDMA_CM_STATE_UNKNOWN:
+ default:
+ irdma_cleanup_retrans_entry(cm_node);
+ break;
+ }
+
+ return 0;
+}
+
+/**
+ * irdma_process_packet - process cm packet
+ * @cm_node: connection's node
+ * @rbuf: receive buffer
+ */
+static void irdma_process_pkt(struct irdma_cm_node *cm_node,
+ struct irdma_puda_buf *rbuf)
+{
+ enum irdma_tcpip_pkt_type pkt_type = IRDMA_PKT_TYPE_UNKNOWN;
+ struct tcphdr *tcph = (struct tcphdr *)rbuf->tcph;
+ u32 fin_set = 0;
+ int err;
+
+ if (tcph->rst) {
+ pkt_type = IRDMA_PKT_TYPE_RST;
+ } else if (tcph->syn) {
+ pkt_type = IRDMA_PKT_TYPE_SYN;
+ if (tcph->ack)
+ pkt_type = IRDMA_PKT_TYPE_SYNACK;
+ } else if (tcph->ack) {
+ pkt_type = IRDMA_PKT_TYPE_ACK;
+ }
+ if (tcph->fin)
+ fin_set = 1;
+
+ switch (pkt_type) {
+ case IRDMA_PKT_TYPE_SYN:
+ irdma_handle_syn_pkt(cm_node, rbuf);
+ break;
+ case IRDMA_PKT_TYPE_SYNACK:
+ irdma_handle_synack_pkt(cm_node, rbuf);
+ break;
+ case IRDMA_PKT_TYPE_ACK:
+ err = irdma_handle_ack_pkt(cm_node, rbuf);
+ if (fin_set && !err)
+ irdma_handle_fin_pkt(cm_node);
+ break;
+ case IRDMA_PKT_TYPE_RST:
+ irdma_handle_rst_pkt(cm_node, rbuf);
+ break;
+ default:
+ if (fin_set &&
+ (!irdma_check_seq(cm_node, (struct tcphdr *)rbuf->tcph)))
+ irdma_handle_fin_pkt(cm_node);
+ break;
+ }
+}
+
+/**
+ * irdma_make_listen_node - create a listen node with params
+ * @cm_core: cm's core
+ * @iwdev: iwarp device structure
+ * @cm_info: quad info for connection
+ */
+static struct irdma_cm_listener *
+irdma_make_listen_node(struct irdma_cm_core *cm_core,
+ struct irdma_device *iwdev,
+ struct irdma_cm_info *cm_info)
+{
+ struct irdma_cm_listener *listener;
+ unsigned long flags;
+
+ /* cannot have multiple matching listeners */
+ listener = irdma_find_listener(cm_core, cm_info->loc_addr,
+ cm_info->loc_port, cm_info->vlan_id,
+ IRDMA_CM_LISTENER_EITHER_STATE);
+ if (listener &&
+ listener->listener_state == IRDMA_CM_LISTENER_ACTIVE_STATE) {
+ atomic_dec(&listener->ref_count);
+ return NULL;
+ }
+
+ if (!listener) {
+ /* create a CM listen node
+ * 1/2 node to compare incoming traffic to
+ */
+ listener = kzalloc(sizeof(*listener), GFP_KERNEL);
+ if (!listener)
+ return NULL;
+ cm_core->stats_listen_nodes_created++;
+ memcpy(listener->loc_addr, cm_info->loc_addr,
+ sizeof(listener->loc_addr));
+ listener->loc_port = cm_info->loc_port;
+
+ INIT_LIST_HEAD(&listener->child_listen_list);
+
+ atomic_set(&listener->ref_count, 1);
+ } else {
+ listener->reused_node = 1;
+ }
+
+ listener->cm_id = cm_info->cm_id;
+ listener->ipv4 = cm_info->ipv4;
+ listener->vlan_id = cm_info->vlan_id;
+ atomic_set(&listener->pend_accepts_cnt, 0);
+ listener->cm_core = cm_core;
+ listener->iwdev = iwdev;
+
+ listener->backlog = cm_info->backlog;
+ listener->listener_state = IRDMA_CM_LISTENER_ACTIVE_STATE;
+
+ if (!listener->reused_node) {
+ spin_lock_irqsave(&cm_core->listen_list_lock, flags);
+ list_add(&listener->list, &cm_core->listen_nodes);
+ spin_unlock_irqrestore(&cm_core->listen_list_lock, flags);
+ }
+
+ return listener;
+}
+
+/**
+ * irdma_create_cm_node - make a connection node with params
+ * @cm_core: cm's core
+ * @iwdev: iwarp device structure
+ * @conn_param: connection parameters
+ * @cm_info: quad info for connection
+ * @caller_cm_node: pointer to cm_node structure to return
+ */
+static int irdma_create_cm_node(struct irdma_cm_core *cm_core,
+ struct irdma_device *iwdev,
+ struct iw_cm_conn_param *conn_param,
+ struct irdma_cm_info *cm_info,
+ struct irdma_cm_node **caller_cm_node)
+{
+ struct irdma_cm_node *cm_node;
+ u16 private_data_len = conn_param->private_data_len;
+ const void *private_data = conn_param->private_data;
+
+ /* create a CM connection node */
+ cm_node = irdma_make_cm_node(cm_core, iwdev, cm_info, NULL);
+ if (!cm_node)
+ return -ENOMEM;
+
+ /* set our node side to client (active) side */
+ cm_node->tcp_cntxt.client = 1;
+ cm_node->tcp_cntxt.rcv_wscale = IRDMA_CM_DEFAULT_RCV_WND_SCALE;
+
+ irdma_record_ird_ord(cm_node, conn_param->ird, conn_param->ord);
+
+ cm_node->pdata.size = private_data_len;
+ cm_node->pdata.addr = cm_node->pdata_buf;
+
+ memcpy(cm_node->pdata_buf, private_data, private_data_len);
+ *caller_cm_node = cm_node;
+
+ return 0;
+}
+
+/**
+ * irdma_cm_reject - reject and teardown a connection
+ * @cm_node: connection's node
+ * @pdata: ptr to private data for reject
+ * @plen: size of private data
+ */
+static int irdma_cm_reject(struct irdma_cm_node *cm_node, const void *pdata,
+ u8 plen)
+{
+ int ret;
+ int passive_state;
+ struct iw_cm_id *cm_id = cm_node->cm_id;
+
+ if (cm_node->tcp_cntxt.client)
+ return 0;
+
+ irdma_cleanup_retrans_entry(cm_node);
+
+ passive_state = atomic_add_return(1, &cm_node->passive_state);
+ if (passive_state == IRDMA_SEND_RESET_EVENT) {
+ cm_node->state = IRDMA_CM_STATE_CLOSED;
+ irdma_rem_ref_cm_node(cm_node);
+ return 0;
+ }
+
+ if (cm_node->state == IRDMA_CM_STATE_LISTENER_DESTROYED) {
+ irdma_rem_ref_cm_node(cm_node);
+ return 0;
+ }
+
+ ret = irdma_send_mpa_reject(cm_node, pdata, plen);
+ if (!ret) {
+ cm_id->add_ref(cm_id);
+ return 0;
+ }
+
+ cm_node->state = IRDMA_CM_STATE_CLOSED;
+ if (irdma_send_reset(cm_node))
+ ibdev_dbg(to_ibdev(cm_node->iwdev), "CM: send reset failed\n");
+ return ret;
+}
+
+/**
+ * irdma_cm_close - close of cm connection
+ * @cm_node: connection's node
+ */
+static int irdma_cm_close(struct irdma_cm_node *cm_node)
+{
+ if (!cm_node)
+ return -EINVAL;
+
+ switch (cm_node->state) {
+ case IRDMA_CM_STATE_SYN_RCVD:
+ case IRDMA_CM_STATE_SYN_SENT:
+ case IRDMA_CM_STATE_ONE_SIDE_ESTABLISHED:
+ case IRDMA_CM_STATE_ESTABLISHED:
+ case IRDMA_CM_STATE_ACCEPTING:
+ case IRDMA_CM_STATE_MPAREQ_SENT:
+ case IRDMA_CM_STATE_MPAREQ_RCVD:
+ irdma_cleanup_retrans_entry(cm_node);
+ irdma_send_reset(cm_node);
+ break;
+ case IRDMA_CM_STATE_CLOSE_WAIT:
+ cm_node->state = IRDMA_CM_STATE_LAST_ACK;
+ irdma_send_fin(cm_node);
+ break;
+ case IRDMA_CM_STATE_FIN_WAIT1:
+ case IRDMA_CM_STATE_FIN_WAIT2:
+ case IRDMA_CM_STATE_LAST_ACK:
+ case IRDMA_CM_STATE_TIME_WAIT:
+ case IRDMA_CM_STATE_CLOSING:
+ return -1;
+ case IRDMA_CM_STATE_LISTENING:
+ irdma_cleanup_retrans_entry(cm_node);
+ irdma_send_reset(cm_node);
+ break;
+ case IRDMA_CM_STATE_MPAREJ_RCVD:
+ case IRDMA_CM_STATE_UNKNOWN:
+ case IRDMA_CM_STATE_INITED:
+ case IRDMA_CM_STATE_CLOSED:
+ case IRDMA_CM_STATE_LISTENER_DESTROYED:
+ irdma_rem_ref_cm_node(cm_node);
+ break;
+ case IRDMA_CM_STATE_OFFLOADED:
+ if (cm_node->send_entry)
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: CM send_entry in OFFLOADED state\n");
+ irdma_rem_ref_cm_node(cm_node);
+ break;
+ }
+
+ return 0;
+}
+
+/**
+ * irdma_receive_ilq - recv an ETHERNET packet, and process it
+ * through CM
+ * @vsi: VSI structure of dev
+ * @rbuf: receive buffer
+ */
+void irdma_receive_ilq(struct irdma_sc_vsi *vsi, struct irdma_puda_buf *rbuf)
+{
+ struct irdma_cm_node *cm_node;
+ struct irdma_cm_listener *listener;
+ struct iphdr *iph;
+ struct ipv6hdr *ip6h;
+ struct tcphdr *tcph;
+ struct irdma_cm_info cm_info = {};
+ struct irdma_device *iwdev = vsi->back_vsi;
+ struct irdma_cm_core *cm_core = &iwdev->cm_core;
+ struct vlan_ethhdr *ethh;
+ u16 vtag;
+
+ /* if vlan, then maclen = 18 else 14 */
+ iph = (struct iphdr *)rbuf->iph;
+ irdma_debug_buf(vsi->dev, IRDMA_DEBUG_ILQ, "RECEIVE ILQ BUFFER",
+ rbuf->mem.va, rbuf->totallen);
+ if (iwdev->rf->sc_dev.hw_attrs.uk_attrs.hw_rev > IRDMA_GEN_1) {
+ if (rbuf->vlan_valid) {
+ vtag = rbuf->vlan_id;
+ cm_info.user_pri = (vtag & VLAN_PRIO_MASK) >>
+ VLAN_PRIO_SHIFT;
+ cm_info.vlan_id = vtag & VLAN_VID_MASK;
+ } else {
+ cm_info.vlan_id = 0xFFFF;
+ }
+ } else {
+ ethh = rbuf->mem.va;
+
+ if (ethh->h_vlan_proto == htons(ETH_P_8021Q)) {
+ vtag = ntohs(ethh->h_vlan_TCI);
+ cm_info.user_pri = (vtag & VLAN_PRIO_MASK) >>
+ VLAN_PRIO_SHIFT;
+ cm_info.vlan_id = vtag & VLAN_VID_MASK;
+ ibdev_dbg(to_ibdev(cm_core->iwdev),
+ "CM: vlan_id=%d\n", cm_info.vlan_id);
+ } else {
+ cm_info.vlan_id = 0xFFFF;
+ }
+ }
+ tcph = (struct tcphdr *)rbuf->tcph;
+
+ if (rbuf->ipv4) {
+ cm_info.loc_addr[0] = ntohl(iph->daddr);
+ cm_info.rem_addr[0] = ntohl(iph->saddr);
+ cm_info.ipv4 = true;
+ cm_info.tos = iph->tos;
+ } else {
+ ip6h = (struct ipv6hdr *)rbuf->iph;
+ irdma_copy_ip_ntohl(cm_info.loc_addr,
+ ip6h->daddr.in6_u.u6_addr32);
+ irdma_copy_ip_ntohl(cm_info.rem_addr,
+ ip6h->saddr.in6_u.u6_addr32);
+ cm_info.ipv4 = false;
+ cm_info.tos = (ip6h->priority << 4) | (ip6h->flow_lbl[0] >> 4);
+ }
+ cm_info.loc_port = ntohs(tcph->dest);
+ cm_info.rem_port = ntohs(tcph->source);
+ cm_node = irdma_find_node(cm_core, cm_info.rem_port, cm_info.rem_addr,
+ cm_info.loc_port, cm_info.loc_addr, true,
+ false);
+
+ if (!cm_node) {
+ /* Only type of packet accepted are for the
+ * PASSIVE open (syn only)
+ */
+ if (!tcph->syn || tcph->ack)
+ return;
+
+ listener = irdma_find_listener(cm_core,
+ cm_info.loc_addr,
+ cm_info.loc_port,
+ cm_info.vlan_id,
+ IRDMA_CM_LISTENER_ACTIVE_STATE);
+ if (!listener) {
+ cm_info.cm_id = NULL;
+ ibdev_dbg(to_ibdev(cm_core->iwdev),
+ "CM: no listener found\n");
+ return;
+ }
+
+ cm_info.cm_id = listener->cm_id;
+ cm_node = irdma_make_cm_node(cm_core, iwdev, &cm_info,
+ listener);
+ if (!cm_node) {
+ ibdev_dbg(to_ibdev(cm_core->iwdev),
+ "CM: allocate node failed\n");
+ atomic_dec(&listener->ref_count);
+ return;
+ }
+
+ if (!tcph->rst && !tcph->fin) {
+ cm_node->state = IRDMA_CM_STATE_LISTENING;
+ } else {
+ irdma_rem_ref_cm_node(cm_node);
+ return;
+ }
+
+ atomic_inc(&cm_node->ref_count);
+ } else if (cm_node->state == IRDMA_CM_STATE_OFFLOADED) {
+ irdma_rem_ref_cm_node(cm_node);
+ return;
+ }
+
+ irdma_process_pkt(cm_node, rbuf);
+ irdma_rem_ref_cm_node(cm_node);
+}
+
+static int irdma_cm_create_ah_nop(struct irdma_cm_node *cm_node, bool wait)
+{
+ return 0;
+}
+
+static void irdma_cm_free_ah_nop(struct irdma_cm_node *cm_node)
+{
+}
+
+/**
+ * irdma_setup_cm_core - setup top level instance of a cm core
+ * @iwdev: iwarp device structure
+ * @rdma_ver: HW version
+ */
+enum irdma_status_code irdma_setup_cm_core(struct irdma_device *iwdev,
+ u8 rdma_ver)
+{
+ struct irdma_cm_core *cm_core = &iwdev->cm_core;
+
+ cm_core->iwdev = iwdev;
+ cm_core->dev = &iwdev->rf->sc_dev;
+
+ cm_core->event_wq = alloc_ordered_workqueue("iwewq", WQ_MEM_RECLAIM);
+ if (!cm_core->event_wq)
+ return IRDMA_ERR_NO_MEMORY;
+ cm_core->disconn_wq = alloc_ordered_workqueue("iwdwq", WQ_MEM_RECLAIM);
+ if (!cm_core->disconn_wq) {
+ destroy_workqueue(cm_core->event_wq);
+ return IRDMA_ERR_NO_MEMORY;
+ }
+
+ INIT_LIST_HEAD(&cm_core->accelerated_list);
+ INIT_LIST_HEAD(&cm_core->non_accelerated_list);
+ INIT_LIST_HEAD(&cm_core->listen_nodes);
+
+ timer_setup(&cm_core->tcp_timer, irdma_cm_timer_tick, 0);
+
+ spin_lock_init(&cm_core->ht_lock);
+ spin_lock_init(&cm_core->listen_list_lock);
+ spin_lock_init(&cm_core->apbvt_lock);
+
+ switch (rdma_ver) {
+ case IRDMA_GEN_1:
+ cm_core->form_cm_frame = irdma_form_uda_cm_frame;
+ cm_core->cm_create_ah = irdma_cm_create_ah_nop;
+ cm_core->cm_free_ah = irdma_cm_free_ah_nop;
+ break;
+ case IRDMA_GEN_2:
+ default:
+ cm_core->form_cm_frame = irdma_form_ah_cm_frame;
+ cm_core->cm_create_ah = irdma_cm_create_ah;
+ cm_core->cm_free_ah = irdma_cm_free_ah;
+ }
+
+ return 0;
+}
+
+/**
+ * irdma_cleanup_cm_core - deallocate a top level instance of a
+ * cm core
+ * @cm_core: cm's core
+ */
+void irdma_cleanup_cm_core(struct irdma_cm_core *cm_core)
+{
+ unsigned long flags;
+
+ if (!cm_core)
+ return;
+
+ spin_lock_irqsave(&cm_core->ht_lock, flags);
+ if (timer_pending(&cm_core->tcp_timer))
+ del_timer_sync(&cm_core->tcp_timer);
+ spin_unlock_irqrestore(&cm_core->ht_lock, flags);
+
+ destroy_workqueue(cm_core->event_wq);
+ destroy_workqueue(cm_core->disconn_wq);
+ cm_core->dev->ws_reset(&cm_core->iwdev->vsi);
+}
+
+/**
+ * irdma_init_tcp_ctx - setup qp context
+ * @cm_node: connection's node
+ * @tcp_info: offload info for tcp
+ * @iwqp: associate qp for the connection
+ */
+static void irdma_init_tcp_ctx(struct irdma_cm_node *cm_node,
+ struct irdma_tcp_offload_info *tcp_info,
+ struct irdma_qp *iwqp)
+{
+ tcp_info->ipv4 = cm_node->ipv4;
+ tcp_info->drop_ooo_seg = !iwqp->iwdev->rf->ooo;
+ tcp_info->wscale = true;
+ tcp_info->ignore_tcp_opt = true;
+ tcp_info->ignore_tcp_uns_opt = true;
+ tcp_info->no_nagle = false;
+
+ tcp_info->ttl = IRDMA_DEFAULT_TTL;
+ tcp_info->rtt_var = IRDMA_DEFAULT_RTT_VAR;
+ tcp_info->ss_thresh = IRDMA_DEFAULT_SS_THRESH;
+ tcp_info->rexmit_thresh = IRDMA_DEFAULT_REXMIT_THRESH;
+
+ tcp_info->tcp_state = IRDMA_TCP_STATE_ESTABLISHED;
+ tcp_info->snd_wscale = cm_node->tcp_cntxt.snd_wscale;
+ tcp_info->rcv_wscale = cm_node->tcp_cntxt.rcv_wscale;
+
+ tcp_info->snd_nxt = cm_node->tcp_cntxt.loc_seq_num;
+ tcp_info->snd_wnd = cm_node->tcp_cntxt.snd_wnd;
+ tcp_info->rcv_nxt = cm_node->tcp_cntxt.rcv_nxt;
+ tcp_info->snd_max = cm_node->tcp_cntxt.loc_seq_num;
+
+ tcp_info->snd_una = cm_node->tcp_cntxt.loc_seq_num;
+ tcp_info->cwnd = 2 * cm_node->tcp_cntxt.mss;
+ tcp_info->snd_wl1 = cm_node->tcp_cntxt.rcv_nxt;
+ tcp_info->snd_wl2 = cm_node->tcp_cntxt.loc_seq_num;
+ tcp_info->max_snd_window = cm_node->tcp_cntxt.max_snd_wnd;
+ tcp_info->rcv_wnd = cm_node->tcp_cntxt.rcv_wnd
+ << cm_node->tcp_cntxt.rcv_wscale;
+
+ tcp_info->flow_label = 0;
+ tcp_info->snd_mss = (u32)cm_node->tcp_cntxt.mss;
+ if (cm_node->vlan_id < VLAN_N_VID) {
+ tcp_info->insert_vlan_tag = true;
+ tcp_info->vlan_tag = cm_node->vlan_id;
+ tcp_info->vlan_tag |= cm_node->user_pri << VLAN_PRIO_SHIFT;
+ tcp_info->tos = cm_node->tos;
+ }
+ if (cm_node->ipv4) {
+ tcp_info->src_port = cm_node->loc_port;
+ tcp_info->dst_port = cm_node->rem_port;
+
+ tcp_info->dest_ip_addr3 = cm_node->rem_addr[0];
+ tcp_info->local_ipaddr3 = cm_node->loc_addr[0];
+ tcp_info->arp_idx = (u16)irdma_arp_table(iwqp->iwdev->rf,
+ &tcp_info->dest_ip_addr3,
+ true, NULL,
+ IRDMA_ARP_RESOLVE);
+ } else {
+ tcp_info->src_port = cm_node->loc_port;
+ tcp_info->dst_port = cm_node->rem_port;
+ tcp_info->dest_ip_addr0 = cm_node->rem_addr[0];
+ tcp_info->dest_ip_addr1 = cm_node->rem_addr[1];
+ tcp_info->dest_ip_addr2 = cm_node->rem_addr[2];
+ tcp_info->dest_ip_addr3 = cm_node->rem_addr[3];
+ tcp_info->local_ipaddr0 = cm_node->loc_addr[0];
+ tcp_info->local_ipaddr1 = cm_node->loc_addr[1];
+ tcp_info->local_ipaddr2 = cm_node->loc_addr[2];
+ tcp_info->local_ipaddr3 = cm_node->loc_addr[3];
+ tcp_info->arp_idx = (u16)irdma_arp_table(iwqp->iwdev->rf,
+ &tcp_info->dest_ip_addr0,
+ false, NULL,
+ IRDMA_ARP_RESOLVE);
+ }
+}
+
+/**
+ * irdma_cm_init_tsa_conn - setup qp for RTS
+ * @iwqp: associate qp for the connection
+ * @cm_node: connection's node
+ */
+static void irdma_cm_init_tsa_conn(struct irdma_qp *iwqp,
+ struct irdma_cm_node *cm_node)
+{
+ struct irdma_iwarp_offload_info *iwarp_info;
+ struct irdma_qp_host_ctx_info *ctx_info;
+ struct irdma_sc_dev *dev = &iwqp->iwdev->rf->sc_dev;
+
+ iwarp_info = &iwqp->iwarp_info;
+ ctx_info = &iwqp->ctx_info;
+
+ ctx_info->tcp_info = &iwqp->tcp_info;
+ ctx_info->send_cq_num = iwqp->iwscq->sc_cq.cq_uk.cq_id;
+ ctx_info->rcv_cq_num = iwqp->iwrcq->sc_cq.cq_uk.cq_id;
+
+ iwarp_info->ord_size = cm_node->ord_size;
+ iwarp_info->ird_size = irdma_derive_hw_ird_setting(cm_node->ird_size);
+ iwarp_info->rd_en = true;
+ iwarp_info->rdmap_ver = 1;
+ iwarp_info->ddp_ver = 1;
+ iwarp_info->pd_id = iwqp->iwpd->sc_pd.pd_id;
+
+ ctx_info->tcp_info_valid = true;
+ ctx_info->iwarp_info_valid = true;
+ ctx_info->user_pri = cm_node->user_pri;
+
+ irdma_init_tcp_ctx(cm_node, &iwqp->tcp_info, iwqp);
+ if (cm_node->snd_mark_en) {
+ iwarp_info->snd_mark_en = true;
+ iwarp_info->snd_mark_offset = (iwqp->tcp_info.snd_nxt & SNDMARKER_SEQNMASK) +
+ cm_node->lsmm_size;
+ }
+
+ cm_node->state = IRDMA_CM_STATE_OFFLOADED;
+ iwqp->tcp_info.tcp_state = IRDMA_TCP_STATE_ESTABLISHED;
+ iwqp->tcp_info.src_mac_addr_idx = iwqp->iwdev->mac_ip_table_idx;
+
+ dev->iw_priv_qp_ops->qp_setctx(&iwqp->sc_qp, iwqp->host_ctx.va,
+ ctx_info);
+
+ /* once tcp_info is set, no need to do it again */
+ ctx_info->tcp_info_valid = false;
+ ctx_info->iwarp_info_valid = false;
+}
+
+/**
+ * irdma_cm_disconn - when a connection is being closed
+ * @iwqp: associated qp for the connection
+ */
+void irdma_cm_disconn(struct irdma_qp *iwqp)
+{
+ struct disconn_work *work;
+ struct irdma_device *iwdev = iwqp->iwdev;
+ struct irdma_cm_core *cm_core = &iwdev->cm_core;
+ unsigned long flags;
+
+ work = kzalloc(sizeof(*work), GFP_ATOMIC);
+ if (!work)
+ return;
+
+ spin_lock_irqsave(&iwdev->rf->qptable_lock, flags);
+ if (!iwdev->rf->qp_table[iwqp->ibqp.qp_num]) {
+ spin_unlock_irqrestore(&iwdev->rf->qptable_lock, flags);
+ ibdev_dbg(to_ibdev(iwdev), "CM: qp_id %d is already freed\n",
+ iwqp->ibqp.qp_num);
+ kfree(work);
+ return;
+ }
+ irdma_add_ref(&iwqp->ibqp);
+ spin_unlock_irqrestore(&iwdev->rf->qptable_lock, flags);
+
+ work->iwqp = iwqp;
+ INIT_WORK(&work->work, irdma_disconnect_worker);
+ queue_work(cm_core->disconn_wq, &work->work);
+}
+
+/**
+ * irdma_qp_disconnect - free qp and close cm
+ * @iwqp: associate qp for the connection
+ */
+static void irdma_qp_disconnect(struct irdma_qp *iwqp)
+{
+ struct irdma_device *iwdev;
+ struct irdma_ib_device *iwibdev;
+
+ iwdev = iwqp->iwdev;
+ iwibdev = iwdev->iwibdev;
+
+ if (iwqp->active_conn) {
+ /* indicate this connection is NOT active */
+ iwqp->active_conn = 0;
+ } else {
+ /* Need to free the Last Streaming Mode Message */
+ if (iwqp->ietf_mem.va) {
+ if (iwqp->lsmm_mr)
+ iwibdev->ibdev.ops.dereg_mr(iwqp->lsmm_mr, NULL);
+ dma_free_coherent(hw_to_dev(iwdev->rf->sc_dev.hw),
+ iwqp->ietf_mem.size,
+ iwqp->ietf_mem.va,
+ iwqp->ietf_mem.pa);
+ iwqp->ietf_mem.va = NULL;
+ }
+ }
+
+ /* close the CM node down if it is still active */
+ if (iwqp->cm_node) {
+ ibdev_dbg(to_ibdev(iwdev), "CM: Call close API\n");
+ irdma_cm_close(iwqp->cm_node);
+ }
+}
+
+/**
+ * irdma_cm_disconn_true - called by worker thread to disconnect qp
+ * @iwqp: associate qp for the connection
+ */
+static void irdma_cm_disconn_true(struct irdma_qp *iwqp)
+{
+ struct iw_cm_id *cm_id;
+ struct irdma_device *iwdev;
+ struct irdma_sc_qp *qp = &iwqp->sc_qp;
+ u16 last_ae;
+ u8 original_hw_tcp_state;
+ u8 original_ibqp_state;
+ int disconn_status = 0;
+ int issue_disconn = 0;
+ int issue_close = 0;
+ int issue_flush = 0;
+ struct ib_event ibevent;
+ unsigned long flags;
+ int err;
+
+ iwdev = iwqp->iwdev;
+ spin_lock_irqsave(&iwqp->lock, flags);
+ if (rdma_protocol_roce(&iwdev->iwibdev->ibdev, 1)) {
+ struct ib_qp_attr attr;
+
+ if (iwqp->flush_issued || iwqp->destroyed) {
+ spin_unlock_irqrestore(&iwqp->lock, flags);
+ return;
+ }
+
+ spin_unlock_irqrestore(&iwqp->lock, flags);
+
+ attr.qp_state = IB_QPS_ERR;
+ irdma_modify_qp_roce(&iwqp->ibqp, &attr, IB_QP_STATE, NULL);
+ if (iwqp->ibqp.event_handler) {
+ ibevent.device = iwqp->ibqp.device;
+ ibevent.event = IB_EVENT_QP_FATAL;
+ ibevent.element.qp = &iwqp->ibqp;
+ iwqp->ibqp.event_handler(&ibevent,
+ iwqp->ibqp.qp_context);
+ }
+ return;
+ }
+
+ cm_id = iwqp->cm_id;
+ /* make sure we havent already closed this connection */
+ if (!cm_id) {
+ spin_unlock_irqrestore(&iwqp->lock, flags);
+ return;
+ }
+
+ original_hw_tcp_state = iwqp->hw_tcp_state;
+ original_ibqp_state = iwqp->ibqp_state;
+ last_ae = iwqp->last_aeq;
+
+ if (qp->term_flags) {
+ issue_disconn = 1;
+ issue_close = 1;
+ iwqp->cm_id = NULL;
+ irdma_terminate_del_timer(qp);
+ if (!iwqp->flush_issued) {
+ iwqp->flush_issued = 1;
+ issue_flush = 1;
+ }
+ } else if ((original_hw_tcp_state == IRDMA_TCP_STATE_CLOSE_WAIT) ||
+ ((original_ibqp_state == IB_QPS_RTS) &&
+ (last_ae == IRDMA_AE_LLP_CONNECTION_RESET))) {
+ issue_disconn = 1;
+ if (last_ae == IRDMA_AE_LLP_CONNECTION_RESET)
+ disconn_status = -ECONNRESET;
+ }
+
+ if ((original_hw_tcp_state == IRDMA_TCP_STATE_CLOSED ||
+ original_hw_tcp_state == IRDMA_TCP_STATE_TIME_WAIT ||
+ last_ae == IRDMA_AE_RDMAP_ROE_BAD_LLP_CLOSE ||
+ last_ae == IRDMA_AE_LLP_CONNECTION_RESET || iwdev->reset)) {
+ issue_close = 1;
+ iwqp->cm_id = NULL;
+ qp->term_flags = 0;
+ if (!iwqp->flush_issued) {
+ iwqp->flush_issued = 1;
+ issue_flush = 1;
+ }
+ }
+
+ spin_unlock_irqrestore(&iwqp->lock, flags);
+ if (issue_flush && !iwqp->destroyed) {
+ /* Flush the queues */
+ irdma_flush_wqes(iwdev->rf, iwqp);
+
+ if (qp->term_flags && iwqp->ibqp.event_handler) {
+ ibevent.device = iwqp->ibqp.device;
+ ibevent.event = (qp->eventtype == TERM_EVENT_QP_FATAL) ?
+ IB_EVENT_QP_FATAL :
+ IB_EVENT_QP_ACCESS_ERR;
+ ibevent.element.qp = &iwqp->ibqp;
+ iwqp->ibqp.event_handler(&ibevent,
+ iwqp->ibqp.qp_context);
+ }
+ }
+
+ if (!cm_id || !cm_id->event_handler)
+ return;
+ if (issue_disconn) {
+ err = irdma_send_cm_event(NULL, cm_id, IW_CM_EVENT_DISCONNECT,
+ disconn_status);
+ if (err)
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: disconnect event failed: - cm_id = %p\n",
+ cm_id);
+ }
+ if (issue_close) {
+ irdma_qp_disconnect(iwqp);
+ cm_id->provider_data = iwqp;
+ err = irdma_send_cm_event(NULL, cm_id, IW_CM_EVENT_CLOSE, 0);
+ if (err)
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: close event failed: - cm_id = %p\n",
+ cm_id);
+ cm_id->rem_ref(cm_id);
+ }
+}
+
+/**
+ * irdma_disconnect_worker - worker for connection close
+ * @work: points or disconn structure
+ */
+static void irdma_disconnect_worker(struct work_struct *work)
+{
+ struct disconn_work *dwork = container_of(work, struct disconn_work, work);
+ struct irdma_qp *iwqp = dwork->iwqp;
+
+ kfree(dwork);
+ irdma_cm_disconn_true(iwqp);
+ irdma_rem_ref(&iwqp->ibqp);
+}
+
+/**
+ * irdma_accept - registered call for connection to be accepted
+ * @cm_id: cm information for passive connection
+ * @conn_param: accpet parameters
+ */
+int irdma_accept(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param)
+{
+ struct ib_qp *ibqp;
+ struct irdma_qp *iwqp;
+ struct irdma_device *iwdev;
+ struct irdma_sc_dev *dev;
+ struct irdma_cm_core *cm_core;
+ struct irdma_cm_node *cm_node;
+ struct ib_qp_attr attr = {};
+ int passive_state;
+ struct ib_mr *ibmr;
+ struct irdma_pd *iwpd;
+ u16 buf_len = 0;
+ struct irdma_kmem_info accept;
+ unsigned long flags;
+ u64 tagged_offset;
+ int wait_ret;
+ int ret;
+
+ ibqp = irdma_get_qp(cm_id->device, conn_param->qpn);
+ if (!ibqp)
+ return -EINVAL;
+
+ iwqp = to_iwqp(ibqp);
+ iwdev = iwqp->iwdev;
+ dev = &iwdev->rf->sc_dev;
+ cm_core = &iwdev->cm_core;
+ cm_node = cm_id->provider_data;
+
+ if (((struct sockaddr_in *)&cm_id->local_addr)->sin_family == AF_INET) {
+ cm_node->ipv4 = true;
+ cm_node->vlan_id = irdma_get_vlan_ipv4(cm_node->loc_addr);
+ } else {
+ cm_node->ipv4 = false;
+ irdma_netdev_vlan_ipv6(cm_node->loc_addr, &cm_node->vlan_id,
+ NULL);
+ }
+ ibdev_dbg(to_ibdev(iwdev), "CM: Accept vlan_id=%d\n",
+ cm_node->vlan_id);
+
+ trace_irdma_accept(cm_node, 0, NULL);
+
+ if (cm_node->state == IRDMA_CM_STATE_LISTENER_DESTROYED) {
+ ret = -EINVAL;
+ goto error;
+ }
+
+ passive_state = atomic_add_return(1, &cm_node->passive_state);
+ if (passive_state == IRDMA_SEND_RESET_EVENT) {
+ ret = -ECONNRESET;
+ goto error;
+ }
+
+ cm_node->cm_core->stats_accepts++;
+ iwqp->cm_node = (void *)cm_node;
+ cm_node->iwqp = iwqp;
+
+ buf_len = conn_param->private_data_len + IRDMA_MAX_IETF_SIZE;
+
+ iwqp->ietf_mem.size = ALIGN(buf_len, 1);
+ iwqp->ietf_mem.va = dma_alloc_coherent(hw_to_dev(dev->hw),
+ iwqp->ietf_mem.size,
+ &iwqp->ietf_mem.pa, GFP_KERNEL);
+ if (!iwqp->ietf_mem.va) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ cm_node->pdata.size = conn_param->private_data_len;
+ accept.addr = iwqp->ietf_mem.va;
+ accept.size = irdma_cm_build_mpa_frame(cm_node, &accept, MPA_KEY_REPLY);
+ memcpy(accept.addr + accept.size, conn_param->private_data,
+ conn_param->private_data_len);
+
+ if (cm_node->dev->ws_add(iwqp->sc_qp.vsi, cm_node->user_pri)) {
+ ret = -ENOMEM;
+ goto error;
+ }
+ irdma_qp_add_qos(&iwqp->sc_qp);
+
+ /* setup our first outgoing iWarp send WQE (the IETF frame response) */
+ iwpd = iwqp->iwpd;
+ tagged_offset = (uintptr_t)iwqp->ietf_mem.va;
+ ibmr = irdma_reg_phys_mr(&iwpd->ibpd, iwqp->ietf_mem.pa, buf_len,
+ IB_ACCESS_LOCAL_WRITE, &tagged_offset);
+ if (IS_ERR(ibmr)) {
+ ret = -ENOMEM;
+ goto error;
+ }
+
+ ibmr->pd = &iwpd->ibpd;
+ ibmr->device = iwpd->ibpd.device;
+ iwqp->lsmm_mr = ibmr;
+ if (iwqp->page)
+ iwqp->sc_qp.qp_uk.sq_base = kmap(iwqp->page);
+ dev->iw_priv_qp_ops->qp_send_lsmm(&iwqp->sc_qp,
+ iwqp->ietf_mem.va,
+ (accept.size + conn_param->private_data_len),
+ ibmr->lkey);
+
+ if (iwqp->page)
+ kunmap(iwqp->page);
+
+ iwqp->cm_id = cm_id;
+ cm_node->cm_id = cm_id;
+
+ cm_id->provider_data = (void *)iwqp;
+ iwqp->active_conn = 0;
+
+ cm_node->lsmm_size = accept.size + conn_param->private_data_len;
+
+ irdma_cm_init_tsa_conn(iwqp, cm_node);
+ cm_id->add_ref(cm_id);
+ irdma_add_ref(&iwqp->ibqp);
+
+ attr.qp_state = IB_QPS_RTS;
+ cm_node->qhash_set = false;
+ cm_node->cm_core->cm_free_ah(cm_node);
+
+ irdma_modify_qp(&iwqp->ibqp, &attr, IB_QP_STATE, NULL);
+ spin_lock_irqsave(&cm_core->ht_lock, flags);
+ list_move_tail(&cm_node->list, &cm_core->accelerated_list);
+ spin_unlock_irqrestore(&cm_core->ht_lock, flags);
+
+ if (dev->hw_attrs.uk_attrs.feature_flags & IRDMA_FEATURE_RTS_AE) {
+ wait_ret = wait_event_interruptible_timeout(iwqp->waitq,
+ iwqp->rts_ae_rcvd,
+ IRDMA_MAX_TIMEOUT);
+ if (!wait_ret)
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: Slow Connection: cm_node=%p, loc_port=%d, rem_port=%d, cm_id=%p\n",
+ cm_node, cm_node->loc_port,
+ cm_node->rem_port, cm_node->cm_id);
+ }
+
+ cm_node->accelerated = true;
+ irdma_send_cm_event(cm_node, cm_id, IW_CM_EVENT_ESTABLISHED, 0);
+
+ if (cm_node->accept_pend) {
+ atomic_dec(&cm_node->listener->pend_accepts_cnt);
+ cm_node->accept_pend = 0;
+ }
+
+ ibdev_dbg(to_ibdev(iwdev), "CM: rem_port=0x%04x, loc_port=0x%04x\n",
+ cm_node->rem_port, cm_node->loc_port);
+
+ return 0;
+error:
+ if (iwqp->ietf_mem.va) {
+ dma_free_coherent(hw_to_dev(dev->hw), iwqp->ietf_mem.size,
+ iwqp->ietf_mem.va, iwqp->ietf_mem.pa);
+ iwqp->ietf_mem.va = NULL;
+ iwqp->ietf_mem.va = NULL;
+ }
+ irdma_rem_ref_cm_node(cm_node);
+
+ return ret;
+}
+
+/**
+ * irdma_reject - registered call for connection to be rejected
+ * @cm_id: cm information for passive connection
+ * @pdata: private data to be sent
+ * @pdata_len: private data length
+ */
+int irdma_reject(struct iw_cm_id *cm_id, const void *pdata, u8 pdata_len)
+{
+ struct irdma_device *iwdev;
+ struct irdma_cm_node *cm_node;
+
+ cm_node = cm_id->provider_data;
+ cm_node->cm_id = cm_id;
+ cm_node->pdata.size = pdata_len;
+
+ trace_irdma_reject(cm_node, 0, NULL);
+
+ iwdev = to_iwdev(cm_id->device);
+ if (!iwdev)
+ return -EINVAL;
+
+ cm_node->cm_core->stats_rejects++;
+
+ if (pdata_len + sizeof(struct ietf_mpa_v2) > IRDMA_MAX_CM_BUF)
+ return -EINVAL;
+
+ return irdma_cm_reject(cm_node, pdata, pdata_len);
+}
+
+/**
+ * irdma_connect - registered call for connection to be established
+ * @cm_id: cm information for passive connection
+ * @conn_param: Information about the connection
+ */
+int irdma_connect(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param)
+{
+ struct ib_qp *ibqp;
+ struct irdma_qp *iwqp;
+ struct irdma_device *iwdev;
+ struct irdma_cm_node *cm_node;
+ struct irdma_cm_info cm_info;
+ struct sockaddr_in *laddr;
+ struct sockaddr_in *raddr;
+ struct sockaddr_in6 *laddr6;
+ struct sockaddr_in6 *raddr6;
+ int ret = 0;
+
+ ibqp = irdma_get_qp(cm_id->device, conn_param->qpn);
+ if (!ibqp)
+ return -EINVAL;
+ iwqp = to_iwqp(ibqp);
+ if (!iwqp)
+ return -EINVAL;
+ iwdev = iwqp->iwdev;
+ if (!iwdev)
+ return -EINVAL;
+
+ laddr = (struct sockaddr_in *)&cm_id->m_local_addr;
+ raddr = (struct sockaddr_in *)&cm_id->m_remote_addr;
+ laddr6 = (struct sockaddr_in6 *)&cm_id->m_local_addr;
+ raddr6 = (struct sockaddr_in6 *)&cm_id->m_remote_addr;
+
+ if (!(laddr->sin_port) || !(raddr->sin_port))
+ return -EINVAL;
+
+ iwqp->active_conn = 1;
+ iwqp->cm_id = NULL;
+ cm_id->provider_data = iwqp;
+
+ /* set up the connection params for the node */
+ if (cm_id->remote_addr.ss_family == AF_INET) {
+ if (iwdev->vsi.mtu < IRDMA_MIN_MTU_IPV4)
+ return -EINVAL;
+
+ cm_info.ipv4 = true;
+ memset(cm_info.loc_addr, 0, sizeof(cm_info.loc_addr));
+ memset(cm_info.rem_addr, 0, sizeof(cm_info.rem_addr));
+ cm_info.loc_addr[0] = ntohl(laddr->sin_addr.s_addr);
+ cm_info.rem_addr[0] = ntohl(raddr->sin_addr.s_addr);
+ cm_info.loc_port = ntohs(laddr->sin_port);
+ cm_info.rem_port = ntohs(raddr->sin_port);
+ cm_info.vlan_id = irdma_get_vlan_ipv4(cm_info.loc_addr);
+ } else {
+ if (iwdev->vsi.mtu < IRDMA_MIN_MTU_IPV6)
+ return -EINVAL;
+
+ cm_info.ipv4 = false;
+ irdma_copy_ip_ntohl(cm_info.loc_addr,
+ laddr6->sin6_addr.in6_u.u6_addr32);
+ irdma_copy_ip_ntohl(cm_info.rem_addr,
+ raddr6->sin6_addr.in6_u.u6_addr32);
+ cm_info.loc_port = ntohs(laddr6->sin6_port);
+ cm_info.rem_port = ntohs(raddr6->sin6_port);
+ irdma_netdev_vlan_ipv6(cm_info.loc_addr, &cm_info.vlan_id,
+ NULL);
+ }
+ cm_info.cm_id = cm_id;
+ cm_info.qh_qpid = iwdev->vsi.ilq->qp_id;
+ cm_info.tos = cm_id->tos;
+ cm_info.user_pri = rt_tos2priority(cm_id->tos);
+
+ if (iwqp->sc_qp.dev->ws_add(iwqp->sc_qp.vsi, cm_info.user_pri))
+ return -ENOMEM;
+ irdma_qp_add_qos(&iwqp->sc_qp);
+
+ ibdev_dbg(to_ibdev(iwdev), "DCB: DCB: TOS:[%d] UP:[%d]\n", cm_id->tos,
+ cm_info.user_pri);
+
+ trace_irdma_dcb_tos(iwdev, cm_id->tos, cm_info.user_pri);
+
+ cm_id->add_ref(cm_id);
+ ret = irdma_create_cm_node(&iwdev->cm_core, iwdev, conn_param, &cm_info,
+ &cm_node);
+ if (ret) {
+ cm_id->rem_ref(cm_id);
+ return ret;
+ }
+ ret = cm_node->cm_core->cm_create_ah(cm_node, true);
+ if (ret)
+ goto err;
+ if (irdma_manage_qhash(iwdev, &cm_info,
+ IRDMA_QHASH_TYPE_TCP_ESTABLISHED,
+ IRDMA_QHASH_MANAGE_TYPE_ADD, NULL, true)) {
+ ret = -EINVAL;
+ goto err;
+ }
+ cm_node->qhash_set = true;
+
+ if (irdma_manage_apbvt(iwdev, cm_info.loc_port,
+ IRDMA_MANAGE_APBVT_ADD)) {
+ ret = -EINVAL;
+ goto err;
+ }
+
+ cm_node->apbvt_set = true;
+ iwqp->cm_node = cm_node;
+ cm_node->iwqp = iwqp;
+ iwqp->cm_id = cm_id;
+ irdma_add_ref(&iwqp->ibqp);
+
+ if (cm_node->state != IRDMA_CM_STATE_OFFLOADED) {
+ cm_node->state = IRDMA_CM_STATE_SYN_SENT;
+ ret = irdma_send_syn(cm_node, 0);
+ if (ret)
+ goto err;
+ }
+
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: rem_port=0x%04x, loc_port=0x%04x, cm_node=%p, cm_id = %p.\n",
+ cm_node->rem_port, cm_node->loc_port, cm_node,
+ cm_node->cm_id);
+
+ trace_irdma_connect(cm_node, 0, NULL);
+
+ return 0;
+
+err:
+ if (cm_info.ipv4)
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: connect() FAILED: dest addr=%pI4",
+ cm_info.rem_addr);
+ else
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: connect() FAILED: dest addr=%pI6",
+ cm_info.rem_addr);
+ irdma_rem_ref_cm_node(cm_node);
+ cm_id->rem_ref(cm_id);
+ iwdev->cm_core.stats_connect_errs++;
+
+ return ret;
+}
+
+/**
+ * irdma_create_listen - registered call creating listener
+ * @cm_id: cm information for passive connection
+ * @backlog: to max accept pending count
+ */
+int irdma_create_listen(struct iw_cm_id *cm_id, int backlog)
+{
+ struct irdma_device *iwdev;
+ struct irdma_cm_listener *cm_listen_node;
+ struct irdma_cm_info cm_info = {};
+ enum irdma_status_code err;
+ struct sockaddr_in *laddr;
+ struct sockaddr_in6 *laddr6;
+ bool wildcard = false;
+
+ iwdev = to_iwdev(cm_id->device);
+ if (!iwdev)
+ return -EINVAL;
+
+ laddr = (struct sockaddr_in *)&cm_id->m_local_addr;
+ laddr6 = (struct sockaddr_in6 *)&cm_id->m_local_addr;
+ cm_info.qh_qpid = iwdev->vsi.ilq->qp_id;
+
+ if (laddr->sin_family == AF_INET) {
+ if (iwdev->vsi.mtu < IRDMA_MIN_MTU_IPV4)
+ return -EINVAL;
+
+ cm_info.ipv4 = true;
+ cm_info.loc_addr[0] = ntohl(laddr->sin_addr.s_addr);
+ cm_info.loc_port = ntohs(laddr->sin_port);
+
+ if (laddr->sin_addr.s_addr != htonl(INADDR_ANY)) {
+ cm_info.vlan_id = irdma_get_vlan_ipv4(cm_info.loc_addr);
+ } else {
+ cm_info.vlan_id = 0xFFFF;
+ wildcard = true;
+ }
+ } else {
+ if (iwdev->vsi.mtu < IRDMA_MIN_MTU_IPV6)
+ return -EINVAL;
+
+ cm_info.ipv4 = false;
+ irdma_copy_ip_ntohl(cm_info.loc_addr,
+ laddr6->sin6_addr.in6_u.u6_addr32);
+ cm_info.loc_port = ntohs(laddr6->sin6_port);
+ if (ipv6_addr_type(&laddr6->sin6_addr) != IPV6_ADDR_ANY) {
+ irdma_netdev_vlan_ipv6(cm_info.loc_addr,
+ &cm_info.vlan_id, NULL);
+ } else {
+ cm_info.vlan_id = 0xFFFF;
+ wildcard = true;
+ }
+ }
+
+ if (cm_info.vlan_id >= VLAN_N_VID && iwdev->dcb)
+ cm_info.vlan_id = 0;
+ cm_info.backlog = backlog;
+ cm_info.cm_id = cm_id;
+
+ trace_irdma_create_listen(iwdev, &cm_info);
+
+ cm_listen_node = irdma_make_listen_node(&iwdev->cm_core, iwdev,
+ &cm_info);
+ if (!cm_listen_node) {
+ ibdev_dbg(to_ibdev(iwdev), "CM: cm_listen_node == NULL\n");
+ return -ENOMEM;
+ }
+
+ cm_id->provider_data = cm_listen_node;
+
+ cm_listen_node->tos = cm_id->tos;
+ cm_listen_node->user_pri = rt_tos2priority(cm_id->tos);
+ cm_info.user_pri = cm_listen_node->user_pri;
+ if (!cm_listen_node->reused_node) {
+ if (wildcard) {
+ if (cm_info.ipv4)
+ err = irdma_add_mqh_4(iwdev, &cm_info,
+ cm_listen_node);
+ else
+ err = irdma_add_mqh_6(iwdev, &cm_info,
+ cm_listen_node);
+ if (err)
+ goto error;
+
+ err = irdma_manage_apbvt(iwdev, cm_info.loc_port,
+ IRDMA_MANAGE_APBVT_ADD);
+
+ if (err)
+ goto error;
+ } else {
+ err = irdma_manage_qhash(iwdev, &cm_info,
+ IRDMA_QHASH_TYPE_TCP_SYN,
+ IRDMA_QHASH_MANAGE_TYPE_ADD,
+ NULL, true);
+ if (err)
+ goto error;
+
+ cm_listen_node->qhash_set = true;
+ err = irdma_manage_apbvt(iwdev, cm_info.loc_port,
+ IRDMA_MANAGE_APBVT_ADD);
+ if (err)
+ goto error;
+ }
+ }
+ cm_id->add_ref(cm_id);
+ cm_listen_node->cm_core->stats_listen_created++;
+ return 0;
+
+error:
+
+ irdma_cm_del_listen(&iwdev->cm_core, (void *)cm_listen_node, false);
+
+ return -EINVAL;
+}
+
+/**
+ * irdma_destroy_listen - registered call to destroy listener
+ * @cm_id: cm information for passive connection
+ */
+int irdma_destroy_listen(struct iw_cm_id *cm_id)
+{
+ struct irdma_device *iwdev;
+
+ iwdev = to_iwdev(cm_id->device);
+ if (cm_id->provider_data)
+ irdma_cm_del_listen(&iwdev->cm_core, cm_id->provider_data,
+ true);
+ else
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: cm_id->provider_data was NULL\n");
+
+ cm_id->rem_ref(cm_id);
+
+ return 0;
+}
+
+/**
+ * irdma_cm_event_connected - handle connected active node
+ * @event: the info for cm_node of connection
+ */
+static void irdma_cm_event_connected(struct irdma_cm_event *event)
+{
+ struct irdma_qp *iwqp;
+ struct irdma_device *iwdev;
+ struct irdma_cm_core *cm_core;
+ struct irdma_cm_node *cm_node;
+ struct irdma_sc_dev *dev;
+ struct ib_qp_attr attr = {};
+ struct iw_cm_id *cm_id;
+ unsigned long flags;
+ int status;
+ bool read0;
+ int wait_ret = 0;
+
+ cm_node = event->cm_node;
+ cm_id = cm_node->cm_id;
+ iwqp = cm_id->provider_data;
+ iwdev = iwqp->iwdev;
+ dev = &iwdev->rf->sc_dev;
+ cm_core = &iwdev->cm_core;
+
+ if (iwqp->destroyed) {
+ status = -ETIMEDOUT;
+ goto error;
+ }
+
+ irdma_cm_init_tsa_conn(iwqp, cm_node);
+ read0 = (cm_node->send_rdma0_op == SEND_RDMA_READ_ZERO);
+ if (iwqp->page)
+ iwqp->sc_qp.qp_uk.sq_base = kmap(iwqp->page);
+ dev->iw_priv_qp_ops->qp_send_rtt(&iwqp->sc_qp, read0);
+ if (iwqp->page)
+ kunmap(iwqp->page);
+
+ attr.qp_state = IB_QPS_RTS;
+ cm_node->qhash_set = false;
+ irdma_modify_qp(&iwqp->ibqp, &attr, IB_QP_STATE, NULL);
+ spin_lock_irqsave(&cm_core->ht_lock, flags);
+ list_move_tail(&cm_node->list, &cm_core->accelerated_list);
+ spin_unlock_irqrestore(&cm_core->ht_lock, flags);
+
+ if (dev->hw_attrs.uk_attrs.feature_flags & IRDMA_FEATURE_RTS_AE) {
+ wait_ret = wait_event_interruptible_timeout(iwqp->waitq,
+ iwqp->rts_ae_rcvd,
+ IRDMA_MAX_TIMEOUT);
+ if (!wait_ret)
+ ibdev_dbg(to_ibdev(iwdev),
+ "CM: Slow Connection: cm_node=%p, loc_port=%d, rem_port=%d, cm_id=%p\n",
+ cm_node, cm_node->loc_port,
+ cm_node->rem_port, cm_node->cm_id);
+ }
+
+ cm_node->accelerated = true;
+ irdma_send_cm_event(cm_node, cm_id, IW_CM_EVENT_CONNECT_REPLY, 0);
+ cm_node->cm_core->cm_free_ah(cm_node);
+ return;
+
+error:
+ iwqp->cm_id = NULL;
+ cm_id->provider_data = NULL;
+ irdma_send_cm_event(event->cm_node, cm_id, IW_CM_EVENT_CONNECT_REPLY,
+ status);
+ cm_id->rem_ref(cm_id);
+ irdma_rem_ref_cm_node(event->cm_node);
+}
+
+/**
+ * irdma_cm_event_reset - handle reset
+ * @event: the info for cm_node of connection
+ */
+static void irdma_cm_event_reset(struct irdma_cm_event *event)
+{
+ struct irdma_cm_node *cm_node = event->cm_node;
+ struct iw_cm_id *cm_id = cm_node->cm_id;
+ struct irdma_qp *iwqp;
+
+ if (!cm_id)
+ return;
+
+ iwqp = cm_id->provider_data;
+ if (!iwqp)
+ return;
+
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: reset event %p - cm_id = %p\n", event->cm_node, cm_id);
+ iwqp->cm_id = NULL;
+
+ irdma_send_cm_event(cm_node, cm_node->cm_id, IW_CM_EVENT_DISCONNECT,
+ -ECONNRESET);
+ irdma_send_cm_event(cm_node, cm_node->cm_id, IW_CM_EVENT_CLOSE, 0);
+}
+
+/**
+ * irdma_cm_event_handler - send event to cm upper layer
+ * @work: pointer of cm event info.
+ */
+static void irdma_cm_event_handler(struct work_struct *work)
+{
+ struct irdma_cm_event *event = container_of(work, struct irdma_cm_event, event_work);
+ struct irdma_cm_node *cm_node;
+
+ if (!event || !event->cm_node || !event->cm_node->cm_core)
+ return;
+
+ cm_node = event->cm_node;
+ trace_irdma_cm_event_handler(cm_node, event->type, NULL);
+
+ switch (event->type) {
+ case IRDMA_CM_EVENT_MPA_REQ:
+ irdma_send_cm_event(cm_node, cm_node->cm_id,
+ IW_CM_EVENT_CONNECT_REQUEST, 0);
+ break;
+ case IRDMA_CM_EVENT_RESET:
+ irdma_cm_event_reset(event);
+ break;
+ case IRDMA_CM_EVENT_CONNECTED:
+ if (!event->cm_node->cm_id ||
+ event->cm_node->state != IRDMA_CM_STATE_OFFLOADED)
+ break;
+ irdma_cm_event_connected(event);
+ break;
+ case IRDMA_CM_EVENT_MPA_REJECT:
+ if (!event->cm_node->cm_id ||
+ cm_node->state == IRDMA_CM_STATE_OFFLOADED)
+ break;
+ irdma_send_cm_event(cm_node, cm_node->cm_id,
+ IW_CM_EVENT_CONNECT_REPLY, -ECONNREFUSED);
+ break;
+ case IRDMA_CM_EVENT_ABORTED:
+ if (!event->cm_node->cm_id ||
+ event->cm_node->state == IRDMA_CM_STATE_OFFLOADED)
+ break;
+ irdma_event_connect_error(event);
+ break;
+ default:
+ ibdev_dbg(to_ibdev(cm_node->iwdev),
+ "CM: bad event type = %d\n", event->type);
+ break;
+ }
+
+ event->cm_info.cm_id->rem_ref(event->cm_info.cm_id);
+ irdma_rem_ref_cm_node(event->cm_node);
+ kfree(event);
+}
+
+/**
+ * irdma_cm_post_event - queue event request for worker thread
+ * @event: cm node's info for up event call
+ */
+static void irdma_cm_post_event(struct irdma_cm_event *event)
+{
+ atomic_inc(&event->cm_node->ref_count);
+ event->cm_info.cm_id->add_ref(event->cm_info.cm_id);
+ INIT_WORK(&event->event_work, irdma_cm_event_handler);
+ queue_work(event->cm_node->cm_core->event_wq, &event->event_work);
+}
+
+/**
+ * irdma_qhash_ctrl - enable/disable qhash for list
+ * @iwdev: device pointer
+ * @parent_listen_node: parent listen node
+ * @nfo: cm info node
+ * @ipaddr: Pointer to IPv4 or IPv6 address
+ * @ipv4: flag indicating IPv4 when true
+ * @ifup: flag indicating interface up when true
+ *
+ * Enables or disables the qhash for the node in the child
+ * listen list that matches ipaddr. If no matching IP was found
+ * it will allocate and add a new child listen node to the
+ * parent listen node. The listen_list_lock is assumed to be
+ * held when called.
+ */
+static void irdma_qhash_ctrl(struct irdma_device *iwdev,
+ struct irdma_cm_listener *parent_listen_node,
+ struct irdma_cm_info *nfo, u32 *ipaddr, bool ipv4,
+ bool ifup)
+{
+ struct list_head *child_listen_list = &parent_listen_node->child_listen_list;
+ struct irdma_cm_listener *child_listen_node;
+ struct list_head *pos, *tpos;
+ enum irdma_status_code err;
+ bool node_allocated = false;
+ enum irdma_quad_hash_manage_type op = ifup ?
+ IRDMA_QHASH_MANAGE_TYPE_ADD :
+ IRDMA_QHASH_MANAGE_TYPE_DELETE;
+
+ list_for_each_safe(pos, tpos, child_listen_list) {
+ child_listen_node = list_entry(pos, struct irdma_cm_listener,
+ child_listen_list);
+ if (!memcmp(child_listen_node->loc_addr, ipaddr, ipv4 ? 4 : 16))
+ goto set_qhash;
+ }
+
+ /* if not found then add a child listener if interface is going up */
+ if (!ifup)
+ return;
+ child_listen_node = kzalloc(sizeof(*child_listen_node), GFP_ATOMIC);
+ if (!child_listen_node)
+ return;
+
+ node_allocated = true;
+ memcpy(child_listen_node, parent_listen_node,
+ sizeof(*child_listen_node));
+
+ memcpy(child_listen_node->loc_addr, ipaddr, ipv4 ? 4 : 16);
+
+set_qhash:
+ memcpy(nfo->loc_addr, child_listen_node->loc_addr,
+ sizeof(nfo->loc_addr));
+ nfo->vlan_id = child_listen_node->vlan_id;
+ err = irdma_manage_qhash(iwdev, nfo, IRDMA_QHASH_TYPE_TCP_SYN, op, NULL,
+ false);
+ if (!err) {
+ child_listen_node->qhash_set = ifup;
+ if (node_allocated)
+ list_add(&child_listen_node->child_listen_list,
+ &parent_listen_node->child_listen_list);
+ } else if (node_allocated) {
+ kfree(child_listen_node);
+ }
+}
+
+/**
+ * irdma_cm_teardown_connections - teardown QPs
+ * @iwdev: device pointer
+ * @ipaddr: Pointer to IPv4 or IPv6 address
+ * @nfo: Connection info
+ * @disconnect_all: flag indicating disconnect all QPs
+ *
+ * teardown QPs where source or destination addr matches ip addr
+ */
+void irdma_cm_teardown_connections(struct irdma_device *iwdev, u32 *ipaddr,
+ struct irdma_cm_info *nfo,
+ bool disconnect_all)
+{
+ struct irdma_cm_core *cm_core = &iwdev->cm_core;
+ struct list_head *list_core_temp;
+ struct list_head *list_node;
+ struct irdma_cm_node *cm_node;
+ struct list_head teardown_list;
+ struct ib_qp_attr attr;
+ struct irdma_sc_vsi *vsi = &iwdev->vsi;
+ struct irdma_sc_qp *sc_qp;
+ struct irdma_qp *qp;
+ int i;
+ unsigned long flags;
+
+ INIT_LIST_HEAD(&teardown_list);
+
+ spin_lock_irqsave(&cm_core->ht_lock, flags);
+ list_for_each_safe (list_node, list_core_temp,
+ &cm_core->accelerated_list) {
+ cm_node = container_of(list_node, struct irdma_cm_node, list);
+ if (disconnect_all ||
+ (nfo->vlan_id == cm_node->vlan_id &&
+ !memcmp(cm_node->loc_addr, ipaddr, nfo->ipv4 ? 4 : 16))) {
+ atomic_inc(&cm_node->ref_count);
+ list_add(&cm_node->teardown_entry, &teardown_list);
+ }
+ }
+ list_for_each_safe(list_node, list_core_temp,
+ &cm_core->non_accelerated_list) {
+ cm_node = container_of(list_node, struct irdma_cm_node, list);
+ if (disconnect_all ||
+ (nfo->vlan_id == cm_node->vlan_id &&
+ !memcmp(cm_node->loc_addr, ipaddr, nfo->ipv4 ? 4 : 16))) {
+ atomic_inc(&cm_node->ref_count);
+ list_add(&cm_node->teardown_entry, &teardown_list);
+ }
+ }
+ spin_unlock_irqrestore(&cm_core->ht_lock, flags);
+
+ list_for_each_safe(list_node, list_core_temp, &teardown_list) {
+ cm_node = container_of(list_node, struct irdma_cm_node,
+ teardown_entry);
+ attr.qp_state = IB_QPS_ERR;
+ irdma_modify_qp(&cm_node->iwqp->ibqp, &attr, IB_QP_STATE, NULL);
+ if (iwdev->reset)
+ irdma_cm_disconn(cm_node->iwqp);
+ irdma_rem_ref_cm_node(cm_node);
+ }
+ if (!iwdev->roce_mode)
+ return;
+
+ INIT_LIST_HEAD(&teardown_list);
+ for (i = 0; i < IRDMA_MAX_USER_PRIORITY; i++) {
+ spin_lock_irqsave(&vsi->qos[i].lock, flags);
+ list_for_each_safe(list_node, list_core_temp,
+ &vsi->qos[i].qplist) {
+ u32 qp_ip[4];
+
+ sc_qp = container_of(list_node, struct irdma_sc_qp,
+ list);
+ if (sc_qp->qp_type != IRDMA_QP_TYPE_ROCE_RC)
+ continue;
+
+ qp = sc_qp->qp_uk.back_qp;
+ if (!disconnect_all) {
+ if (nfo->ipv4)
+ qp_ip[0] = qp->udp_info.local_ipaddr3;
+ else
+ memcpy(qp_ip,
+ &qp->udp_info.local_ipaddr0,
+ sizeof(qp_ip));
+ }
+
+ if (disconnect_all ||
+ (nfo->vlan_id == (qp->udp_info.vlan_tag & VLAN_VID_MASK) &&
+ !memcmp(qp_ip, ipaddr, nfo->ipv4 ? 4 : 16))) {
+ spin_lock(&iwdev->rf->qptable_lock);
+ if (iwdev->rf->qp_table[sc_qp->qp_uk.qp_id]) {
+ irdma_add_ref(&qp->ibqp);
+ list_add(&qp->teardown_entry,
+ &teardown_list);
+ }
+ spin_unlock(&iwdev->rf->qptable_lock);
+ }
+ }
+ spin_unlock_irqrestore(&vsi->qos[i].lock, flags);
+ }
+
+ list_for_each_safe(list_node, list_core_temp, &teardown_list) {
+ qp = container_of(list_node, struct irdma_qp, teardown_entry);
+ attr.qp_state = IB_QPS_ERR;
+ irdma_modify_qp_roce(&qp->ibqp, &attr, IB_QP_STATE, NULL);
+ irdma_rem_ref(&qp->ibqp);
+ }
+}
+
+/**
+ * irdma_ifdown_notify - process an ifdown on an interface
+ * @iwdev: device pointer
+ * @netdev: network device structure
+ * @ipaddr: Pointer to IPv4 or IPv6 address
+ * @ipv4: flag indicating IPv4 when true
+ * @ifup: flag indicating interface up when true
+ */
+void irdma_if_notify(struct irdma_device *iwdev, struct net_device *netdev,
+ u32 *ipaddr, bool ipv4, bool ifup)
+{
+ struct irdma_cm_core *cm_core = &iwdev->cm_core;
+ unsigned long flags;
+ struct irdma_cm_listener *listen_node;
+ static const u32 ip_zero[4] = { 0, 0, 0, 0 };
+ struct irdma_cm_info nfo;
+ u16 vlan_id = rdma_vlan_dev_vlan_id(netdev);
+ enum irdma_quad_hash_manage_type op = ifup ?
+ IRDMA_QHASH_MANAGE_TYPE_ADD :
+ IRDMA_QHASH_MANAGE_TYPE_DELETE;
+
+ nfo.vlan_id = vlan_id;
+ nfo.ipv4 = ipv4;
+
+ /* Disable or enable qhash for listeners */
+ spin_lock_irqsave(&cm_core->listen_list_lock, flags);
+ list_for_each_entry(listen_node, &cm_core->listen_nodes, list) {
+ if (vlan_id != listen_node->vlan_id ||
+ (memcmp(listen_node->loc_addr, ipaddr, ipv4 ? 4 : 16) &&
+ memcmp(listen_node->loc_addr, ip_zero, ipv4 ? 4 : 16)))
+ continue;
+
+ memcpy(nfo.loc_addr, listen_node->loc_addr,
+ sizeof(nfo.loc_addr));
+ nfo.loc_port = listen_node->loc_port;
+ nfo.user_pri = listen_node->user_pri;
+ if (!list_empty(&listen_node->child_listen_list)) {
+ irdma_qhash_ctrl(iwdev, listen_node, &nfo, ipaddr, ipv4,
+ ifup);
+ } else if (memcmp(listen_node->loc_addr, ip_zero,
+ ipv4 ? 4 : 16)) {
+ if (!irdma_manage_qhash(iwdev, &nfo,
+ IRDMA_QHASH_TYPE_TCP_SYN, op,
+ NULL, false))
+ listen_node->qhash_set = ifup;
+ }
+ }
+ spin_unlock_irqrestore(&cm_core->listen_list_lock, flags);
+
+ /* disconnect any connected qp's on ifdown */
+ if (!ifup)
+ irdma_cm_teardown_connections(iwdev, ipaddr, &nfo, false);
+}
diff --git a/drivers/infiniband/hw/irdma/cm.h b/drivers/infiniband/hw/irdma/cm.h
new file mode 100644
index 000000000000..2e11f68fa60e
--- /dev/null
+++ b/drivers/infiniband/hw/irdma/cm.h
@@ -0,0 +1,415 @@
+/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
+/* Copyright (c) 2019, Intel Corporation. */
+
+#ifndef IRDMA_CM_H
+#define IRDMA_CM_H
+
+#define IRDMA_MANAGE_APBVT_DEL 0
+#define IRDMA_MANAGE_APBVT_ADD 1
+
+#define IRDMA_MPA_REQUEST_ACCEPT 1
+#define IRDMA_MPA_REQUEST_REJECT 2
+
+/* IETF MPA -- defines */
+#define IEFT_MPA_KEY_REQ "MPA ID Req Frame"
+#define IEFT_MPA_KEY_REP "MPA ID Rep Frame"
+#define IETF_MPA_KEY_SIZE 16
+#define IETF_MPA_VER 1
+#define IETF_MAX_PRIV_DATA_LEN 512
+#define IETF_MPA_FRAME_SIZE 20
+#define IETF_RTR_MSG_SIZE 4
+#define IETF_MPA_V2_FLAG 0x10
+#define SNDMARKER_SEQNMASK 0x000001ff
+#define IRDMA_MAX_IETF_SIZE 32
+
+/* IETF RTR MSG Fields */
+#define IETF_PEER_TO_PEER 0x8000
+#define IETF_FLPDU_ZERO_LEN 0x4000
+#define IETF_RDMA0_WRITE 0x8000
+#define IETF_RDMA0_READ 0x4000
+#define IETF_NO_IRD_ORD 0x3fff
+
+/* HW-supported IRD sizes*/
+#define IRDMA_HW_IRD_SETTING_2 2
+#define IRDMA_HW_IRD_SETTING_4 4
+#define IRDMA_HW_IRD_SETTING_8 8
+#define IRDMA_HW_IRD_SETTING_16 16
+#define IRDMA_HW_IRD_SETTING_32 32
+#define IRDMA_HW_IRD_SETTING_64 64
+#define IRDMA_HW_IRD_SETTING_128 128
+
+#define MAX_PORTS 65536
+
+#define IRDMA_PASSIVE_STATE_INDICATED 0
+#define IRDMA_DO_NOT_SEND_RESET_EVENT 1
+#define IRDMA_SEND_RESET_EVENT 2
+
+#define MAX_IRDMA_IFS 4
+
+#define SET_ACK 1
+#define SET_SYN 2
+#define SET_FIN 4
+#define SET_RST 8
+
+#define TCP_OPTIONS_PADDING 3
+
+#define IRDMA_DEFAULT_RETRYS 64
+#define IRDMA_DEFAULT_RETRANS 8
+#define IRDMA_DEFAULT_TTL 0x40
+#define IRDMA_DEFAULT_RTT_VAR 6
+#define IRDMA_DEFAULT_SS_THRESH 0x3fffffff
+#define IRDMA_DEFAULT_REXMIT_THRESH 8
+
+#define IRDMA_RETRY_TIMEOUT HZ
+#define IRDMA_SHORT_TIME 10
+#define IRDMA_LONG_TIME (2 * HZ)
+#define IRDMA_MAX_TIMEOUT ((unsigned long)(12 * HZ))
+
+#define IRDMA_CM_HASHTABLE_SIZE 1024
+#define IRDMA_CM_TCP_TIMER_INTERVAL 3000
+#define IRDMA_CM_DEFAULT_MTU 1540
+#define IRDMA_CM_DEFAULT_FRAME_CNT 10
+#define IRDMA_CM_THREAD_STACK_SIZE 256
+#define IRDMA_CM_DEFAULT_RCV_WND 64240
+#define IRDMA_CM_DEFAULT_RCV_WND_SCALED 0x3fffc
+#define IRDMA_CM_DEFAULT_RCV_WND_SCALE 2
+#define IRDMA_CM_DEFAULT_FREE_PKTS 10
+#define IRDMA_CM_FREE_PKT_LO_WATERMARK 2
+#define IRDMA_CM_DEFAULT_MSS 536
+#define IRDMA_CM_DEFAULT_MPA_VER 2
+#define IRDMA_CM_DEFAULT_SEQ 0x159bf75f
+#define IRDMA_CM_DEFAULT_LOCAL_ID 0x3b47
+#define IRDMA_CM_DEFAULT_SEQ2 0x18ed5740
+#define IRDMA_CM_DEFAULT_LOCAL_ID2 0xb807
+#define IRDMA_MAX_CM_BUF (IRDMA_MAX_IETF_SIZE + IETF_MAX_PRIV_DATA_LEN)
+
+enum ietf_mpa_flags {
+ IETF_MPA_FLAGS_REJECT = 0x20,
+ IETF_MPA_FLAGS_CRC = 0x40,
+ IETF_MPA_FLAGS_MARKERS = 0x80,
+};
+
+enum irdma_timer_type {
+ IRDMA_TIMER_TYPE_SEND,
+ IRDMA_TIMER_TYPE_CLOSE,
+};
+
+enum option_nums {
+ OPTION_NUM_EOL,
+ OPTION_NUM_NONE,
+ OPTION_NUM_MSS,
+ OPTION_NUM_WINDOW_SCALE,
+ OPTION_NUM_SACK_PERM,
+ OPTION_NUM_SACK,
+ OPTION_NUM_WRITE0 = 0xbc,
+};
+
+/* cm node transition states */
+enum irdma_cm_node_state {
+ IRDMA_CM_STATE_UNKNOWN,
+ IRDMA_CM_STATE_INITED,
+ IRDMA_CM_STATE_LISTENING,
+ IRDMA_CM_STATE_SYN_RCVD,
+ IRDMA_CM_STATE_SYN_SENT,
+ IRDMA_CM_STATE_ONE_SIDE_ESTABLISHED,
+ IRDMA_CM_STATE_ESTABLISHED,
+ IRDMA_CM_STATE_ACCEPTING,
+ IRDMA_CM_STATE_MPAREQ_SENT,
+ IRDMA_CM_STATE_MPAREQ_RCVD,
+ IRDMA_CM_STATE_MPAREJ_RCVD,
+ IRDMA_CM_STATE_OFFLOADED,
+ IRDMA_CM_STATE_FIN_WAIT1,
+ IRDMA_CM_STATE_FIN_WAIT2,
+ IRDMA_CM_STATE_CLOSE_WAIT,
+ IRDMA_CM_STATE_TIME_WAIT,
+ IRDMA_CM_STATE_LAST_ACK,
+ IRDMA_CM_STATE_CLOSING,
+ IRDMA_CM_STATE_LISTENER_DESTROYED,
+ IRDMA_CM_STATE_CLOSED,
+};
+
+enum mpa_frame_ver {
+ IETF_MPA_V1 = 1,
+ IETF_MPA_V2 = 2,
+};
+
+enum mpa_frame_key {
+ MPA_KEY_REQUEST,
+ MPA_KEY_REPLY,
+};
+
+enum send_rdma0 {
+ SEND_RDMA_READ_ZERO = 1,
+ SEND_RDMA_WRITE_ZERO = 2,
+};
+
+enum irdma_tcpip_pkt_type {
+ IRDMA_PKT_TYPE_UNKNOWN,
+ IRDMA_PKT_TYPE_SYN,
+ IRDMA_PKT_TYPE_SYNACK,
+ IRDMA_PKT_TYPE_ACK,
+ IRDMA_PKT_TYPE_FIN,
+ IRDMA_PKT_TYPE_RST,
+};
+
+enum irdma_cm_listener_state {
+ IRDMA_CM_LISTENER_PASSIVE_STATE = 1,
+ IRDMA_CM_LISTENER_ACTIVE_STATE = 2,
+ IRDMA_CM_LISTENER_EITHER_STATE = 3,
+};
+
+/* CM event codes */
+enum irdma_cm_event_type {
+ IRDMA_CM_EVENT_UNKNOWN,
+ IRDMA_CM_EVENT_ESTABLISHED,
+ IRDMA_CM_EVENT_MPA_REQ,
+ IRDMA_CM_EVENT_MPA_CONNECT,
+ IRDMA_CM_EVENT_MPA_ACCEPT,
+ IRDMA_CM_EVENT_MPA_REJECT,
+ IRDMA_CM_EVENT_MPA_ESTABLISHED,
+ IRDMA_CM_EVENT_CONNECTED,
+ IRDMA_CM_EVENT_RESET,
+ IRDMA_CM_EVENT_ABORTED,
+};
+
+struct irdma_bth { /* Base Trasnport Header */
+ u8 opcode;
+ u8 flags;
+ __be16 pkey;
+ __be32 qpn;
+ __be32 apsn;
+};
+
+struct ietf_mpa_v1 {
+ u8 key[IETF_MPA_KEY_SIZE];
+ u8 flags;
+ u8 rev;
+ __be16 priv_data_len;
+ u8 priv_data[0];
+};
+
+struct ietf_rtr_msg {
+ __be16 ctrl_ird;
+ __be16 ctrl_ord;
+};
+
+struct ietf_mpa_v2 {
+ u8 key[IETF_MPA_KEY_SIZE];
+ u8 flags;
+ u8 rev;
+ __be16 priv_data_len;
+ struct ietf_rtr_msg rtr_msg;
+ u8 priv_data[0];
+};
+
+struct option_base {
+ u8 optionnum;
+ u8 len;
+};
+
+struct option_mss {
+ u8 optionnum;
+ u8 len;
+ __be16 mss;
+};
+
+struct option_windowscale {
+ u8 optionnum;
+ u8 len;
+ u8 shiftcount;
+};
+
+union all_known_options {
+ char eol;
+ struct option_base base;
+ struct option_mss mss;
+ struct option_windowscale windowscale;
+};
+
+struct irdma_timer_entry {
+ struct list_head list;
+ unsigned long timetosend; /* jiffies */
+ struct irdma_puda_buf *sqbuf;
+ u32 type;
+ u32 retrycount;
+ u32 retranscount;
+ u32 context;
+ u32 send_retrans;
+ int close_when_complete;
+};
+
+/* CM context params */
+struct irdma_cm_tcp_context {
+ u8 client;
+ u32 loc_seq_num;
+ u32 loc_ack_num;
+ u32 rem_ack_num;
+ u32 rcv_nxt;
+ u32 loc_id;
+ u32 rem_id;
+ u32 snd_wnd;
+ u32 max_snd_wnd;
+ u32 rcv_wnd;
+ u32 mss;
+ u8 snd_wscale;
+ u8 rcv_wscale;
+};
+
+struct irdma_cm_listener {
+ struct list_head list;
+ struct iw_cm_id *cm_id;
+ struct irdma_cm_core *cm_core;
+ struct irdma_device *iwdev;
+ struct list_head child_listen_list;
+ enum irdma_cm_listener_state listener_state;
+ atomic_t ref_count;
+ atomic_t pend_accepts_cnt;
+ u32 loc_addr[4];
+ u32 reused_node;
+ int backlog;
+ u16 loc_port;
+ u16 vlan_id;
+ u8 loc_mac[ETH_ALEN];
+ u8 user_pri;
+ u8 tos;
+ bool qhash_set;
+ bool ipv4;
+};
+
+struct irdma_kmem_info {
+ void *addr;
+ u32 size;
+};
+
+struct irdma_cm_node {
+ struct irdma_qp *iwqp;
+ struct irdma_device *iwdev;
+ struct irdma_sc_dev *dev;
+ struct irdma_cm_tcp_context tcp_cntxt;
+ struct irdma_cm_core *cm_core;
+ struct irdma_timer_entry *send_entry;
+ struct irdma_timer_entry *close_entry;
+ struct irdma_cm_listener *listener;
+ struct list_head timer_entry;
+ struct list_head reset_entry;
+ struct list_head teardown_entry;
+ struct irdma_kmem_info pdata;
+ struct irdma_sc_ah *ah;
+ union {
+ struct ietf_mpa_v1 mpa_frame;
+ struct ietf_mpa_v2 mpa_v2_frame;
+ };
+ struct irdma_kmem_info mpa_hdr;
+ struct iw_cm_id *cm_id;
+ struct list_head list;
+ spinlock_t retrans_list_lock; /* protect CM node rexmit updates*/
+ atomic_t passive_state;
+ atomic_t ref_count;
+ enum irdma_cm_node_state state;
+ enum send_rdma0 send_rdma0_op;
+ enum mpa_frame_ver mpa_frame_rev;
+ u32 loc_addr[4], rem_addr[4];
+ u16 loc_port, rem_port;
+ bool accelerated;
+ int apbvt_set;
+ int accept_pend;
+ u16 vlan_id;
+ u16 ird_size;
+ u16 ord_size;
+ u16 mpav2_ird_ord;
+ u16 lsmm_size;
+ u8 pdata_buf[IETF_MAX_PRIV_DATA_LEN];
+ u8 loc_mac[ETH_ALEN];
+ u8 rem_mac[ETH_ALEN];
+ u8 user_pri;
+ u8 tos;
+ bool ack_rcvd;
+ bool qhash_set;
+ bool ipv4;
+ bool snd_mark_en;
+ bool do_lpb;
+};
+
+/* Used by internal CM APIs to pass CM information*/
+struct irdma_cm_info {
+ struct iw_cm_id *cm_id;
+ u16 loc_port;
+ u16 rem_port;
+ u32 loc_addr[4];
+ u32 rem_addr[4];
+ u32 qh_qpid;
+ u16 vlan_id;
+ int backlog;
+ u8 user_pri;
+ u8 tos;
+ bool ipv4;
+};
+
+struct irdma_cm_event {
+ enum irdma_cm_event_type type;
+ struct irdma_cm_info cm_info;
+ struct work_struct event_work;
+ struct irdma_cm_node *cm_node;
+};
+
+struct irdma_cm_core {
+ struct irdma_device *iwdev;
+ struct irdma_sc_dev *dev;
+ struct list_head listen_nodes;
+ struct list_head accelerated_list;
+ struct list_head non_accelerated_list;
+ struct timer_list tcp_timer;
+ struct workqueue_struct *event_wq;
+ struct workqueue_struct *disconn_wq;
+ spinlock_t ht_lock; /* protect CM node (active side) list */
+ spinlock_t listen_list_lock; /* protect listener list */
+ spinlock_t apbvt_lock; /*serialize apbvt add/del entries*/
+ unsigned long ports_in_use[BITS_TO_LONGS(MAX_PORTS)];
+ u64 stats_nodes_created;
+ u64 stats_nodes_destroyed;
+ u64 stats_listen_created;
+ u64 stats_listen_destroyed;
+ u64 stats_listen_nodes_created;
+ u64 stats_listen_nodes_destroyed;
+ u64 stats_lpbs;
+ u64 stats_accepts;
+ u64 stats_rejects;
+ u64 stats_connect_errs;
+ u64 stats_passive_errs;
+ u64 stats_pkt_retrans;
+ u64 stats_backlog_drops;
+ struct irdma_puda_buf *(*form_cm_frame)(struct irdma_cm_node *cm_node,
+ struct irdma_kmem_info *options,
+ struct irdma_kmem_info *hdr,
+ struct irdma_kmem_info *pdata,
+ u8 flags);
+ int (*cm_create_ah)(struct irdma_cm_node *cm_node, bool wait);
+ void (*cm_free_ah)(struct irdma_cm_node *cm_node);
+};
+
+int irdma_schedule_cm_timer(struct irdma_cm_node *cm_node,
+ struct irdma_puda_buf *sqbuf,
+ enum irdma_timer_type type, int send_retrans,
+ int close_when_complete);
+int irdma_accept(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param);
+int irdma_reject(struct iw_cm_id *cm_id, const void *pdata, u8 pdata_len);
+int irdma_connect(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param);
+int irdma_create_listen(struct iw_cm_id *cm_id, int backlog);
+int irdma_destroy_listen(struct iw_cm_id *cm_id);
+void irdma_cm_teardown_connections(struct irdma_device *iwdev, u32 *ipaddr,
+ struct irdma_cm_info *nfo,
+ bool disconnect_all);
+int irdma_cm_start(struct irdma_device *dev);
+int irdma_cm_stop(struct irdma_device *dev);
+bool irdma_ipv4_is_lpb(u32 loc_addr, u32 rem_addr);
+bool irdma_ipv6_is_lpb(u32 *loc_addr, u32 *rem_addr);
+int irdma_arp_table(struct irdma_pci_f *rf, u32 *ip_addr, bool ipv4,
+ u8 *mac_addr, u32 action);
+int irdma_add_arp(struct irdma_pci_f *rf, u32 *ip, bool ipv4, u8 *mac);
+void irdma_if_notify(struct irdma_device *iwdev, struct net_device *netdev,
+ u32 *ipaddr, bool ipv4, bool ifup);
+bool irdma_port_in_use(struct irdma_cm_core *cm_core, u16 port);
+void irdma_send_ack(struct irdma_cm_node *cm_node);
+void irdma_lpb_nop(struct irdma_sc_qp *qp);
+u8 irdma_derive_hw_ird_setting(u16 cm_ird);
+#endif /* IRDMA_CM_H */
--
2.21.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox