* Re: [PATCH] qca_debug: Reduce function calls for sequence output in qcaspi_info_show()
From: Stefan Wahren @ 2017-05-09 7:07 UTC (permalink / raw)
To: SF Markus Elfring, netdev, David S. Miller, Philippe Reynes
Cc: LKML, kernel-janitors
In-Reply-To: <9932ee63-f766-49ce-d760-9a0bd06f0657@users.sourceforge.net>
Am 08.05.2017 um 19:29 schrieb SF Markus Elfring:
> From: Markus Elfring <elfring@users.sourceforge.net>
> Date: Mon, 8 May 2017 19:21:27 +0200
>
> A bit of data was put into a sequence by separate function calls.
> Print the same data together with adjusted seq_printf() calls instead.
Sorry, i'm not happy with this patch. It doesn't increase readabilityand
mixes the output of multiple lines.
The only benefit is a little bit higher performance. But for debugfs
this won't be necessary.
Stefan
>
> This issue was detected by using the Coccinelle software.
>
> Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
> ---
> drivers/net/ethernet/qualcomm/qca_debug.c | 10 ++--------
> 1 file changed, 2 insertions(+), 8 deletions(-)
>
^ permalink raw reply
* [PATCH 0/3] ath9k: Fine-tuning for some function implementations
From: SF Markus Elfring @ 2017-05-09 7:25 UTC (permalink / raw)
To: ath9k-devel, linux-wireless, netdev, Kalle Valo; +Cc: LKML, kernel-janitors
From: Markus Elfring <elfring@users.sourceforge.net>
Date: Tue, 9 May 2017 09:19:09 +0200
Three update suggestions were taken into account
from static source code analysis.
Markus Elfring (3):
Reduce function calls for sequence output in read_file_dma()
Replace four seq_printf() calls by seq_putc()
Adjust a null pointer check in three functions
drivers/net/wireless/ath/ath9k/debug.c | 25 ++++++++++---------------
1 file changed, 10 insertions(+), 15 deletions(-)
--
2.12.2
^ permalink raw reply
* [PATCH 1/3] ath9k: Reduce function calls for sequence output in read_file_dma()
From: SF Markus Elfring @ 2017-05-09 7:26 UTC (permalink / raw)
To: ath9k-devel, linux-wireless, netdev, Kalle Valo; +Cc: LKML, kernel-janitors
In-Reply-To: <03575993-754d-924c-9736-2a8d19e98fa5@users.sourceforge.net>
From: Markus Elfring <elfring@users.sourceforge.net>
Date: Mon, 8 May 2017 21:55:09 +0200
Some data were put into a sequence by separate function calls.
Print the same data with two function calls less.
This issue was detected by using the Coccinelle software.
Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
---
drivers/net/wireless/ath/ath9k/debug.c | 9 +++------
1 file changed, 3 insertions(+), 6 deletions(-)
diff --git a/drivers/net/wireless/ath/ath9k/debug.c b/drivers/net/wireless/ath/ath9k/debug.c
index 2e64977a8ab6..981b38a1e352 100644
--- a/drivers/net/wireless/ath/ath9k/debug.c
+++ b/drivers/net/wireless/ath/ath9k/debug.c
@@ -427,9 +427,7 @@ static int read_file_dma(struct seq_file *file, void *data)
seq_printf(file, "%d: %08x ", i, val[i]);
}
- seq_puts(file, "\n\n");
- seq_puts(file, "Num QCU: chain_st fsp_ok fsp_st DCU: chain_st\n");
-
+ seq_puts(file, "\n\nNum QCU: chain_st fsp_ok fsp_st DCU: chain_st\n");
for (i = 0; i < ATH9K_NUM_QUEUES; i++, qcuOffset += 4, dcuOffset += 5) {
if (i == 8) {
qcuOffset = 0;
@@ -448,9 +446,8 @@ static int read_file_dma(struct seq_file *file, void *data)
(*dcuBase & (0x1f << dcuOffset)) >> dcuOffset);
}
- seq_puts(file, "\n");
-
- seq_printf(file, "qcu_stitch state: %2x qcu_fetch state: %2x\n",
+ seq_printf(file,
+ "\nqcu_stitch state: %2x qcu_fetch state: %2x\n",
(val[3] & 0x003c0000) >> 18, (val[3] & 0x03c00000) >> 22);
seq_printf(file, "qcu_complete state: %2x dcu_complete state: %2x\n",
(val[3] & 0x1c000000) >> 26, (val[6] & 0x3));
--
2.12.2
^ permalink raw reply related
* [PATCH 2/3] ath9k: Replace four seq_printf() calls by seq_putc()
From: SF Markus Elfring @ 2017-05-09 7:28 UTC (permalink / raw)
To: ath9k-devel, linux-wireless, netdev, Kalle Valo; +Cc: LKML, kernel-janitors
In-Reply-To: <03575993-754d-924c-9736-2a8d19e98fa5@users.sourceforge.net>
From: Markus Elfring <elfring@users.sourceforge.net>
Date: Mon, 8 May 2017 22:04:47 +0200
Four single characters (line breaks) should be put into a sequence.
Thus use the corresponding function "seq_putc".
This issue was detected by using the Coccinelle software.
Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
---
drivers/net/wireless/ath/ath9k/debug.c | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/drivers/net/wireless/ath/ath9k/debug.c b/drivers/net/wireless/ath/ath9k/debug.c
index 981b38a1e352..0d215598193c 100644
--- a/drivers/net/wireless/ath/ath9k/debug.c
+++ b/drivers/net/wireless/ath/ath9k/debug.c
@@ -421,7 +421,7 @@ static int read_file_dma(struct seq_file *file, void *data)
for (i = 0; i < ATH9K_NUM_DMA_DEBUG_REGS; i++) {
if (i % 4 == 0)
- seq_puts(file, "\n");
+ seq_putc(file, '\n');
val[i] = REG_READ_D(ah, AR_DMADBG_0 + (i * sizeof(u32)));
seq_printf(file, "%d: %08x ", i, val[i]);
@@ -702,8 +702,7 @@ static int read_file_misc(struct seq_file *file, void *data)
if (rxfilter & ATH9K_RX_FILTER_CONTROL_WRAPPER)
seq_puts(file, " CONTROL_WRAPPER");
- seq_puts(file, "\n");
-
+ seq_putc(file, '\n');
reg = sc->sc_ah->imask;
seq_printf(file, "INTERRUPT-MASK: 0x%x", reg);
@@ -723,8 +722,7 @@ static int read_file_misc(struct seq_file *file, void *data)
if (reg & ATH9K_INT_BB_WATCHDOG)
seq_puts(file, " BB_WATCHDOG");
- seq_puts(file, "\n");
-
+ seq_putc(file, '\n');
i = 0;
ath_for_each_chanctx(sc, ctx) {
if (list_empty(&ctx->vifs))
@@ -981,7 +979,7 @@ static int read_file_dump_nfcal(struct seq_file *file, void *data)
seq_printf(file, " %d\t %d\t %d\t\t", i, h[i].privNF, nread);
for (j = 0; j < nread; j++)
seq_printf(file, " %d", h[i].nfCalBuffer[j]);
- seq_puts(file, "\n");
+ seq_putc(file, '\n');
}
return 0;
--
2.12.2
^ permalink raw reply related
* [PATCH 3/3] ath9k: Adjust a null pointer check in three functions
From: SF Markus Elfring @ 2017-05-09 7:29 UTC (permalink / raw)
To: ath9k-devel, linux-wireless, netdev, Kalle Valo; +Cc: LKML, kernel-janitors
In-Reply-To: <03575993-754d-924c-9736-2a8d19e98fa5@users.sourceforge.net>
From: Markus Elfring <elfring@users.sourceforge.net>
Date: Mon, 8 May 2017 22:17:13 +0200
The script "checkpatch.pl" pointed information out like the following.
Comparison to NULL could be written "!buf"
Thus adjust this expression.
Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
---
drivers/net/wireless/ath/ath9k/debug.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/net/wireless/ath/ath9k/debug.c b/drivers/net/wireless/ath/ath9k/debug.c
index 0d215598193c..5f2c1d8e8e70 100644
--- a/drivers/net/wireless/ath/ath9k/debug.c
+++ b/drivers/net/wireless/ath/ath9k/debug.c
@@ -161,7 +161,7 @@ static ssize_t read_file_ani(struct file *file, char __user *user_buf,
};
buf = kzalloc(size, GFP_KERNEL);
- if (buf == NULL)
+ if (!buf)
return -ENOMEM;
len += scnprintf(buf + len, size - len, "%15s: %s\n", "ANI",
@@ -315,7 +315,7 @@ static ssize_t read_file_antenna_diversity(struct file *file,
};
buf = kzalloc(size, GFP_KERNEL);
- if (buf == NULL)
+ if (!buf)
return -ENOMEM;
if (!(pCap->hw_caps & ATH9K_HW_CAP_ANT_DIV_COMB)) {
@@ -1008,7 +1008,7 @@ static ssize_t read_file_btcoex(struct file *file, char __user *user_buf,
size_t retval;
buf = kzalloc(size, GFP_KERNEL);
- if (buf == NULL)
+ if (!buf)
return -ENOMEM;
if (!sc->sc_ah->common.btcoex_enabled) {
--
2.12.2
^ permalink raw reply related
* DQL and TCQ_F_CAN_BYPASS destroy performance under virtualizaiton (Was: "Re: net_sched strange in 4.11")
From: Anton Ivanov @ 2017-05-09 7:46 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, Stefan Hajnoczi
In-Reply-To: <66c5c2d3-65df-8506-6934-5628e709e229@cambridgegreys.com>
I have figured it out. Two issues.
1) skb->xmit_more is hardly ever set under virtualization because the
qdisc is usually bypassed because of TCQ_F_CAN_BYPASS. Once
TCQ_F_CAN_BYPASS is set a virtual NIC driver is not likely see
skb->xmit_more (this answers my "how does this work at all" question).
2) If that flag is turned off (I patched sched_generic to turn it off in
pfifo_fast while testing), DQL keeps xmit_more from being set. If the
driver is not DQL enabled xmit_more is never ever set. If the driver is
DQL enabled the queue is adjusted to ensure xmit_more stops happening
within 10-15 xmit cycles.
That is plain *wrong* for virtual NICs - virtio, emulated NICs, etc.
There, the BIG cost is telling the hypervisor that it needs to "kick"
the packets. The cost of putting them into the vNIC buffers is
negligible. You want xmit_more to happen - it makes between 50% and 300%
(depending on vNIC design) difference. If there is no xmit_more the vNIC
will immediately "kick" the hypervisor and try to signal that the
packet needs to move straight away (as for example in virtio_net).
In addition to that, the perceived line rate is proportional to this
cost, so I am not sure that the current dql math holds. In fact, I think
it does not - it is trying to adjust something which influences the
perceived line rate.
So - how do we turn BOTH bypass and DQL adjustment while under
virtualization and set them to be "always qdisc" + "always xmit_more
allowed"
A.
P.S. Cc-ing virtio maintainer
A.
On 08/05/17 08:15, Anton Ivanov wrote:
> Hi all,
>
> I was revising some of my old work for UML to prepare it for
> submission and I noticed that skb->xmit_more does not seem to be set
> any more.
>
> I traced the issue as far as net/sched/sched_generic.c
>
> try_bulk_dequeue_skb() is never invoked (the drivers I am working on
> are dql enabled so that is not the problem).
>
> More interestingly, if I put a breakpoint and debug output into
> dequeue_skb() around line 147 - right before the bulk: tag that skb
> there is always NULL. ???
>
> Similarly, debug in pfifo_fast_dequeue shows only NULLs being
> dequeued. Again - ???
>
> First and foremost, I apologize for the silly question, but how can
> this work at all? I see the skbs showing up at the driver level, why
> are NULLs being returned at qdisc dequeue and where do the skbs at the
> driver level come from?
>
> Second, where should I look to fix it?
>
> A.
>
--
Anton R. Ivanov
Cambridge Greys Limited, England company No 10273661
http://www.cambridgegreys.com/
^ permalink raw reply
* [PATCH] wil6210: Replace five seq_puts() calls by seq_putc()
From: SF Markus Elfring @ 2017-05-09 7:50 UTC (permalink / raw)
To: wil6210, linux-wireless, netdev, Kalle Valo, Maya Erez
Cc: LKML, kernel-janitors
From: Markus Elfring <elfring@users.sourceforge.net>
Date: Mon, 8 May 2017 22:22:04 +0200
Five single characters (line breaks) should be put into a sequence.
Thus use the corresponding function "seq_putc".
This issue was detected by using the Coccinelle software.
Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
---
drivers/net/wireless/ath/wil6210/debugfs.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/drivers/net/wireless/ath/wil6210/debugfs.c b/drivers/net/wireless/ath/wil6210/debugfs.c
index 5648ebbd0e16..90118d286fb9 100644
--- a/drivers/net/wireless/ath/wil6210/debugfs.c
+++ b/drivers/net/wireless/ath/wil6210/debugfs.c
@@ -76,11 +76,11 @@ static void wil_print_vring(struct seq_file *s, struct wil6210_priv *wil,
volatile struct vring_tx_desc *d = &vring->va[i].tx;
if ((i % 128) == 0 && (i != 0))
- seq_puts(s, "\n");
+ seq_putc(s, '\n');
seq_printf(s, "%c", (d->dma.status & BIT(0)) ?
_s : (vring->ctx[i].skb ? _h : 'h'));
}
- seq_puts(s, "\n");
+ seq_putc(s, '\n');
}
seq_puts(s, "}\n");
}
@@ -233,7 +233,7 @@ static void wil_print_ring(struct seq_file *s, const char *prefix,
wil_seq_hexdump(s, databuf, len, " : ");
}
} else {
- seq_puts(s, "\n");
+ seq_putc(s, '\n');
}
}
out:
@@ -1366,7 +1366,7 @@ static void wil_print_rxtid_crypto(struct seq_file *s, int tid,
seq_printf(s, " [%i%s]%6phN", i, cc->key_set ? "+" : "-",
cc->pn);
}
- seq_puts(s, "\n");
+ seq_putc(s, '\n');
}
static int wil_sta_debugfs_show(struct seq_file *s, void *data)
@@ -1423,7 +1423,7 @@ __acquires(&p->tid_rx_lock) __releases(&p->tid_rx_lock)
mcs++)
seq_printf(s, " %lld",
p->stats.rx_per_mcs[mcs]);
- seq_puts(s, "\n");
+ seq_putc(s, '\n');
}
}
--
2.12.2
^ permalink raw reply related
* Re: [PATCH] wil6210: Replace five seq_puts() calls by seq_putc()
From: Johannes Berg @ 2017-05-09 7:53 UTC (permalink / raw)
To: SF Markus Elfring, wil6210, linux-wireless, netdev, Kalle Valo,
Maya Erez
Cc: LKML, kernel-janitors
In-Reply-To: <64747f85-e373-a0ff-b6dc-70cdfe35f71a@users.sourceforge.net>
On Tue, 2017-05-09 at 09:50 +0200, SF Markus Elfring wrote:
> From: Markus Elfring <elfring@users.sourceforge.net>
> Date: Mon, 8 May 2017 22:22:04 +0200
>
> Five single characters (line breaks) should be put into a sequence.
> Thus use the corresponding function "seq_putc".
Please stop, this isn't really an issue worth worrying about.
johannes
^ permalink raw reply
* Re: [PATCH v1] ACPI: Switch to use generic UUID API
From: Felipe Balbi @ 2017-05-09 7:56 UTC (permalink / raw)
To: Andy Shevchenko, linux-acpi, tpmdd-devel, intel-gfx, nouveau,
linux-input, iommu, linux-mmc, netdev, linux-pci, linux-usb,
alsa-devel, linux-kernel
Cc: Andy Shevchenko, Rafael J . Wysocki, Mika Westerberg,
Borislav Petkov, Dan Williams, Amir Goldstein, Jarkko Sakkinen,
Jani Nikula, Ben Skeggs, Benjamin Tissoires, Joerg Roedel,
Adrian Hunter, Yisen Zhuang, Bjorn Helgaas, Zhang Rui,
Mathias Nyman, Heikki Krogerus, Liam Girdwood, Mark
In-Reply-To: <20170504092151.88646-1-andriy.shevchenko@linux.intel.com>
Hi,
Andy Shevchenko <andriy.shevchenko@linux.intel.com> writes:
> acpi_evaluate_dsm() and friends take a pointer to a raw buffer of 16
> bytes. Instead we convert them to use uuid_le type. At the same time we
> convert current users.
>
> acpi_str_to_uuid() becomes useless after the conversion and it's safe to
> get rid of it.
>
> The conversion fixes a potential bug in int340x_thermal as well since
> we have to use memcmp() on binary data.
>
> Cc: Rafael J. Wysocki <rjw@rjwysocki.net>
> Cc: Mika Westerberg <mika.westerberg@linux.intel.com>
> Cc: Borislav Petkov <bp@suse.de>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Cc: Amir Goldstein <amir73il@gmail.com>
> Cc: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
> Cc: Jani Nikula <jani.nikula@linux.intel.com>
> Cc: Ben Skeggs <bskeggs@redhat.com>
> Cc: Benjamin Tissoires <benjamin.tissoires@redhat.com>
> Cc: Joerg Roedel <joro@8bytes.org>
> Cc: Adrian Hunter <adrian.hunter@intel.com>
> Cc: Yisen Zhuang <yisen.zhuang@huawei.com>
> Cc: Bjorn Helgaas <bhelgaas@google.com>
> Cc: Zhang Rui <rui.zhang@intel.com>
> Cc: Felipe Balbi <balbi@kernel.org>
> Cc: Mathias Nyman <mathias.nyman@intel.com>
> Cc: Heikki Krogerus <heikki.krogerus@linux.intel.com>
> Cc: Liam Girdwood <lgirdwood@gmail.com>
> Cc: Mark Brown <broonie@kernel.org>
> Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
> diff --git a/drivers/usb/dwc3/dwc3-pci.c b/drivers/usb/dwc3/dwc3-pci.c
> index a15ec71d0423..6b5284ec76df 100644
> --- a/drivers/usb/dwc3/dwc3-pci.c
> +++ b/drivers/usb/dwc3/dwc3-pci.c
> @@ -56,7 +56,7 @@ struct dwc3_pci {
> struct platform_device *dwc3;
> struct pci_dev *pci;
>
> - u8 uuid[16];
> + uuid_le uuid;
>
> unsigned int has_dsm_for_pm:1;
> };
> @@ -118,7 +118,7 @@ static int dwc3_pci_quirks(struct dwc3_pci *dwc)
>
> if (pdev->device == PCI_DEVICE_ID_INTEL_BXT ||
> pdev->device == PCI_DEVICE_ID_INTEL_BXT_M) {
> - acpi_str_to_uuid(PCI_INTEL_BXT_DSM_UUID, dwc->uuid);
> + uuid_le_to_bin(PCI_INTEL_BXT_DSM_UUID, &dwc->uuid);
> dwc->has_dsm_for_pm = true;
> }
>
> @@ -288,7 +288,7 @@ static int dwc3_pci_dsm(struct dwc3_pci *dwc, int param)
> tmp.type = ACPI_TYPE_INTEGER;
> tmp.integer.value = param;
>
> - obj = acpi_evaluate_dsm(ACPI_HANDLE(&dwc->pci->dev), dwc->uuid,
> + obj = acpi_evaluate_dsm(ACPI_HANDLE(&dwc->pci->dev), &dwc->uuid,
> 1, PCI_INTEL_BXT_FUNC_PMU_PWR, &argv4);
> if (!obj) {
> dev_err(&dwc->pci->dev, "failed to evaluate _DSM\n");
Acked-by: Felipe Balbi <felipe.balbi@linux.intel.com>
--
balbi
^ permalink raw reply
* Re: qca_debug: Reduce function calls for sequence output in qcaspi_info_show()
From: SF Markus Elfring @ 2017-05-09 7:57 UTC (permalink / raw)
To: Stefan Wahren, netdev
Cc: David S. Miller, Philippe Reynes, LKML, kernel-janitors
In-Reply-To: <290d2473-73bd-70db-b6ef-c50331812f31@i2se.com>
> The only benefit is a little bit higher performance.
I would prefer such a small code reduction. I am not so concerned
about an other readability impression at this place.
> But for debugfs this won't be necessary.
I would appreciate also another improvement there.
Regards,
Markus
^ permalink raw reply
* Re: [PATCH v1] ACPI: Switch to use generic UUID API
From: Mathias Nyman @ 2017-05-09 8:22 UTC (permalink / raw)
To: Andy Shevchenko, linux-acpi, tpmdd-devel, intel-gfx, nouveau,
linux-input, iommu, linux-mmc, netdev, linux-pci, linux-usb,
alsa-devel, linux-kernel
Cc: Felipe Balbi, Heikki Krogerus, Mathias Nyman, Yisen Zhuang,
Joerg Roedel, Amir Goldstein, Rafael J . Wysocki, Adrian Hunter,
Jarkko Sakkinen, Liam Girdwood, Benjamin Tissoires, Mark Brown,
Bjorn Helgaas, Dan Williams, Borislav Petkov, Mika Westerberg,
Zhang Rui, Ben Skeggs
In-Reply-To: <20170504092151.88646-1-andriy.shevchenko@linux.intel.com>
> diff --git a/drivers/usb/host/xhci-pci.c b/drivers/usb/host/xhci-pci.c
> index 7b86508ac8cf..93b4f0de9418 100644
> --- a/drivers/usb/host/xhci-pci.c
> +++ b/drivers/usb/host/xhci-pci.c
> @@ -210,13 +210,12 @@ static void xhci_pci_quirks(struct device *dev, struct xhci_hcd *xhci)
> #ifdef CONFIG_ACPI
> static void xhci_pme_acpi_rtd3_enable(struct pci_dev *dev)
> {
> - static const u8 intel_dsm_uuid[] = {
> - 0xb7, 0x0c, 0x34, 0xac, 0x01, 0xe9, 0xbf, 0x45,
> - 0xb7, 0xe6, 0x2b, 0x34, 0xec, 0x93, 0x1e, 0x23,
> - };
> + static const uuid_le intel_dsm_uuid =
> + UUID_LE(0xac340cb7, 0xe901, 0x45bf,
> + 0xb7, 0xe6, 0x2b, 0x34, 0xec, 0x93, 0x1e, 0x23);
> union acpi_object *obj;
>
> - obj = acpi_evaluate_dsm(ACPI_HANDLE(&dev->dev), intel_dsm_uuid, 3, 1,
> + obj = acpi_evaluate_dsm(ACPI_HANDLE(&dev->dev), &intel_dsm_uuid, 3, 1,
> NULL);
> ACPI_FREE(obj);
> }
For the xhci part above:
Acked-by: Mathias Nyman <mathias.nyman@linux.intel.com>
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply
* WARREN BUFFETT DONATION GESTURE
From: WARREN BUFFETT FOUNDATION @ 2017-05-09 8:13 UTC (permalink / raw)
To: Recipients
$1,500,000 was donated to you For more details email warrenbb@163.com
^ permalink raw reply
* [PATCH net] driver: vrf: Fix one possible use-after-free issue
From: gfree.wind @ 2017-05-09 8:54 UTC (permalink / raw)
To: dsa, shm, davem, netdev; +Cc: Gao Feng
From: Gao Feng <gfree.wind@vip.163.com>
The current codes only deal with the case that the skb is dropped, it
may meet one use-after-free issue when NF_HOOK returns 0 that means
the skb is stolen by one netfilter rule or hook.
When one netfilter rule or hook stoles the skb and return NF_STOLEN,
it means the skb is taken by the rule, and other modules should not
touch this skb ever. Maybe the skb is queued or freed directly by the
rule.
Now uses the nf_hook instead of NF_HOOK to get the result of netfilter,
and check the return value of nf_hook. Only when its value equals 1, it
means the skb could go ahead. Or reset the skb as NULL.
BTW, because vrf_rcv_finish is empty function, so needn't invoke it
even though nf_hook returns 1.
Signed-off-by: Gao Feng <gfree.wind@vip.163.com>
---
drivers/net/vrf.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/vrf.c b/drivers/net/vrf.c
index ceda586..8960f44 100644
--- a/drivers/net/vrf.c
+++ b/drivers/net/vrf.c
@@ -998,7 +998,7 @@ static struct sk_buff *vrf_rcv_nfhook(u8 pf, unsigned int hook,
{
struct net *net = dev_net(dev);
- if (NF_HOOK(pf, hook, net, NULL, skb, dev, NULL, vrf_rcv_finish) < 0)
+ if (nf_hook(pf, hook, net, NULL, skb, dev, NULL, vrf_rcv_finish) != 1)
skb = NULL; /* kfree_skb(skb) handled by nf code */
return skb;
--
1.9.1
^ permalink raw reply related
* WARREN BUFFETT DONATION GESTURE
From: WARREN BUFFETT FOUNDATION @ 2017-05-09 8:52 UTC (permalink / raw)
To: Recipients
$1,500,000 was donated to you For more details email warrenbb@163.com
^ permalink raw reply
* WARREN BUFFETT DONATION GESTURE
From: WARREN BUFFETT FOUNDATION @ 2017-05-09 8:53 UTC (permalink / raw)
To: Recipients
$1,500,000 was donated to you For more details email warrenbb@163.com
^ permalink raw reply
* Re: [PATCH net] driver: vrf: Fix one possible use-after-free issue
From: Florian Westphal @ 2017-05-09 9:21 UTC (permalink / raw)
To: gfree.wind; +Cc: dsa, shm, davem, netdev
In-Reply-To: <1494320069-39638-1-git-send-email-gfree.wind@vip.163.com>
gfree.wind@vip.163.com <gfree.wind@vip.163.com> wrote:
> When one netfilter rule or hook stoles the skb and return NF_STOLEN,
> it means the skb is taken by the rule, and other modules should not
> touch this skb ever. Maybe the skb is queued or freed directly by the
> rule.
>
> Now uses the nf_hook instead of NF_HOOK to get the result of netfilter,
> and check the return value of nf_hook. Only when its value equals 1, it
> means the skb could go ahead. Or reset the skb as NULL.
>
> BTW, because vrf_rcv_finish is empty function, so needn't invoke it
> even though nf_hook returns 1.
Thats a bug then.
The okfn (if called) takes ownership of skb and must free it eventually.
Otherwise userspace queueing leaks skb on reinjection.
(see nf_reinject() and its use of okfn()).
^ permalink raw reply
* [PATCH 1/4] net: macb: Add support for PTP timestamps in DMA descriptors
From: Rafal Ozieblo @ 2017-05-09 9:24 UTC (permalink / raw)
To: David Miller, nicolas.ferre-AIFe0yeh4nAAvxtiuMwx3w,
Richard Cochran, netdev-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
harini.katakam-gjFFaj9aHVfQT0dZR+AlfA,
andrei.pistirica-UWL1GkI3JZL3oGB3hsPCZA
Cc: Rafal Ozieblo
This patch adds support for PTP timestamps in
DMA buffer descriptors. It checks capability at runtime
and uses appropriate buffer descriptor.
Signed-off-by: Rafal Ozieblo <rafalo-vna1KIf7WgpBDgjK7y7TUQ@public.gmane.org>
---
drivers/net/ethernet/cadence/Kconfig | 10 ++-
drivers/net/ethernet/cadence/macb.c | 117 ++++++++++++++++++++++++++---------
drivers/net/ethernet/cadence/macb.h | 32 +++++++---
3 files changed, 122 insertions(+), 37 deletions(-)
diff --git a/drivers/net/ethernet/cadence/Kconfig b/drivers/net/ethernet/cadence/Kconfig
index 608bea1..427d65a 100644
--- a/drivers/net/ethernet/cadence/Kconfig
+++ b/drivers/net/ethernet/cadence/Kconfig
@@ -29,7 +29,15 @@ config MACB
support for the MACB/GEM chip.
To compile this driver as a module, choose M here: the module
- will be called macb.
+ will be macb.
+
+config MACB_USE_HWSTAMP
+ bool "Use IEEE 1588 hwstamp"
+ depends on MACB
+ default y
+ imply PTP_1588_CLOCK
+ ---help---
+ Enable IEEE 1588 Precision Time Protocol (PTP) support for MACB.
config MACB_PCI
tristate "Cadence PCI MACB/GEM support"
diff --git a/drivers/net/ethernet/cadence/macb.c b/drivers/net/ethernet/cadence/macb.c
index 91f7492..3151429 100644
--- a/drivers/net/ethernet/cadence/macb.c
+++ b/drivers/net/ethernet/cadence/macb.c
@@ -79,33 +79,84 @@
#define MACB_HALT_TIMEOUT 1230
/* DMA buffer descriptor might be different size
- * depends on hardware configuration.
+ * depends on hardware configuration:
+ *
+ * 1. dma address width 32 bits:
+ * word 1: 32 bit address of Data Buffer
+ * word 2: control
+ *
+ * 2. dma address width 64 bits:
+ * word 1: 32 bit address of Data Buffer
+ * word 2: control
+ * word 3: upper 32 bit address of Data Buffer
+ * word 4: unused
+ *
+ * 3. dma address width 32 bits with hardware timestamping:
+ * word 1: 32 bit address of Data Buffer
+ * word 2: control
+ * word 3: timestamp word 1
+ * word 4: timestamp word 2
+ *
+ * 4. dma address width 64 bits with hardware timestamping:
+ * word 1: 32 bit address of Data Buffer
+ * word 2: control
+ * word 3: upper 32 bit address of Data Buffer
+ * word 4: unused
+ * word 5: timestamp word 1
+ * word 6: timestamp word 2
*/
static unsigned int macb_dma_desc_get_size(struct macb *bp)
{
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (bp->hw_dma_cap == HW_DMA_CAP_64B)
- return sizeof(struct macb_dma_desc) + sizeof(struct macb_dma_desc_64);
+#ifdef MACB_EXT_DESC
+ unsigned int desc_size;
+
+ switch (bp->hw_dma_cap) {
+ case HW_DMA_CAP_64B:
+ desc_size = sizeof(struct macb_dma_desc)
+ + sizeof(struct macb_dma_desc_64);
+ break;
+ case HW_DMA_CAP_PTP:
+ desc_size = sizeof(struct macb_dma_desc)
+ + sizeof(struct macb_dma_desc_ptp);
+ break;
+ case HW_DMA_CAP_64B_PTP:
+ desc_size = sizeof(struct macb_dma_desc)
+ + sizeof(struct macb_dma_desc_64)
+ + sizeof(struct macb_dma_desc_ptp);
+ break;
+ default:
+ desc_size = sizeof(struct macb_dma_desc);
+ }
+ return desc_size;
#endif
return sizeof(struct macb_dma_desc);
}
-static unsigned int macb_adj_dma_desc_idx(struct macb *bp, unsigned int idx)
+static unsigned int macb_adj_dma_desc_idx(struct macb *bp, unsigned int desc_idx)
{
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- /* Dma buffer descriptor is 4 words length (instead of 2 words)
- * for 64b GEM.
- */
- if (bp->hw_dma_cap == HW_DMA_CAP_64B)
- idx <<= 1;
+#ifdef MACB_EXT_DESC
+ switch (bp->hw_dma_cap) {
+ case HW_DMA_CAP_64B:
+ case HW_DMA_CAP_PTP:
+ desc_idx <<= 1;
+ break;
+ case HW_DMA_CAP_64B_PTP:
+ desc_idx *= 3;
+ break;
+ default:
+ break;
+ }
+ return desc_idx;
#endif
- return idx;
+ return desc_idx;
}
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
static struct macb_dma_desc_64 *macb_64b_desc(struct macb *bp, struct macb_dma_desc *desc)
{
- return (struct macb_dma_desc_64 *)((void *)desc + sizeof(struct macb_dma_desc));
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B)
+ return (struct macb_dma_desc_64 *)((void *)desc + sizeof(struct macb_dma_desc));
+ return NULL;
}
#endif
@@ -602,7 +653,7 @@ static void macb_set_addr(struct macb *bp, struct macb_dma_desc *desc, dma_addr_
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
struct macb_dma_desc_64 *desc_64;
- if (bp->hw_dma_cap == HW_DMA_CAP_64B) {
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B) {
desc_64 = macb_64b_desc(bp, desc);
desc_64->addrh = upper_32_bits(addr);
}
@@ -616,7 +667,7 @@ static dma_addr_t macb_get_addr(struct macb *bp, struct macb_dma_desc *desc)
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
struct macb_dma_desc_64 *desc_64;
- if (bp->hw_dma_cap == HW_DMA_CAP_64B) {
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B) {
desc_64 = macb_64b_desc(bp, desc);
addr = ((u64)(desc_64->addrh) << 32);
}
@@ -715,7 +766,7 @@ static void macb_tx_error_task(struct work_struct *work)
/* Reinitialize the TX desc queue */
queue_writel(queue, TBQP, lower_32_bits(queue->tx_ring_dma));
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (bp->hw_dma_cap == HW_DMA_CAP_64B)
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B)
queue_writel(queue, TBQPH, upper_32_bits(queue->tx_ring_dma));
#endif
/* Make TX ring reflect state of hardware */
@@ -1923,9 +1974,13 @@ static void macb_configure_dma(struct macb *bp)
dmacfg &= ~GEM_BIT(TXCOEN);
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (bp->hw_dma_cap == HW_DMA_CAP_64B)
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B)
dmacfg |= GEM_BIT(ADDR64);
#endif
+#ifdef CONFIG_MACB_USE_HWSTAMP
+ if (bp->hw_dma_cap & HW_DMA_CAP_PTP)
+ dmacfg |= GEM_BIT(RXEXT) | GEM_BIT(TXEXT);
+#endif
netdev_dbg(bp->dev, "Cadence configure DMA with 0x%08x\n",
dmacfg);
gem_writel(bp, DMACFG, dmacfg);
@@ -1973,13 +2028,13 @@ static void macb_init_hw(struct macb *bp)
/* Initialize TX and RX buffers */
macb_writel(bp, RBQP, lower_32_bits(bp->rx_ring_dma));
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (bp->hw_dma_cap == HW_DMA_CAP_64B)
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B)
macb_writel(bp, RBQPH, upper_32_bits(bp->rx_ring_dma));
#endif
for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
queue_writel(queue, TBQP, lower_32_bits(queue->tx_ring_dma));
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (bp->hw_dma_cap == HW_DMA_CAP_64B)
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B)
queue_writel(queue, TBQPH, upper_32_bits(queue->tx_ring_dma));
#endif
@@ -2581,6 +2636,12 @@ static void macb_configure_caps(struct macb *bp,
dcfg = gem_readl(bp, DCFG2);
if ((dcfg & (GEM_BIT(RX_PKT_BUFF) | GEM_BIT(TX_PKT_BUFF))) == 0)
bp->caps |= MACB_CAPS_FIFO_MODE;
+ if (IS_ENABLED(CONFIG_MACB_USE_HWSTAMP) && gem_has_ptp(bp)) {
+ if (!GEM_BFEXT(TSU, gem_readl(bp, DCFG5)))
+ pr_err("GEM doesn't support hardware ptp.\n");
+ else
+ bp->hw_dma_cap |= HW_DMA_CAP_PTP;
+ }
}
dev_dbg(&bp->pdev->dev, "Cadence caps 0x%08x\n", bp->caps);
@@ -2718,7 +2779,7 @@ static int macb_init(struct platform_device *pdev)
queue->IMR = GEM_IMR(hw_q - 1);
queue->TBQP = GEM_TBQP(hw_q - 1);
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (bp->hw_dma_cap == HW_DMA_CAP_64B)
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B)
queue->TBQPH = GEM_TBQPH(hw_q - 1);
#endif
} else {
@@ -2729,7 +2790,7 @@ static int macb_init(struct platform_device *pdev)
queue->IMR = MACB_IMR;
queue->TBQP = MACB_TBQP;
#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (bp->hw_dma_cap == HW_DMA_CAP_64B)
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B)
queue->TBQPH = MACB_TBQPH;
#endif
}
@@ -3309,19 +3370,17 @@ static int macb_probe(struct platform_device *pdev)
bp->wol |= MACB_WOL_HAS_MAGIC_PACKET;
device_init_wakeup(&pdev->dev, bp->wol & MACB_WOL_HAS_MAGIC_PACKET);
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (GEM_BFEXT(DAW64, gem_readl(bp, DCFG6))) {
- dma_set_mask(&pdev->dev, DMA_BIT_MASK(44));
- bp->hw_dma_cap = HW_DMA_CAP_64B;
- } else
- bp->hw_dma_cap = HW_DMA_CAP_32B;
-#endif
-
spin_lock_init(&bp->lock);
/* setup capabilities */
macb_configure_caps(bp, macb_config);
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ if (GEM_BFEXT(DAW64, gem_readl(bp, DCFG6))) {
+ dma_set_mask(&pdev->dev, DMA_BIT_MASK(44));
+ bp->hw_dma_cap |= HW_DMA_CAP_64B;
+ }
+#endif
platform_set_drvdata(pdev, dev);
dev->irq = platform_get_irq(pdev, 0);
diff --git a/drivers/net/ethernet/cadence/macb.h b/drivers/net/ethernet/cadence/macb.h
index ec037b0..4359b08 100644
--- a/drivers/net/ethernet/cadence/macb.h
+++ b/drivers/net/ethernet/cadence/macb.h
@@ -12,6 +12,10 @@
#include <linux/phy.h>
+#if defined(CONFIG_ARCH_DMA_ADDR_T_64BIT) || defined(CONFIG_MACB_USE_HWSTAMP)
+#define MACB_EXT_DESC
+#endif
+
#define MACB_GREGS_NBR 16
#define MACB_GREGS_VERSION 2
#define MACB_MAX_QUEUES 8
@@ -269,6 +273,10 @@
#define GEM_RXBS_SIZE 8
#define GEM_DDRP_OFFSET 24 /* disc_when_no_ahb */
#define GEM_DDRP_SIZE 1
+#define GEM_RXEXT_OFFSET 28 /* RX extended Buffer Descriptor mode */
+#define GEM_RXEXT_SIZE 1
+#define GEM_TXEXT_OFFSET 29 /* TX extended Buffer Descriptor mode */
+#define GEM_TXEXT_SIZE 1
#define GEM_ADDR64_OFFSET 30 /* Address bus width - 64b or 32b */
#define GEM_ADDR64_SIZE 1
@@ -425,6 +433,11 @@
#define GEM_TX_PKT_BUFF_OFFSET 21
#define GEM_TX_PKT_BUFF_SIZE 1
+
+/* Bitfields in DCFG5. */
+#define GEM_TSU_OFFSET 8
+#define GEM_TSU_SIZE 1
+
/* Bitfields in DCFG6. */
#define GEM_PBUF_LSO_OFFSET 27
#define GEM_PBUF_LSO_SIZE 1
@@ -546,16 +559,21 @@ struct macb_dma_desc {
u32 ctrl;
};
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
-enum macb_hw_dma_cap {
- HW_DMA_CAP_32B,
- HW_DMA_CAP_64B,
-};
+#ifdef MACB_EXT_DESC
+#define HW_DMA_CAP_32B 0
+#define HW_DMA_CAP_64B (1 << 0)
+#define HW_DMA_CAP_PTP (1 << 1)
+#define HW_DMA_CAP_64B_PTP (HW_DMA_CAP_64B | HW_DMA_CAP_PTP)
struct macb_dma_desc_64 {
u32 addrh;
u32 resvd;
};
+
+struct macb_dma_desc_ptp {
+ u32 ts_1;
+ u32 ts_2;
+};
#endif
/* DMA descriptor bitfields */
@@ -954,8 +972,8 @@ struct macb {
u32 wol;
struct macb_ptp_info *ptp_info; /* macb-ptp interface */
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- enum macb_hw_dma_cap hw_dma_cap;
+#ifdef MACB_EXT_DESC
+ uint8_t hw_dma_cap;
#endif
};
--
2.4.5
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH 2/4] net: macb: Add tsu_clk to device tree
From: Rafal Ozieblo @ 2017-05-09 9:26 UTC (permalink / raw)
To: David Miller, nicolas.ferre, Richard Cochran, netdev,
linux-kernel, devicetree, linux-arm-kernel, harini.katakam,
andrei.pistirica
Cc: Rafal Ozieblo
In-Reply-To: <1494321885-14384-1-git-send-email-rafalo@cadence.com>
Signed-off-by: Rafal Ozieblo <rafalo@cadence.com>
---
Documentation/devicetree/bindings/net/macb.txt | 1 +
1 file changed, 1 insertion(+)
diff --git a/Documentation/devicetree/bindings/net/macb.txt b/Documentation/devicetree/bindings/net/macb.txt
index 1506e94..27966ae 100644
--- a/Documentation/devicetree/bindings/net/macb.txt
+++ b/Documentation/devicetree/bindings/net/macb.txt
@@ -22,6 +22,7 @@ Required properties:
Required elements: 'pclk', 'hclk'
Optional elements: 'tx_clk'
Optional elements: 'rx_clk' applies to cdns,zynqmp-gem
+ Optional elements: 'tsu_clk'
- clocks: Phandles to input clocks.
Optional properties for PHY child node:
--
2.4.5
^ permalink raw reply related
* [PATCH 3/4] net: macb: macb.c changed to macb_main.c
From: Rafal Ozieblo @ 2017-05-09 9:27 UTC (permalink / raw)
To: David Miller, nicolas.ferre-AIFe0yeh4nAAvxtiuMwx3w,
Richard Cochran, netdev-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA,
linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
harini.katakam-gjFFaj9aHVfQT0dZR+AlfA,
andrei.pistirica-UWL1GkI3JZL3oGB3hsPCZA
Cc: Rafal Ozieblo
In-Reply-To: <1494321885-14384-1-git-send-email-rafalo-vna1KIf7WgpBDgjK7y7TUQ@public.gmane.org>
In case that macb is compiled as a module, macb.c has been renamed to
macb_main.c to avoid naming confusion in Makefile.
Signed-off-by: Rafal Ozieblo <rafalo-vna1KIf7WgpBDgjK7y7TUQ@public.gmane.org>
---
drivers/net/ethernet/cadence/Makefile | 1 +
drivers/net/ethernet/cadence/macb.c | 3568 ------------------------------
drivers/net/ethernet/cadence/macb_main.c | 3568 ++++++++++++++++++++++++++++++
3 files changed, 3569 insertions(+), 3568 deletions(-)
delete mode 100644 drivers/net/ethernet/cadence/macb.c
create mode 100644 drivers/net/ethernet/cadence/macb_main.c
diff --git a/drivers/net/ethernet/cadence/Makefile b/drivers/net/ethernet/cadence/Makefile
index 4ba7559..31ea6e3 100644
--- a/drivers/net/ethernet/cadence/Makefile
+++ b/drivers/net/ethernet/cadence/Makefile
@@ -1,6 +1,7 @@
#
# Makefile for the Atmel network device drivers.
#
+macb-y := macb_main.o
obj-$(CONFIG_MACB) += macb.o
obj-$(CONFIG_MACB_PCI) += macb_pci.o
diff --git a/drivers/net/ethernet/cadence/macb.c b/drivers/net/ethernet/cadence/macb.c
deleted file mode 100644
index 3151429..0000000
--- a/drivers/net/ethernet/cadence/macb.c
+++ /dev/null
@@ -1,3568 +0,0 @@
-/*
- * Cadence MACB/GEM Ethernet Controller driver
- *
- * Copyright (C) 2004-2006 Atmel Corporation
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-#include <linux/clk.h>
-#include <linux/module.h>
-#include <linux/moduleparam.h>
-#include <linux/kernel.h>
-#include <linux/types.h>
-#include <linux/circ_buf.h>
-#include <linux/slab.h>
-#include <linux/init.h>
-#include <linux/io.h>
-#include <linux/gpio.h>
-#include <linux/gpio/consumer.h>
-#include <linux/interrupt.h>
-#include <linux/netdevice.h>
-#include <linux/etherdevice.h>
-#include <linux/dma-mapping.h>
-#include <linux/platform_data/macb.h>
-#include <linux/platform_device.h>
-#include <linux/phy.h>
-#include <linux/of.h>
-#include <linux/of_device.h>
-#include <linux/of_gpio.h>
-#include <linux/of_mdio.h>
-#include <linux/of_net.h>
-#include <linux/ip.h>
-#include <linux/udp.h>
-#include <linux/tcp.h>
-#include "macb.h"
-
-#define MACB_RX_BUFFER_SIZE 128
-#define RX_BUFFER_MULTIPLE 64 /* bytes */
-
-#define DEFAULT_RX_RING_SIZE 512 /* must be power of 2 */
-#define MIN_RX_RING_SIZE 64
-#define MAX_RX_RING_SIZE 8192
-#define RX_RING_BYTES(bp) (macb_dma_desc_get_size(bp) \
- * (bp)->rx_ring_size)
-
-#define DEFAULT_TX_RING_SIZE 512 /* must be power of 2 */
-#define MIN_TX_RING_SIZE 64
-#define MAX_TX_RING_SIZE 4096
-#define TX_RING_BYTES(bp) (macb_dma_desc_get_size(bp) \
- * (bp)->tx_ring_size)
-
-/* level of occupied TX descriptors under which we wake up TX process */
-#define MACB_TX_WAKEUP_THRESH(bp) (3 * (bp)->tx_ring_size / 4)
-
-#define MACB_RX_INT_FLAGS (MACB_BIT(RCOMP) | MACB_BIT(RXUBR) \
- | MACB_BIT(ISR_ROVR))
-#define MACB_TX_ERR_FLAGS (MACB_BIT(ISR_TUND) \
- | MACB_BIT(ISR_RLE) \
- | MACB_BIT(TXERR))
-#define MACB_TX_INT_FLAGS (MACB_TX_ERR_FLAGS | MACB_BIT(TCOMP))
-
-/* Max length of transmit frame must be a multiple of 8 bytes */
-#define MACB_TX_LEN_ALIGN 8
-#define MACB_MAX_TX_LEN ((unsigned int)((1 << MACB_TX_FRMLEN_SIZE) - 1) & ~((unsigned int)(MACB_TX_LEN_ALIGN - 1)))
-#define GEM_MAX_TX_LEN ((unsigned int)((1 << GEM_TX_FRMLEN_SIZE) - 1) & ~((unsigned int)(MACB_TX_LEN_ALIGN - 1)))
-
-#define GEM_MTU_MIN_SIZE ETH_MIN_MTU
-#define MACB_NETIF_LSO (NETIF_F_TSO | NETIF_F_UFO)
-
-#define MACB_WOL_HAS_MAGIC_PACKET (0x1 << 0)
-#define MACB_WOL_ENABLED (0x1 << 1)
-
-/* Graceful stop timeouts in us. We should allow up to
- * 1 frame time (10 Mbits/s, full-duplex, ignoring collisions)
- */
-#define MACB_HALT_TIMEOUT 1230
-
-/* DMA buffer descriptor might be different size
- * depends on hardware configuration:
- *
- * 1. dma address width 32 bits:
- * word 1: 32 bit address of Data Buffer
- * word 2: control
- *
- * 2. dma address width 64 bits:
- * word 1: 32 bit address of Data Buffer
- * word 2: control
- * word 3: upper 32 bit address of Data Buffer
- * word 4: unused
- *
- * 3. dma address width 32 bits with hardware timestamping:
- * word 1: 32 bit address of Data Buffer
- * word 2: control
- * word 3: timestamp word 1
- * word 4: timestamp word 2
- *
- * 4. dma address width 64 bits with hardware timestamping:
- * word 1: 32 bit address of Data Buffer
- * word 2: control
- * word 3: upper 32 bit address of Data Buffer
- * word 4: unused
- * word 5: timestamp word 1
- * word 6: timestamp word 2
- */
-static unsigned int macb_dma_desc_get_size(struct macb *bp)
-{
-#ifdef MACB_EXT_DESC
- unsigned int desc_size;
-
- switch (bp->hw_dma_cap) {
- case HW_DMA_CAP_64B:
- desc_size = sizeof(struct macb_dma_desc)
- + sizeof(struct macb_dma_desc_64);
- break;
- case HW_DMA_CAP_PTP:
- desc_size = sizeof(struct macb_dma_desc)
- + sizeof(struct macb_dma_desc_ptp);
- break;
- case HW_DMA_CAP_64B_PTP:
- desc_size = sizeof(struct macb_dma_desc)
- + sizeof(struct macb_dma_desc_64)
- + sizeof(struct macb_dma_desc_ptp);
- break;
- default:
- desc_size = sizeof(struct macb_dma_desc);
- }
- return desc_size;
-#endif
- return sizeof(struct macb_dma_desc);
-}
-
-static unsigned int macb_adj_dma_desc_idx(struct macb *bp, unsigned int desc_idx)
-{
-#ifdef MACB_EXT_DESC
- switch (bp->hw_dma_cap) {
- case HW_DMA_CAP_64B:
- case HW_DMA_CAP_PTP:
- desc_idx <<= 1;
- break;
- case HW_DMA_CAP_64B_PTP:
- desc_idx *= 3;
- break;
- default:
- break;
- }
- return desc_idx;
-#endif
- return desc_idx;
-}
-
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
-static struct macb_dma_desc_64 *macb_64b_desc(struct macb *bp, struct macb_dma_desc *desc)
-{
- if (bp->hw_dma_cap & HW_DMA_CAP_64B)
- return (struct macb_dma_desc_64 *)((void *)desc + sizeof(struct macb_dma_desc));
- return NULL;
-}
-#endif
-
-/* Ring buffer accessors */
-static unsigned int macb_tx_ring_wrap(struct macb *bp, unsigned int index)
-{
- return index & (bp->tx_ring_size - 1);
-}
-
-static struct macb_dma_desc *macb_tx_desc(struct macb_queue *queue,
- unsigned int index)
-{
- index = macb_tx_ring_wrap(queue->bp, index);
- index = macb_adj_dma_desc_idx(queue->bp, index);
- return &queue->tx_ring[index];
-}
-
-static struct macb_tx_skb *macb_tx_skb(struct macb_queue *queue,
- unsigned int index)
-{
- return &queue->tx_skb[macb_tx_ring_wrap(queue->bp, index)];
-}
-
-static dma_addr_t macb_tx_dma(struct macb_queue *queue, unsigned int index)
-{
- dma_addr_t offset;
-
- offset = macb_tx_ring_wrap(queue->bp, index) *
- macb_dma_desc_get_size(queue->bp);
-
- return queue->tx_ring_dma + offset;
-}
-
-static unsigned int macb_rx_ring_wrap(struct macb *bp, unsigned int index)
-{
- return index & (bp->rx_ring_size - 1);
-}
-
-static struct macb_dma_desc *macb_rx_desc(struct macb *bp, unsigned int index)
-{
- index = macb_rx_ring_wrap(bp, index);
- index = macb_adj_dma_desc_idx(bp, index);
- return &bp->rx_ring[index];
-}
-
-static void *macb_rx_buffer(struct macb *bp, unsigned int index)
-{
- return bp->rx_buffers + bp->rx_buffer_size *
- macb_rx_ring_wrap(bp, index);
-}
-
-/* I/O accessors */
-static u32 hw_readl_native(struct macb *bp, int offset)
-{
- return __raw_readl(bp->regs + offset);
-}
-
-static void hw_writel_native(struct macb *bp, int offset, u32 value)
-{
- __raw_writel(value, bp->regs + offset);
-}
-
-static u32 hw_readl(struct macb *bp, int offset)
-{
- return readl_relaxed(bp->regs + offset);
-}
-
-static void hw_writel(struct macb *bp, int offset, u32 value)
-{
- writel_relaxed(value, bp->regs + offset);
-}
-
-/* Find the CPU endianness by using the loopback bit of NCR register. When the
- * CPU is in big endian we need to program swapped mode for management
- * descriptor access.
- */
-static bool hw_is_native_io(void __iomem *addr)
-{
- u32 value = MACB_BIT(LLB);
-
- __raw_writel(value, addr + MACB_NCR);
- value = __raw_readl(addr + MACB_NCR);
-
- /* Write 0 back to disable everything */
- __raw_writel(0, addr + MACB_NCR);
-
- return value == MACB_BIT(LLB);
-}
-
-static bool hw_is_gem(void __iomem *addr, bool native_io)
-{
- u32 id;
-
- if (native_io)
- id = __raw_readl(addr + MACB_MID);
- else
- id = readl_relaxed(addr + MACB_MID);
-
- return MACB_BFEXT(IDNUM, id) >= 0x2;
-}
-
-static void macb_set_hwaddr(struct macb *bp)
-{
- u32 bottom;
- u16 top;
-
- bottom = cpu_to_le32(*((u32 *)bp->dev->dev_addr));
- macb_or_gem_writel(bp, SA1B, bottom);
- top = cpu_to_le16(*((u16 *)(bp->dev->dev_addr + 4)));
- macb_or_gem_writel(bp, SA1T, top);
-
- /* Clear unused address register sets */
- macb_or_gem_writel(bp, SA2B, 0);
- macb_or_gem_writel(bp, SA2T, 0);
- macb_or_gem_writel(bp, SA3B, 0);
- macb_or_gem_writel(bp, SA3T, 0);
- macb_or_gem_writel(bp, SA4B, 0);
- macb_or_gem_writel(bp, SA4T, 0);
-}
-
-static void macb_get_hwaddr(struct macb *bp)
-{
- struct macb_platform_data *pdata;
- u32 bottom;
- u16 top;
- u8 addr[6];
- int i;
-
- pdata = dev_get_platdata(&bp->pdev->dev);
-
- /* Check all 4 address register for valid address */
- for (i = 0; i < 4; i++) {
- bottom = macb_or_gem_readl(bp, SA1B + i * 8);
- top = macb_or_gem_readl(bp, SA1T + i * 8);
-
- if (pdata && pdata->rev_eth_addr) {
- addr[5] = bottom & 0xff;
- addr[4] = (bottom >> 8) & 0xff;
- addr[3] = (bottom >> 16) & 0xff;
- addr[2] = (bottom >> 24) & 0xff;
- addr[1] = top & 0xff;
- addr[0] = (top & 0xff00) >> 8;
- } else {
- addr[0] = bottom & 0xff;
- addr[1] = (bottom >> 8) & 0xff;
- addr[2] = (bottom >> 16) & 0xff;
- addr[3] = (bottom >> 24) & 0xff;
- addr[4] = top & 0xff;
- addr[5] = (top >> 8) & 0xff;
- }
-
- if (is_valid_ether_addr(addr)) {
- memcpy(bp->dev->dev_addr, addr, sizeof(addr));
- return;
- }
- }
-
- dev_info(&bp->pdev->dev, "invalid hw address, using random\n");
- eth_hw_addr_random(bp->dev);
-}
-
-static int macb_mdio_read(struct mii_bus *bus, int mii_id, int regnum)
-{
- struct macb *bp = bus->priv;
- int value;
-
- macb_writel(bp, MAN, (MACB_BF(SOF, MACB_MAN_SOF)
- | MACB_BF(RW, MACB_MAN_READ)
- | MACB_BF(PHYA, mii_id)
- | MACB_BF(REGA, regnum)
- | MACB_BF(CODE, MACB_MAN_CODE)));
-
- /* wait for end of transfer */
- while (!MACB_BFEXT(IDLE, macb_readl(bp, NSR)))
- cpu_relax();
-
- value = MACB_BFEXT(DATA, macb_readl(bp, MAN));
-
- return value;
-}
-
-static int macb_mdio_write(struct mii_bus *bus, int mii_id, int regnum,
- u16 value)
-{
- struct macb *bp = bus->priv;
-
- macb_writel(bp, MAN, (MACB_BF(SOF, MACB_MAN_SOF)
- | MACB_BF(RW, MACB_MAN_WRITE)
- | MACB_BF(PHYA, mii_id)
- | MACB_BF(REGA, regnum)
- | MACB_BF(CODE, MACB_MAN_CODE)
- | MACB_BF(DATA, value)));
-
- /* wait for end of transfer */
- while (!MACB_BFEXT(IDLE, macb_readl(bp, NSR)))
- cpu_relax();
-
- return 0;
-}
-
-/**
- * macb_set_tx_clk() - Set a clock to a new frequency
- * @clk Pointer to the clock to change
- * @rate New frequency in Hz
- * @dev Pointer to the struct net_device
- */
-static void macb_set_tx_clk(struct clk *clk, int speed, struct net_device *dev)
-{
- long ferr, rate, rate_rounded;
-
- if (!clk)
- return;
-
- switch (speed) {
- case SPEED_10:
- rate = 2500000;
- break;
- case SPEED_100:
- rate = 25000000;
- break;
- case SPEED_1000:
- rate = 125000000;
- break;
- default:
- return;
- }
-
- rate_rounded = clk_round_rate(clk, rate);
- if (rate_rounded < 0)
- return;
-
- /* RGMII allows 50 ppm frequency error. Test and warn if this limit
- * is not satisfied.
- */
- ferr = abs(rate_rounded - rate);
- ferr = DIV_ROUND_UP(ferr, rate / 100000);
- if (ferr > 5)
- netdev_warn(dev, "unable to generate target frequency: %ld Hz\n",
- rate);
-
- if (clk_set_rate(clk, rate_rounded))
- netdev_err(dev, "adjusting tx_clk failed.\n");
-}
-
-static void macb_handle_link_change(struct net_device *dev)
-{
- struct macb *bp = netdev_priv(dev);
- struct phy_device *phydev = dev->phydev;
- unsigned long flags;
- int status_change = 0;
-
- spin_lock_irqsave(&bp->lock, flags);
-
- if (phydev->link) {
- if ((bp->speed != phydev->speed) ||
- (bp->duplex != phydev->duplex)) {
- u32 reg;
-
- reg = macb_readl(bp, NCFGR);
- reg &= ~(MACB_BIT(SPD) | MACB_BIT(FD));
- if (macb_is_gem(bp))
- reg &= ~GEM_BIT(GBE);
-
- if (phydev->duplex)
- reg |= MACB_BIT(FD);
- if (phydev->speed == SPEED_100)
- reg |= MACB_BIT(SPD);
- if (phydev->speed == SPEED_1000 &&
- bp->caps & MACB_CAPS_GIGABIT_MODE_AVAILABLE)
- reg |= GEM_BIT(GBE);
-
- macb_or_gem_writel(bp, NCFGR, reg);
-
- bp->speed = phydev->speed;
- bp->duplex = phydev->duplex;
- status_change = 1;
- }
- }
-
- if (phydev->link != bp->link) {
- if (!phydev->link) {
- bp->speed = 0;
- bp->duplex = -1;
- }
- bp->link = phydev->link;
-
- status_change = 1;
- }
-
- spin_unlock_irqrestore(&bp->lock, flags);
-
- if (status_change) {
- if (phydev->link) {
- /* Update the TX clock rate if and only if the link is
- * up and there has been a link change.
- */
- macb_set_tx_clk(bp->tx_clk, phydev->speed, dev);
-
- netif_carrier_on(dev);
- netdev_info(dev, "link up (%d/%s)\n",
- phydev->speed,
- phydev->duplex == DUPLEX_FULL ?
- "Full" : "Half");
- } else {
- netif_carrier_off(dev);
- netdev_info(dev, "link down\n");
- }
- }
-}
-
-/* based on au1000_eth. c*/
-static int macb_mii_probe(struct net_device *dev)
-{
- struct macb *bp = netdev_priv(dev);
- struct macb_platform_data *pdata;
- struct phy_device *phydev;
- int phy_irq;
- int ret;
-
- phydev = phy_find_first(bp->mii_bus);
- if (!phydev) {
- netdev_err(dev, "no PHY found\n");
- return -ENXIO;
- }
-
- pdata = dev_get_platdata(&bp->pdev->dev);
- if (pdata) {
- if (gpio_is_valid(pdata->phy_irq_pin)) {
- ret = devm_gpio_request(&bp->pdev->dev,
- pdata->phy_irq_pin, "phy int");
- if (!ret) {
- phy_irq = gpio_to_irq(pdata->phy_irq_pin);
- phydev->irq = (phy_irq < 0) ? PHY_POLL : phy_irq;
- }
- } else {
- phydev->irq = PHY_POLL;
- }
- }
-
- /* attach the mac to the phy */
- ret = phy_connect_direct(dev, phydev, &macb_handle_link_change,
- bp->phy_interface);
- if (ret) {
- netdev_err(dev, "Could not attach to PHY\n");
- return ret;
- }
-
- /* mask with MAC supported features */
- if (macb_is_gem(bp) && bp->caps & MACB_CAPS_GIGABIT_MODE_AVAILABLE)
- phydev->supported &= PHY_GBIT_FEATURES;
- else
- phydev->supported &= PHY_BASIC_FEATURES;
-
- if (bp->caps & MACB_CAPS_NO_GIGABIT_HALF)
- phydev->supported &= ~SUPPORTED_1000baseT_Half;
-
- phydev->advertising = phydev->supported;
-
- bp->link = 0;
- bp->speed = 0;
- bp->duplex = -1;
-
- return 0;
-}
-
-static int macb_mii_init(struct macb *bp)
-{
- struct macb_platform_data *pdata;
- struct device_node *np;
- int err = -ENXIO, i;
-
- /* Enable management port */
- macb_writel(bp, NCR, MACB_BIT(MPE));
-
- bp->mii_bus = mdiobus_alloc();
- if (!bp->mii_bus) {
- err = -ENOMEM;
- goto err_out;
- }
-
- bp->mii_bus->name = "MACB_mii_bus";
- bp->mii_bus->read = &macb_mdio_read;
- bp->mii_bus->write = &macb_mdio_write;
- snprintf(bp->mii_bus->id, MII_BUS_ID_SIZE, "%s-%x",
- bp->pdev->name, bp->pdev->id);
- bp->mii_bus->priv = bp;
- bp->mii_bus->parent = &bp->pdev->dev;
- pdata = dev_get_platdata(&bp->pdev->dev);
-
- dev_set_drvdata(&bp->dev->dev, bp->mii_bus);
-
- np = bp->pdev->dev.of_node;
- if (np) {
- /* try dt phy registration */
- err = of_mdiobus_register(bp->mii_bus, np);
-
- /* fallback to standard phy registration if no phy were
- * found during dt phy registration
- */
- if (!err && !phy_find_first(bp->mii_bus)) {
- for (i = 0; i < PHY_MAX_ADDR; i++) {
- struct phy_device *phydev;
-
- phydev = mdiobus_scan(bp->mii_bus, i);
- if (IS_ERR(phydev) &&
- PTR_ERR(phydev) != -ENODEV) {
- err = PTR_ERR(phydev);
- break;
- }
- }
-
- if (err)
- goto err_out_unregister_bus;
- }
- } else {
- for (i = 0; i < PHY_MAX_ADDR; i++)
- bp->mii_bus->irq[i] = PHY_POLL;
-
- if (pdata)
- bp->mii_bus->phy_mask = pdata->phy_mask;
-
- err = mdiobus_register(bp->mii_bus);
- }
-
- if (err)
- goto err_out_free_mdiobus;
-
- err = macb_mii_probe(bp->dev);
- if (err)
- goto err_out_unregister_bus;
-
- return 0;
-
-err_out_unregister_bus:
- mdiobus_unregister(bp->mii_bus);
-err_out_free_mdiobus:
- mdiobus_free(bp->mii_bus);
-err_out:
- return err;
-}
-
-static void macb_update_stats(struct macb *bp)
-{
- u32 *p = &bp->hw_stats.macb.rx_pause_frames;
- u32 *end = &bp->hw_stats.macb.tx_pause_frames + 1;
- int offset = MACB_PFR;
-
- WARN_ON((unsigned long)(end - p - 1) != (MACB_TPF - MACB_PFR) / 4);
-
- for (; p < end; p++, offset += 4)
- *p += bp->macb_reg_readl(bp, offset);
-}
-
-static int macb_halt_tx(struct macb *bp)
-{
- unsigned long halt_time, timeout;
- u32 status;
-
- macb_writel(bp, NCR, macb_readl(bp, NCR) | MACB_BIT(THALT));
-
- timeout = jiffies + usecs_to_jiffies(MACB_HALT_TIMEOUT);
- do {
- halt_time = jiffies;
- status = macb_readl(bp, TSR);
- if (!(status & MACB_BIT(TGO)))
- return 0;
-
- usleep_range(10, 250);
- } while (time_before(halt_time, timeout));
-
- return -ETIMEDOUT;
-}
-
-static void macb_tx_unmap(struct macb *bp, struct macb_tx_skb *tx_skb)
-{
- if (tx_skb->mapping) {
- if (tx_skb->mapped_as_page)
- dma_unmap_page(&bp->pdev->dev, tx_skb->mapping,
- tx_skb->size, DMA_TO_DEVICE);
- else
- dma_unmap_single(&bp->pdev->dev, tx_skb->mapping,
- tx_skb->size, DMA_TO_DEVICE);
- tx_skb->mapping = 0;
- }
-
- if (tx_skb->skb) {
- dev_kfree_skb_any(tx_skb->skb);
- tx_skb->skb = NULL;
- }
-}
-
-static void macb_set_addr(struct macb *bp, struct macb_dma_desc *desc, dma_addr_t addr)
-{
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- struct macb_dma_desc_64 *desc_64;
-
- if (bp->hw_dma_cap & HW_DMA_CAP_64B) {
- desc_64 = macb_64b_desc(bp, desc);
- desc_64->addrh = upper_32_bits(addr);
- }
-#endif
- desc->addr = lower_32_bits(addr);
-}
-
-static dma_addr_t macb_get_addr(struct macb *bp, struct macb_dma_desc *desc)
-{
- dma_addr_t addr = 0;
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- struct macb_dma_desc_64 *desc_64;
-
- if (bp->hw_dma_cap & HW_DMA_CAP_64B) {
- desc_64 = macb_64b_desc(bp, desc);
- addr = ((u64)(desc_64->addrh) << 32);
- }
-#endif
- addr |= MACB_BF(RX_WADDR, MACB_BFEXT(RX_WADDR, desc->addr));
- return addr;
-}
-
-static void macb_tx_error_task(struct work_struct *work)
-{
- struct macb_queue *queue = container_of(work, struct macb_queue,
- tx_error_task);
- struct macb *bp = queue->bp;
- struct macb_tx_skb *tx_skb;
- struct macb_dma_desc *desc;
- struct sk_buff *skb;
- unsigned int tail;
- unsigned long flags;
-
- netdev_vdbg(bp->dev, "macb_tx_error_task: q = %u, t = %u, h = %u\n",
- (unsigned int)(queue - bp->queues),
- queue->tx_tail, queue->tx_head);
-
- /* Prevent the queue IRQ handlers from running: each of them may call
- * macb_tx_interrupt(), which in turn may call netif_wake_subqueue().
- * As explained below, we have to halt the transmission before updating
- * TBQP registers so we call netif_tx_stop_all_queues() to notify the
- * network engine about the macb/gem being halted.
- */
- spin_lock_irqsave(&bp->lock, flags);
-
- /* Make sure nobody is trying to queue up new packets */
- netif_tx_stop_all_queues(bp->dev);
-
- /* Stop transmission now
- * (in case we have just queued new packets)
- * macb/gem must be halted to write TBQP register
- */
- if (macb_halt_tx(bp))
- /* Just complain for now, reinitializing TX path can be good */
- netdev_err(bp->dev, "BUG: halt tx timed out\n");
-
- /* Treat frames in TX queue including the ones that caused the error.
- * Free transmit buffers in upper layer.
- */
- for (tail = queue->tx_tail; tail != queue->tx_head; tail++) {
- u32 ctrl;
-
- desc = macb_tx_desc(queue, tail);
- ctrl = desc->ctrl;
- tx_skb = macb_tx_skb(queue, tail);
- skb = tx_skb->skb;
-
- if (ctrl & MACB_BIT(TX_USED)) {
- /* skb is set for the last buffer of the frame */
- while (!skb) {
- macb_tx_unmap(bp, tx_skb);
- tail++;
- tx_skb = macb_tx_skb(queue, tail);
- skb = tx_skb->skb;
- }
-
- /* ctrl still refers to the first buffer descriptor
- * since it's the only one written back by the hardware
- */
- if (!(ctrl & MACB_BIT(TX_BUF_EXHAUSTED))) {
- netdev_vdbg(bp->dev, "txerr skb %u (data %p) TX complete\n",
- macb_tx_ring_wrap(bp, tail),
- skb->data);
- bp->dev->stats.tx_packets++;
- bp->dev->stats.tx_bytes += skb->len;
- }
- } else {
- /* "Buffers exhausted mid-frame" errors may only happen
- * if the driver is buggy, so complain loudly about
- * those. Statistics are updated by hardware.
- */
- if (ctrl & MACB_BIT(TX_BUF_EXHAUSTED))
- netdev_err(bp->dev,
- "BUG: TX buffers exhausted mid-frame\n");
-
- desc->ctrl = ctrl | MACB_BIT(TX_USED);
- }
-
- macb_tx_unmap(bp, tx_skb);
- }
-
- /* Set end of TX queue */
- desc = macb_tx_desc(queue, 0);
- macb_set_addr(bp, desc, 0);
- desc->ctrl = MACB_BIT(TX_USED);
-
- /* Make descriptor updates visible to hardware */
- wmb();
-
- /* Reinitialize the TX desc queue */
- queue_writel(queue, TBQP, lower_32_bits(queue->tx_ring_dma));
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (bp->hw_dma_cap & HW_DMA_CAP_64B)
- queue_writel(queue, TBQPH, upper_32_bits(queue->tx_ring_dma));
-#endif
- /* Make TX ring reflect state of hardware */
- queue->tx_head = 0;
- queue->tx_tail = 0;
-
- /* Housework before enabling TX IRQ */
- macb_writel(bp, TSR, macb_readl(bp, TSR));
- queue_writel(queue, IER, MACB_TX_INT_FLAGS);
-
- /* Now we are ready to start transmission again */
- netif_tx_start_all_queues(bp->dev);
- macb_writel(bp, NCR, macb_readl(bp, NCR) | MACB_BIT(TSTART));
-
- spin_unlock_irqrestore(&bp->lock, flags);
-}
-
-static void macb_tx_interrupt(struct macb_queue *queue)
-{
- unsigned int tail;
- unsigned int head;
- u32 status;
- struct macb *bp = queue->bp;
- u16 queue_index = queue - bp->queues;
-
- status = macb_readl(bp, TSR);
- macb_writel(bp, TSR, status);
-
- if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
- queue_writel(queue, ISR, MACB_BIT(TCOMP));
-
- netdev_vdbg(bp->dev, "macb_tx_interrupt status = 0x%03lx\n",
- (unsigned long)status);
-
- head = queue->tx_head;
- for (tail = queue->tx_tail; tail != head; tail++) {
- struct macb_tx_skb *tx_skb;
- struct sk_buff *skb;
- struct macb_dma_desc *desc;
- u32 ctrl;
-
- desc = macb_tx_desc(queue, tail);
-
- /* Make hw descriptor updates visible to CPU */
- rmb();
-
- ctrl = desc->ctrl;
-
- /* TX_USED bit is only set by hardware on the very first buffer
- * descriptor of the transmitted frame.
- */
- if (!(ctrl & MACB_BIT(TX_USED)))
- break;
-
- /* Process all buffers of the current transmitted frame */
- for (;; tail++) {
- tx_skb = macb_tx_skb(queue, tail);
- skb = tx_skb->skb;
-
- /* First, update TX stats if needed */
- if (skb) {
- netdev_vdbg(bp->dev, "skb %u (data %p) TX complete\n",
- macb_tx_ring_wrap(bp, tail),
- skb->data);
- bp->dev->stats.tx_packets++;
- bp->dev->stats.tx_bytes += skb->len;
- }
-
- /* Now we can safely release resources */
- macb_tx_unmap(bp, tx_skb);
-
- /* skb is set only for the last buffer of the frame.
- * WARNING: at this point skb has been freed by
- * macb_tx_unmap().
- */
- if (skb)
- break;
- }
- }
-
- queue->tx_tail = tail;
- if (__netif_subqueue_stopped(bp->dev, queue_index) &&
- CIRC_CNT(queue->tx_head, queue->tx_tail,
- bp->tx_ring_size) <= MACB_TX_WAKEUP_THRESH(bp))
- netif_wake_subqueue(bp->dev, queue_index);
-}
-
-static void gem_rx_refill(struct macb *bp)
-{
- unsigned int entry;
- struct sk_buff *skb;
- dma_addr_t paddr;
- struct macb_dma_desc *desc;
-
- while (CIRC_SPACE(bp->rx_prepared_head, bp->rx_tail,
- bp->rx_ring_size) > 0) {
- entry = macb_rx_ring_wrap(bp, bp->rx_prepared_head);
-
- /* Make hw descriptor updates visible to CPU */
- rmb();
-
- bp->rx_prepared_head++;
- desc = macb_rx_desc(bp, entry);
-
- if (!bp->rx_skbuff[entry]) {
- /* allocate sk_buff for this free entry in ring */
- skb = netdev_alloc_skb(bp->dev, bp->rx_buffer_size);
- if (unlikely(!skb)) {
- netdev_err(bp->dev,
- "Unable to allocate sk_buff\n");
- break;
- }
-
- /* now fill corresponding descriptor entry */
- paddr = dma_map_single(&bp->pdev->dev, skb->data,
- bp->rx_buffer_size,
- DMA_FROM_DEVICE);
- if (dma_mapping_error(&bp->pdev->dev, paddr)) {
- dev_kfree_skb(skb);
- break;
- }
-
- bp->rx_skbuff[entry] = skb;
-
- if (entry == bp->rx_ring_size - 1)
- paddr |= MACB_BIT(RX_WRAP);
- macb_set_addr(bp, desc, paddr);
- desc->ctrl = 0;
-
- /* properly align Ethernet header */
- skb_reserve(skb, NET_IP_ALIGN);
- } else {
- desc->addr &= ~MACB_BIT(RX_USED);
- desc->ctrl = 0;
- }
- }
-
- /* Make descriptor updates visible to hardware */
- wmb();
-
- netdev_vdbg(bp->dev, "rx ring: prepared head %d, tail %d\n",
- bp->rx_prepared_head, bp->rx_tail);
-}
-
-/* Mark DMA descriptors from begin up to and not including end as unused */
-static void discard_partial_frame(struct macb *bp, unsigned int begin,
- unsigned int end)
-{
- unsigned int frag;
-
- for (frag = begin; frag != end; frag++) {
- struct macb_dma_desc *desc = macb_rx_desc(bp, frag);
-
- desc->addr &= ~MACB_BIT(RX_USED);
- }
-
- /* Make descriptor updates visible to hardware */
- wmb();
-
- /* When this happens, the hardware stats registers for
- * whatever caused this is updated, so we don't have to record
- * anything.
- */
-}
-
-static int gem_rx(struct macb *bp, int budget)
-{
- unsigned int len;
- unsigned int entry;
- struct sk_buff *skb;
- struct macb_dma_desc *desc;
- int count = 0;
-
- while (count < budget) {
- u32 ctrl;
- dma_addr_t addr;
- bool rxused;
-
- entry = macb_rx_ring_wrap(bp, bp->rx_tail);
- desc = macb_rx_desc(bp, entry);
-
- /* Make hw descriptor updates visible to CPU */
- rmb();
-
- rxused = (desc->addr & MACB_BIT(RX_USED)) ? true : false;
- addr = macb_get_addr(bp, desc);
- ctrl = desc->ctrl;
-
- if (!rxused)
- break;
-
- bp->rx_tail++;
- count++;
-
- if (!(ctrl & MACB_BIT(RX_SOF) && ctrl & MACB_BIT(RX_EOF))) {
- netdev_err(bp->dev,
- "not whole frame pointed by descriptor\n");
- bp->dev->stats.rx_dropped++;
- break;
- }
- skb = bp->rx_skbuff[entry];
- if (unlikely(!skb)) {
- netdev_err(bp->dev,
- "inconsistent Rx descriptor chain\n");
- bp->dev->stats.rx_dropped++;
- break;
- }
- /* now everything is ready for receiving packet */
- bp->rx_skbuff[entry] = NULL;
- len = ctrl & bp->rx_frm_len_mask;
-
- netdev_vdbg(bp->dev, "gem_rx %u (len %u)\n", entry, len);
-
- skb_put(skb, len);
- dma_unmap_single(&bp->pdev->dev, addr,
- bp->rx_buffer_size, DMA_FROM_DEVICE);
-
- skb->protocol = eth_type_trans(skb, bp->dev);
- skb_checksum_none_assert(skb);
- if (bp->dev->features & NETIF_F_RXCSUM &&
- !(bp->dev->flags & IFF_PROMISC) &&
- GEM_BFEXT(RX_CSUM, ctrl) & GEM_RX_CSUM_CHECKED_MASK)
- skb->ip_summed = CHECKSUM_UNNECESSARY;
-
- bp->dev->stats.rx_packets++;
- bp->dev->stats.rx_bytes += skb->len;
-
-#if defined(DEBUG) && defined(VERBOSE_DEBUG)
- netdev_vdbg(bp->dev, "received skb of length %u, csum: %08x\n",
- skb->len, skb->csum);
- print_hex_dump(KERN_DEBUG, " mac: ", DUMP_PREFIX_ADDRESS, 16, 1,
- skb_mac_header(skb), 16, true);
- print_hex_dump(KERN_DEBUG, "data: ", DUMP_PREFIX_ADDRESS, 16, 1,
- skb->data, 32, true);
-#endif
-
- netif_receive_skb(skb);
- }
-
- gem_rx_refill(bp);
-
- return count;
-}
-
-static int macb_rx_frame(struct macb *bp, unsigned int first_frag,
- unsigned int last_frag)
-{
- unsigned int len;
- unsigned int frag;
- unsigned int offset;
- struct sk_buff *skb;
- struct macb_dma_desc *desc;
-
- desc = macb_rx_desc(bp, last_frag);
- len = desc->ctrl & bp->rx_frm_len_mask;
-
- netdev_vdbg(bp->dev, "macb_rx_frame frags %u - %u (len %u)\n",
- macb_rx_ring_wrap(bp, first_frag),
- macb_rx_ring_wrap(bp, last_frag), len);
-
- /* The ethernet header starts NET_IP_ALIGN bytes into the
- * first buffer. Since the header is 14 bytes, this makes the
- * payload word-aligned.
- *
- * Instead of calling skb_reserve(NET_IP_ALIGN), we just copy
- * the two padding bytes into the skb so that we avoid hitting
- * the slowpath in memcpy(), and pull them off afterwards.
- */
- skb = netdev_alloc_skb(bp->dev, len + NET_IP_ALIGN);
- if (!skb) {
- bp->dev->stats.rx_dropped++;
- for (frag = first_frag; ; frag++) {
- desc = macb_rx_desc(bp, frag);
- desc->addr &= ~MACB_BIT(RX_USED);
- if (frag == last_frag)
- break;
- }
-
- /* Make descriptor updates visible to hardware */
- wmb();
-
- return 1;
- }
-
- offset = 0;
- len += NET_IP_ALIGN;
- skb_checksum_none_assert(skb);
- skb_put(skb, len);
-
- for (frag = first_frag; ; frag++) {
- unsigned int frag_len = bp->rx_buffer_size;
-
- if (offset + frag_len > len) {
- if (unlikely(frag != last_frag)) {
- dev_kfree_skb_any(skb);
- return -1;
- }
- frag_len = len - offset;
- }
- skb_copy_to_linear_data_offset(skb, offset,
- macb_rx_buffer(bp, frag),
- frag_len);
- offset += bp->rx_buffer_size;
- desc = macb_rx_desc(bp, frag);
- desc->addr &= ~MACB_BIT(RX_USED);
-
- if (frag == last_frag)
- break;
- }
-
- /* Make descriptor updates visible to hardware */
- wmb();
-
- __skb_pull(skb, NET_IP_ALIGN);
- skb->protocol = eth_type_trans(skb, bp->dev);
-
- bp->dev->stats.rx_packets++;
- bp->dev->stats.rx_bytes += skb->len;
- netdev_vdbg(bp->dev, "received skb of length %u, csum: %08x\n",
- skb->len, skb->csum);
- netif_receive_skb(skb);
-
- return 0;
-}
-
-static inline void macb_init_rx_ring(struct macb *bp)
-{
- dma_addr_t addr;
- struct macb_dma_desc *desc = NULL;
- int i;
-
- addr = bp->rx_buffers_dma;
- for (i = 0; i < bp->rx_ring_size; i++) {
- desc = macb_rx_desc(bp, i);
- macb_set_addr(bp, desc, addr);
- desc->ctrl = 0;
- addr += bp->rx_buffer_size;
- }
- desc->addr |= MACB_BIT(RX_WRAP);
- bp->rx_tail = 0;
-}
-
-static int macb_rx(struct macb *bp, int budget)
-{
- bool reset_rx_queue = false;
- int received = 0;
- unsigned int tail;
- int first_frag = -1;
-
- for (tail = bp->rx_tail; budget > 0; tail++) {
- struct macb_dma_desc *desc = macb_rx_desc(bp, tail);
- u32 ctrl;
-
- /* Make hw descriptor updates visible to CPU */
- rmb();
-
- ctrl = desc->ctrl;
-
- if (!(desc->addr & MACB_BIT(RX_USED)))
- break;
-
- if (ctrl & MACB_BIT(RX_SOF)) {
- if (first_frag != -1)
- discard_partial_frame(bp, first_frag, tail);
- first_frag = tail;
- }
-
- if (ctrl & MACB_BIT(RX_EOF)) {
- int dropped;
-
- if (unlikely(first_frag == -1)) {
- reset_rx_queue = true;
- continue;
- }
-
- dropped = macb_rx_frame(bp, first_frag, tail);
- first_frag = -1;
- if (unlikely(dropped < 0)) {
- reset_rx_queue = true;
- continue;
- }
- if (!dropped) {
- received++;
- budget--;
- }
- }
- }
-
- if (unlikely(reset_rx_queue)) {
- unsigned long flags;
- u32 ctrl;
-
- netdev_err(bp->dev, "RX queue corruption: reset it\n");
-
- spin_lock_irqsave(&bp->lock, flags);
-
- ctrl = macb_readl(bp, NCR);
- macb_writel(bp, NCR, ctrl & ~MACB_BIT(RE));
-
- macb_init_rx_ring(bp);
- macb_writel(bp, RBQP, bp->rx_ring_dma);
-
- macb_writel(bp, NCR, ctrl | MACB_BIT(RE));
-
- spin_unlock_irqrestore(&bp->lock, flags);
- return received;
- }
-
- if (first_frag != -1)
- bp->rx_tail = first_frag;
- else
- bp->rx_tail = tail;
-
- return received;
-}
-
-static int macb_poll(struct napi_struct *napi, int budget)
-{
- struct macb *bp = container_of(napi, struct macb, napi);
- int work_done;
- u32 status;
-
- status = macb_readl(bp, RSR);
- macb_writel(bp, RSR, status);
-
- work_done = 0;
-
- netdev_vdbg(bp->dev, "poll: status = %08lx, budget = %d\n",
- (unsigned long)status, budget);
-
- work_done = bp->macbgem_ops.mog_rx(bp, budget);
- if (work_done < budget) {
- napi_complete_done(napi, work_done);
-
- /* Packets received while interrupts were disabled */
- status = macb_readl(bp, RSR);
- if (status) {
- if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
- macb_writel(bp, ISR, MACB_BIT(RCOMP));
- napi_reschedule(napi);
- } else {
- macb_writel(bp, IER, MACB_RX_INT_FLAGS);
- }
- }
-
- /* TODO: Handle errors */
-
- return work_done;
-}
-
-static irqreturn_t macb_interrupt(int irq, void *dev_id)
-{
- struct macb_queue *queue = dev_id;
- struct macb *bp = queue->bp;
- struct net_device *dev = bp->dev;
- u32 status, ctrl;
-
- status = queue_readl(queue, ISR);
-
- if (unlikely(!status))
- return IRQ_NONE;
-
- spin_lock(&bp->lock);
-
- while (status) {
- /* close possible race with dev_close */
- if (unlikely(!netif_running(dev))) {
- queue_writel(queue, IDR, -1);
- if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
- queue_writel(queue, ISR, -1);
- break;
- }
-
- netdev_vdbg(bp->dev, "queue = %u, isr = 0x%08lx\n",
- (unsigned int)(queue - bp->queues),
- (unsigned long)status);
-
- if (status & MACB_RX_INT_FLAGS) {
- /* There's no point taking any more interrupts
- * until we have processed the buffers. The
- * scheduling call may fail if the poll routine
- * is already scheduled, so disable interrupts
- * now.
- */
- queue_writel(queue, IDR, MACB_RX_INT_FLAGS);
- if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
- queue_writel(queue, ISR, MACB_BIT(RCOMP));
-
- if (napi_schedule_prep(&bp->napi)) {
- netdev_vdbg(bp->dev, "scheduling RX softirq\n");
- __napi_schedule(&bp->napi);
- }
- }
-
- if (unlikely(status & (MACB_TX_ERR_FLAGS))) {
- queue_writel(queue, IDR, MACB_TX_INT_FLAGS);
- schedule_work(&queue->tx_error_task);
-
- if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
- queue_writel(queue, ISR, MACB_TX_ERR_FLAGS);
-
- break;
- }
-
- if (status & MACB_BIT(TCOMP))
- macb_tx_interrupt(queue);
-
- /* Link change detection isn't possible with RMII, so we'll
- * add that if/when we get our hands on a full-blown MII PHY.
- */
-
- /* There is a hardware issue under heavy load where DMA can
- * stop, this causes endless "used buffer descriptor read"
- * interrupts but it can be cleared by re-enabling RX. See
- * the at91 manual, section 41.3.1 or the Zynq manual
- * section 16.7.4 for details.
- */
- if (status & MACB_BIT(RXUBR)) {
- ctrl = macb_readl(bp, NCR);
- macb_writel(bp, NCR, ctrl & ~MACB_BIT(RE));
- wmb();
- macb_writel(bp, NCR, ctrl | MACB_BIT(RE));
-
- if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
- queue_writel(queue, ISR, MACB_BIT(RXUBR));
- }
-
- if (status & MACB_BIT(ISR_ROVR)) {
- /* We missed at least one packet */
- if (macb_is_gem(bp))
- bp->hw_stats.gem.rx_overruns++;
- else
- bp->hw_stats.macb.rx_overruns++;
-
- if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
- queue_writel(queue, ISR, MACB_BIT(ISR_ROVR));
- }
-
- if (status & MACB_BIT(HRESP)) {
- /* TODO: Reset the hardware, and maybe move the
- * netdev_err to a lower-priority context as well
- * (work queue?)
- */
- netdev_err(dev, "DMA bus error: HRESP not OK\n");
-
- if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
- queue_writel(queue, ISR, MACB_BIT(HRESP));
- }
-
- status = queue_readl(queue, ISR);
- }
-
- spin_unlock(&bp->lock);
-
- return IRQ_HANDLED;
-}
-
-#ifdef CONFIG_NET_POLL_CONTROLLER
-/* Polling receive - used by netconsole and other diagnostic tools
- * to allow network i/o with interrupts disabled.
- */
-static void macb_poll_controller(struct net_device *dev)
-{
- struct macb *bp = netdev_priv(dev);
- struct macb_queue *queue;
- unsigned long flags;
- unsigned int q;
-
- local_irq_save(flags);
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue)
- macb_interrupt(dev->irq, queue);
- local_irq_restore(flags);
-}
-#endif
-
-static unsigned int macb_tx_map(struct macb *bp,
- struct macb_queue *queue,
- struct sk_buff *skb,
- unsigned int hdrlen)
-{
- dma_addr_t mapping;
- unsigned int len, entry, i, tx_head = queue->tx_head;
- struct macb_tx_skb *tx_skb = NULL;
- struct macb_dma_desc *desc;
- unsigned int offset, size, count = 0;
- unsigned int f, nr_frags = skb_shinfo(skb)->nr_frags;
- unsigned int eof = 1, mss_mfs = 0;
- u32 ctrl, lso_ctrl = 0, seq_ctrl = 0;
-
- /* LSO */
- if (skb_shinfo(skb)->gso_size != 0) {
- if (ip_hdr(skb)->protocol == IPPROTO_UDP)
- /* UDP - UFO */
- lso_ctrl = MACB_LSO_UFO_ENABLE;
- else
- /* TCP - TSO */
- lso_ctrl = MACB_LSO_TSO_ENABLE;
- }
-
- /* First, map non-paged data */
- len = skb_headlen(skb);
-
- /* first buffer length */
- size = hdrlen;
-
- offset = 0;
- while (len) {
- entry = macb_tx_ring_wrap(bp, tx_head);
- tx_skb = &queue->tx_skb[entry];
-
- mapping = dma_map_single(&bp->pdev->dev,
- skb->data + offset,
- size, DMA_TO_DEVICE);
- if (dma_mapping_error(&bp->pdev->dev, mapping))
- goto dma_error;
-
- /* Save info to properly release resources */
- tx_skb->skb = NULL;
- tx_skb->mapping = mapping;
- tx_skb->size = size;
- tx_skb->mapped_as_page = false;
-
- len -= size;
- offset += size;
- count++;
- tx_head++;
-
- size = min(len, bp->max_tx_length);
- }
-
- /* Then, map paged data from fragments */
- for (f = 0; f < nr_frags; f++) {
- const skb_frag_t *frag = &skb_shinfo(skb)->frags[f];
-
- len = skb_frag_size(frag);
- offset = 0;
- while (len) {
- size = min(len, bp->max_tx_length);
- entry = macb_tx_ring_wrap(bp, tx_head);
- tx_skb = &queue->tx_skb[entry];
-
- mapping = skb_frag_dma_map(&bp->pdev->dev, frag,
- offset, size, DMA_TO_DEVICE);
- if (dma_mapping_error(&bp->pdev->dev, mapping))
- goto dma_error;
-
- /* Save info to properly release resources */
- tx_skb->skb = NULL;
- tx_skb->mapping = mapping;
- tx_skb->size = size;
- tx_skb->mapped_as_page = true;
-
- len -= size;
- offset += size;
- count++;
- tx_head++;
- }
- }
-
- /* Should never happen */
- if (unlikely(!tx_skb)) {
- netdev_err(bp->dev, "BUG! empty skb!\n");
- return 0;
- }
-
- /* This is the last buffer of the frame: save socket buffer */
- tx_skb->skb = skb;
-
- /* Update TX ring: update buffer descriptors in reverse order
- * to avoid race condition
- */
-
- /* Set 'TX_USED' bit in buffer descriptor at tx_head position
- * to set the end of TX queue
- */
- i = tx_head;
- entry = macb_tx_ring_wrap(bp, i);
- ctrl = MACB_BIT(TX_USED);
- desc = macb_tx_desc(queue, entry);
- desc->ctrl = ctrl;
-
- if (lso_ctrl) {
- if (lso_ctrl == MACB_LSO_UFO_ENABLE)
- /* include header and FCS in value given to h/w */
- mss_mfs = skb_shinfo(skb)->gso_size +
- skb_transport_offset(skb) +
- ETH_FCS_LEN;
- else /* TSO */ {
- mss_mfs = skb_shinfo(skb)->gso_size;
- /* TCP Sequence Number Source Select
- * can be set only for TSO
- */
- seq_ctrl = 0;
- }
- }
-
- do {
- i--;
- entry = macb_tx_ring_wrap(bp, i);
- tx_skb = &queue->tx_skb[entry];
- desc = macb_tx_desc(queue, entry);
-
- ctrl = (u32)tx_skb->size;
- if (eof) {
- ctrl |= MACB_BIT(TX_LAST);
- eof = 0;
- }
- if (unlikely(entry == (bp->tx_ring_size - 1)))
- ctrl |= MACB_BIT(TX_WRAP);
-
- /* First descriptor is header descriptor */
- if (i == queue->tx_head) {
- ctrl |= MACB_BF(TX_LSO, lso_ctrl);
- ctrl |= MACB_BF(TX_TCP_SEQ_SRC, seq_ctrl);
- } else
- /* Only set MSS/MFS on payload descriptors
- * (second or later descriptor)
- */
- ctrl |= MACB_BF(MSS_MFS, mss_mfs);
-
- /* Set TX buffer descriptor */
- macb_set_addr(bp, desc, tx_skb->mapping);
- /* desc->addr must be visible to hardware before clearing
- * 'TX_USED' bit in desc->ctrl.
- */
- wmb();
- desc->ctrl = ctrl;
- } while (i != queue->tx_head);
-
- queue->tx_head = tx_head;
-
- return count;
-
-dma_error:
- netdev_err(bp->dev, "TX DMA map failed\n");
-
- for (i = queue->tx_head; i != tx_head; i++) {
- tx_skb = macb_tx_skb(queue, i);
-
- macb_tx_unmap(bp, tx_skb);
- }
-
- return 0;
-}
-
-static netdev_features_t macb_features_check(struct sk_buff *skb,
- struct net_device *dev,
- netdev_features_t features)
-{
- unsigned int nr_frags, f;
- unsigned int hdrlen;
-
- /* Validate LSO compatibility */
-
- /* there is only one buffer */
- if (!skb_is_nonlinear(skb))
- return features;
-
- /* length of header */
- hdrlen = skb_transport_offset(skb);
- if (ip_hdr(skb)->protocol == IPPROTO_TCP)
- hdrlen += tcp_hdrlen(skb);
-
- /* For LSO:
- * When software supplies two or more payload buffers all payload buffers
- * apart from the last must be a multiple of 8 bytes in size.
- */
- if (!IS_ALIGNED(skb_headlen(skb) - hdrlen, MACB_TX_LEN_ALIGN))
- return features & ~MACB_NETIF_LSO;
-
- nr_frags = skb_shinfo(skb)->nr_frags;
- /* No need to check last fragment */
- nr_frags--;
- for (f = 0; f < nr_frags; f++) {
- const skb_frag_t *frag = &skb_shinfo(skb)->frags[f];
-
- if (!IS_ALIGNED(skb_frag_size(frag), MACB_TX_LEN_ALIGN))
- return features & ~MACB_NETIF_LSO;
- }
- return features;
-}
-
-static inline int macb_clear_csum(struct sk_buff *skb)
-{
- /* no change for packets without checksum offloading */
- if (skb->ip_summed != CHECKSUM_PARTIAL)
- return 0;
-
- /* make sure we can modify the header */
- if (unlikely(skb_cow_head(skb, 0)))
- return -1;
-
- /* initialize checksum field
- * This is required - at least for Zynq, which otherwise calculates
- * wrong UDP header checksums for UDP packets with UDP data len <=2
- */
- *(__sum16 *)(skb_checksum_start(skb) + skb->csum_offset) = 0;
- return 0;
-}
-
-static int macb_start_xmit(struct sk_buff *skb, struct net_device *dev)
-{
- u16 queue_index = skb_get_queue_mapping(skb);
- struct macb *bp = netdev_priv(dev);
- struct macb_queue *queue = &bp->queues[queue_index];
- unsigned long flags;
- unsigned int desc_cnt, nr_frags, frag_size, f;
- unsigned int hdrlen;
- bool is_lso, is_udp = 0;
-
- is_lso = (skb_shinfo(skb)->gso_size != 0);
-
- if (is_lso) {
- is_udp = !!(ip_hdr(skb)->protocol == IPPROTO_UDP);
-
- /* length of headers */
- if (is_udp)
- /* only queue eth + ip headers separately for UDP */
- hdrlen = skb_transport_offset(skb);
- else
- hdrlen = skb_transport_offset(skb) + tcp_hdrlen(skb);
- if (skb_headlen(skb) < hdrlen) {
- netdev_err(bp->dev, "Error - LSO headers fragmented!!!\n");
- /* if this is required, would need to copy to single buffer */
- return NETDEV_TX_BUSY;
- }
- } else
- hdrlen = min(skb_headlen(skb), bp->max_tx_length);
-
-#if defined(DEBUG) && defined(VERBOSE_DEBUG)
- netdev_vdbg(bp->dev,
- "start_xmit: queue %hu len %u head %p data %p tail %p end %p\n",
- queue_index, skb->len, skb->head, skb->data,
- skb_tail_pointer(skb), skb_end_pointer(skb));
- print_hex_dump(KERN_DEBUG, "data: ", DUMP_PREFIX_OFFSET, 16, 1,
- skb->data, 16, true);
-#endif
-
- /* Count how many TX buffer descriptors are needed to send this
- * socket buffer: skb fragments of jumbo frames may need to be
- * split into many buffer descriptors.
- */
- if (is_lso && (skb_headlen(skb) > hdrlen))
- /* extra header descriptor if also payload in first buffer */
- desc_cnt = DIV_ROUND_UP((skb_headlen(skb) - hdrlen), bp->max_tx_length) + 1;
- else
- desc_cnt = DIV_ROUND_UP(skb_headlen(skb), bp->max_tx_length);
- nr_frags = skb_shinfo(skb)->nr_frags;
- for (f = 0; f < nr_frags; f++) {
- frag_size = skb_frag_size(&skb_shinfo(skb)->frags[f]);
- desc_cnt += DIV_ROUND_UP(frag_size, bp->max_tx_length);
- }
-
- spin_lock_irqsave(&bp->lock, flags);
-
- /* This is a hard error, log it. */
- if (CIRC_SPACE(queue->tx_head, queue->tx_tail,
- bp->tx_ring_size) < desc_cnt) {
- netif_stop_subqueue(dev, queue_index);
- spin_unlock_irqrestore(&bp->lock, flags);
- netdev_dbg(bp->dev, "tx_head = %u, tx_tail = %u\n",
- queue->tx_head, queue->tx_tail);
- return NETDEV_TX_BUSY;
- }
-
- if (macb_clear_csum(skb)) {
- dev_kfree_skb_any(skb);
- goto unlock;
- }
-
- /* Map socket buffer for DMA transfer */
- if (!macb_tx_map(bp, queue, skb, hdrlen)) {
- dev_kfree_skb_any(skb);
- goto unlock;
- }
-
- /* Make newly initialized descriptor visible to hardware */
- wmb();
-
- skb_tx_timestamp(skb);
-
- macb_writel(bp, NCR, macb_readl(bp, NCR) | MACB_BIT(TSTART));
-
- if (CIRC_SPACE(queue->tx_head, queue->tx_tail, bp->tx_ring_size) < 1)
- netif_stop_subqueue(dev, queue_index);
-
-unlock:
- spin_unlock_irqrestore(&bp->lock, flags);
-
- return NETDEV_TX_OK;
-}
-
-static void macb_init_rx_buffer_size(struct macb *bp, size_t size)
-{
- if (!macb_is_gem(bp)) {
- bp->rx_buffer_size = MACB_RX_BUFFER_SIZE;
- } else {
- bp->rx_buffer_size = size;
-
- if (bp->rx_buffer_size % RX_BUFFER_MULTIPLE) {
- netdev_dbg(bp->dev,
- "RX buffer must be multiple of %d bytes, expanding\n",
- RX_BUFFER_MULTIPLE);
- bp->rx_buffer_size =
- roundup(bp->rx_buffer_size, RX_BUFFER_MULTIPLE);
- }
- }
-
- netdev_dbg(bp->dev, "mtu [%u] rx_buffer_size [%zu]\n",
- bp->dev->mtu, bp->rx_buffer_size);
-}
-
-static void gem_free_rx_buffers(struct macb *bp)
-{
- struct sk_buff *skb;
- struct macb_dma_desc *desc;
- dma_addr_t addr;
- int i;
-
- if (!bp->rx_skbuff)
- return;
-
- for (i = 0; i < bp->rx_ring_size; i++) {
- skb = bp->rx_skbuff[i];
-
- if (!skb)
- continue;
-
- desc = macb_rx_desc(bp, i);
- addr = macb_get_addr(bp, desc);
-
- dma_unmap_single(&bp->pdev->dev, addr, bp->rx_buffer_size,
- DMA_FROM_DEVICE);
- dev_kfree_skb_any(skb);
- skb = NULL;
- }
-
- kfree(bp->rx_skbuff);
- bp->rx_skbuff = NULL;
-}
-
-static void macb_free_rx_buffers(struct macb *bp)
-{
- if (bp->rx_buffers) {
- dma_free_coherent(&bp->pdev->dev,
- bp->rx_ring_size * bp->rx_buffer_size,
- bp->rx_buffers, bp->rx_buffers_dma);
- bp->rx_buffers = NULL;
- }
-}
-
-static void macb_free_consistent(struct macb *bp)
-{
- struct macb_queue *queue;
- unsigned int q;
-
- bp->macbgem_ops.mog_free_rx_buffers(bp);
- if (bp->rx_ring) {
- dma_free_coherent(&bp->pdev->dev, RX_RING_BYTES(bp),
- bp->rx_ring, bp->rx_ring_dma);
- bp->rx_ring = NULL;
- }
-
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
- kfree(queue->tx_skb);
- queue->tx_skb = NULL;
- if (queue->tx_ring) {
- dma_free_coherent(&bp->pdev->dev, TX_RING_BYTES(bp),
- queue->tx_ring, queue->tx_ring_dma);
- queue->tx_ring = NULL;
- }
- }
-}
-
-static int gem_alloc_rx_buffers(struct macb *bp)
-{
- int size;
-
- size = bp->rx_ring_size * sizeof(struct sk_buff *);
- bp->rx_skbuff = kzalloc(size, GFP_KERNEL);
- if (!bp->rx_skbuff)
- return -ENOMEM;
- else
- netdev_dbg(bp->dev,
- "Allocated %d RX struct sk_buff entries at %p\n",
- bp->rx_ring_size, bp->rx_skbuff);
- return 0;
-}
-
-static int macb_alloc_rx_buffers(struct macb *bp)
-{
- int size;
-
- size = bp->rx_ring_size * bp->rx_buffer_size;
- bp->rx_buffers = dma_alloc_coherent(&bp->pdev->dev, size,
- &bp->rx_buffers_dma, GFP_KERNEL);
- if (!bp->rx_buffers)
- return -ENOMEM;
-
- netdev_dbg(bp->dev,
- "Allocated RX buffers of %d bytes at %08lx (mapped %p)\n",
- size, (unsigned long)bp->rx_buffers_dma, bp->rx_buffers);
- return 0;
-}
-
-static int macb_alloc_consistent(struct macb *bp)
-{
- struct macb_queue *queue;
- unsigned int q;
- int size;
-
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
- size = TX_RING_BYTES(bp);
- queue->tx_ring = dma_alloc_coherent(&bp->pdev->dev, size,
- &queue->tx_ring_dma,
- GFP_KERNEL);
- if (!queue->tx_ring)
- goto out_err;
- netdev_dbg(bp->dev,
- "Allocated TX ring for queue %u of %d bytes at %08lx (mapped %p)\n",
- q, size, (unsigned long)queue->tx_ring_dma,
- queue->tx_ring);
-
- size = bp->tx_ring_size * sizeof(struct macb_tx_skb);
- queue->tx_skb = kmalloc(size, GFP_KERNEL);
- if (!queue->tx_skb)
- goto out_err;
- }
-
- size = RX_RING_BYTES(bp);
- bp->rx_ring = dma_alloc_coherent(&bp->pdev->dev, size,
- &bp->rx_ring_dma, GFP_KERNEL);
- if (!bp->rx_ring)
- goto out_err;
- netdev_dbg(bp->dev,
- "Allocated RX ring of %d bytes at %08lx (mapped %p)\n",
- size, (unsigned long)bp->rx_ring_dma, bp->rx_ring);
-
- if (bp->macbgem_ops.mog_alloc_rx_buffers(bp))
- goto out_err;
-
- return 0;
-
-out_err:
- macb_free_consistent(bp);
- return -ENOMEM;
-}
-
-static void gem_init_rings(struct macb *bp)
-{
- struct macb_queue *queue;
- struct macb_dma_desc *desc = NULL;
- unsigned int q;
- int i;
-
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
- for (i = 0; i < bp->tx_ring_size; i++) {
- desc = macb_tx_desc(queue, i);
- macb_set_addr(bp, desc, 0);
- desc->ctrl = MACB_BIT(TX_USED);
- }
- desc->ctrl |= MACB_BIT(TX_WRAP);
- queue->tx_head = 0;
- queue->tx_tail = 0;
- }
-
- bp->rx_tail = 0;
- bp->rx_prepared_head = 0;
-
- gem_rx_refill(bp);
-}
-
-static void macb_init_rings(struct macb *bp)
-{
- int i;
- struct macb_dma_desc *desc = NULL;
-
- macb_init_rx_ring(bp);
-
- for (i = 0; i < bp->tx_ring_size; i++) {
- desc = macb_tx_desc(&bp->queues[0], i);
- macb_set_addr(bp, desc, 0);
- desc->ctrl = MACB_BIT(TX_USED);
- }
- bp->queues[0].tx_head = 0;
- bp->queues[0].tx_tail = 0;
- desc->ctrl |= MACB_BIT(TX_WRAP);
-}
-
-static void macb_reset_hw(struct macb *bp)
-{
- struct macb_queue *queue;
- unsigned int q;
-
- /* Disable RX and TX (XXX: Should we halt the transmission
- * more gracefully?)
- */
- macb_writel(bp, NCR, 0);
-
- /* Clear the stats registers (XXX: Update stats first?) */
- macb_writel(bp, NCR, MACB_BIT(CLRSTAT));
-
- /* Clear all status flags */
- macb_writel(bp, TSR, -1);
- macb_writel(bp, RSR, -1);
-
- /* Disable all interrupts */
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
- queue_writel(queue, IDR, -1);
- queue_readl(queue, ISR);
- if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
- queue_writel(queue, ISR, -1);
- }
-}
-
-static u32 gem_mdc_clk_div(struct macb *bp)
-{
- u32 config;
- unsigned long pclk_hz = clk_get_rate(bp->pclk);
-
- if (pclk_hz <= 20000000)
- config = GEM_BF(CLK, GEM_CLK_DIV8);
- else if (pclk_hz <= 40000000)
- config = GEM_BF(CLK, GEM_CLK_DIV16);
- else if (pclk_hz <= 80000000)
- config = GEM_BF(CLK, GEM_CLK_DIV32);
- else if (pclk_hz <= 120000000)
- config = GEM_BF(CLK, GEM_CLK_DIV48);
- else if (pclk_hz <= 160000000)
- config = GEM_BF(CLK, GEM_CLK_DIV64);
- else
- config = GEM_BF(CLK, GEM_CLK_DIV96);
-
- return config;
-}
-
-static u32 macb_mdc_clk_div(struct macb *bp)
-{
- u32 config;
- unsigned long pclk_hz;
-
- if (macb_is_gem(bp))
- return gem_mdc_clk_div(bp);
-
- pclk_hz = clk_get_rate(bp->pclk);
- if (pclk_hz <= 20000000)
- config = MACB_BF(CLK, MACB_CLK_DIV8);
- else if (pclk_hz <= 40000000)
- config = MACB_BF(CLK, MACB_CLK_DIV16);
- else if (pclk_hz <= 80000000)
- config = MACB_BF(CLK, MACB_CLK_DIV32);
- else
- config = MACB_BF(CLK, MACB_CLK_DIV64);
-
- return config;
-}
-
-/* Get the DMA bus width field of the network configuration register that we
- * should program. We find the width from decoding the design configuration
- * register to find the maximum supported data bus width.
- */
-static u32 macb_dbw(struct macb *bp)
-{
- if (!macb_is_gem(bp))
- return 0;
-
- switch (GEM_BFEXT(DBWDEF, gem_readl(bp, DCFG1))) {
- case 4:
- return GEM_BF(DBW, GEM_DBW128);
- case 2:
- return GEM_BF(DBW, GEM_DBW64);
- case 1:
- default:
- return GEM_BF(DBW, GEM_DBW32);
- }
-}
-
-/* Configure the receive DMA engine
- * - use the correct receive buffer size
- * - set best burst length for DMA operations
- * (if not supported by FIFO, it will fallback to default)
- * - set both rx/tx packet buffers to full memory size
- * These are configurable parameters for GEM.
- */
-static void macb_configure_dma(struct macb *bp)
-{
- u32 dmacfg;
-
- if (macb_is_gem(bp)) {
- dmacfg = gem_readl(bp, DMACFG) & ~GEM_BF(RXBS, -1L);
- dmacfg |= GEM_BF(RXBS, bp->rx_buffer_size / RX_BUFFER_MULTIPLE);
- if (bp->dma_burst_length)
- dmacfg = GEM_BFINS(FBLDO, bp->dma_burst_length, dmacfg);
- dmacfg |= GEM_BIT(TXPBMS) | GEM_BF(RXBMS, -1L);
- dmacfg &= ~GEM_BIT(ENDIA_PKT);
-
- if (bp->native_io)
- dmacfg &= ~GEM_BIT(ENDIA_DESC);
- else
- dmacfg |= GEM_BIT(ENDIA_DESC); /* CPU in big endian */
-
- if (bp->dev->features & NETIF_F_HW_CSUM)
- dmacfg |= GEM_BIT(TXCOEN);
- else
- dmacfg &= ~GEM_BIT(TXCOEN);
-
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (bp->hw_dma_cap & HW_DMA_CAP_64B)
- dmacfg |= GEM_BIT(ADDR64);
-#endif
-#ifdef CONFIG_MACB_USE_HWSTAMP
- if (bp->hw_dma_cap & HW_DMA_CAP_PTP)
- dmacfg |= GEM_BIT(RXEXT) | GEM_BIT(TXEXT);
-#endif
- netdev_dbg(bp->dev, "Cadence configure DMA with 0x%08x\n",
- dmacfg);
- gem_writel(bp, DMACFG, dmacfg);
- }
-}
-
-static void macb_init_hw(struct macb *bp)
-{
- struct macb_queue *queue;
- unsigned int q;
-
- u32 config;
-
- macb_reset_hw(bp);
- macb_set_hwaddr(bp);
-
- config = macb_mdc_clk_div(bp);
- if (bp->phy_interface == PHY_INTERFACE_MODE_SGMII)
- config |= GEM_BIT(SGMIIEN) | GEM_BIT(PCSSEL);
- config |= MACB_BF(RBOF, NET_IP_ALIGN); /* Make eth data aligned */
- config |= MACB_BIT(PAE); /* PAuse Enable */
- config |= MACB_BIT(DRFCS); /* Discard Rx FCS */
- if (bp->caps & MACB_CAPS_JUMBO)
- config |= MACB_BIT(JFRAME); /* Enable jumbo frames */
- else
- config |= MACB_BIT(BIG); /* Receive oversized frames */
- if (bp->dev->flags & IFF_PROMISC)
- config |= MACB_BIT(CAF); /* Copy All Frames */
- else if (macb_is_gem(bp) && bp->dev->features & NETIF_F_RXCSUM)
- config |= GEM_BIT(RXCOEN);
- if (!(bp->dev->flags & IFF_BROADCAST))
- config |= MACB_BIT(NBC); /* No BroadCast */
- config |= macb_dbw(bp);
- macb_writel(bp, NCFGR, config);
- if ((bp->caps & MACB_CAPS_JUMBO) && bp->jumbo_max_len)
- gem_writel(bp, JML, bp->jumbo_max_len);
- bp->speed = SPEED_10;
- bp->duplex = DUPLEX_HALF;
- bp->rx_frm_len_mask = MACB_RX_FRMLEN_MASK;
- if (bp->caps & MACB_CAPS_JUMBO)
- bp->rx_frm_len_mask = MACB_RX_JFRMLEN_MASK;
-
- macb_configure_dma(bp);
-
- /* Initialize TX and RX buffers */
- macb_writel(bp, RBQP, lower_32_bits(bp->rx_ring_dma));
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (bp->hw_dma_cap & HW_DMA_CAP_64B)
- macb_writel(bp, RBQPH, upper_32_bits(bp->rx_ring_dma));
-#endif
- for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
- queue_writel(queue, TBQP, lower_32_bits(queue->tx_ring_dma));
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (bp->hw_dma_cap & HW_DMA_CAP_64B)
- queue_writel(queue, TBQPH, upper_32_bits(queue->tx_ring_dma));
-#endif
-
- /* Enable interrupts */
- queue_writel(queue, IER,
- MACB_RX_INT_FLAGS |
- MACB_TX_INT_FLAGS |
- MACB_BIT(HRESP));
- }
-
- /* Enable TX and RX */
- macb_writel(bp, NCR, MACB_BIT(RE) | MACB_BIT(TE) | MACB_BIT(MPE));
-}
-
-/* The hash address register is 64 bits long and takes up two
- * locations in the memory map. The least significant bits are stored
- * in EMAC_HSL and the most significant bits in EMAC_HSH.
- *
- * The unicast hash enable and the multicast hash enable bits in the
- * network configuration register enable the reception of hash matched
- * frames. The destination address is reduced to a 6 bit index into
- * the 64 bit hash register using the following hash function. The
- * hash function is an exclusive or of every sixth bit of the
- * destination address.
- *
- * hi[5] = da[5] ^ da[11] ^ da[17] ^ da[23] ^ da[29] ^ da[35] ^ da[41] ^ da[47]
- * hi[4] = da[4] ^ da[10] ^ da[16] ^ da[22] ^ da[28] ^ da[34] ^ da[40] ^ da[46]
- * hi[3] = da[3] ^ da[09] ^ da[15] ^ da[21] ^ da[27] ^ da[33] ^ da[39] ^ da[45]
- * hi[2] = da[2] ^ da[08] ^ da[14] ^ da[20] ^ da[26] ^ da[32] ^ da[38] ^ da[44]
- * hi[1] = da[1] ^ da[07] ^ da[13] ^ da[19] ^ da[25] ^ da[31] ^ da[37] ^ da[43]
- * hi[0] = da[0] ^ da[06] ^ da[12] ^ da[18] ^ da[24] ^ da[30] ^ da[36] ^ da[42]
- *
- * da[0] represents the least significant bit of the first byte
- * received, that is, the multicast/unicast indicator, and da[47]
- * represents the most significant bit of the last byte received. If
- * the hash index, hi[n], points to a bit that is set in the hash
- * register then the frame will be matched according to whether the
- * frame is multicast or unicast. A multicast match will be signalled
- * if the multicast hash enable bit is set, da[0] is 1 and the hash
- * index points to a bit set in the hash register. A unicast match
- * will be signalled if the unicast hash enable bit is set, da[0] is 0
- * and the hash index points to a bit set in the hash register. To
- * receive all multicast frames, the hash register should be set with
- * all ones and the multicast hash enable bit should be set in the
- * network configuration register.
- */
-
-static inline int hash_bit_value(int bitnr, __u8 *addr)
-{
- if (addr[bitnr / 8] & (1 << (bitnr % 8)))
- return 1;
- return 0;
-}
-
-/* Return the hash index value for the specified address. */
-static int hash_get_index(__u8 *addr)
-{
- int i, j, bitval;
- int hash_index = 0;
-
- for (j = 0; j < 6; j++) {
- for (i = 0, bitval = 0; i < 8; i++)
- bitval ^= hash_bit_value(i * 6 + j, addr);
-
- hash_index |= (bitval << j);
- }
-
- return hash_index;
-}
-
-/* Add multicast addresses to the internal multicast-hash table. */
-static void macb_sethashtable(struct net_device *dev)
-{
- struct netdev_hw_addr *ha;
- unsigned long mc_filter[2];
- unsigned int bitnr;
- struct macb *bp = netdev_priv(dev);
-
- mc_filter[0] = 0;
- mc_filter[1] = 0;
-
- netdev_for_each_mc_addr(ha, dev) {
- bitnr = hash_get_index(ha->addr);
- mc_filter[bitnr >> 5] |= 1 << (bitnr & 31);
- }
-
- macb_or_gem_writel(bp, HRB, mc_filter[0]);
- macb_or_gem_writel(bp, HRT, mc_filter[1]);
-}
-
-/* Enable/Disable promiscuous and multicast modes. */
-static void macb_set_rx_mode(struct net_device *dev)
-{
- unsigned long cfg;
- struct macb *bp = netdev_priv(dev);
-
- cfg = macb_readl(bp, NCFGR);
-
- if (dev->flags & IFF_PROMISC) {
- /* Enable promiscuous mode */
- cfg |= MACB_BIT(CAF);
-
- /* Disable RX checksum offload */
- if (macb_is_gem(bp))
- cfg &= ~GEM_BIT(RXCOEN);
- } else {
- /* Disable promiscuous mode */
- cfg &= ~MACB_BIT(CAF);
-
- /* Enable RX checksum offload only if requested */
- if (macb_is_gem(bp) && dev->features & NETIF_F_RXCSUM)
- cfg |= GEM_BIT(RXCOEN);
- }
-
- if (dev->flags & IFF_ALLMULTI) {
- /* Enable all multicast mode */
- macb_or_gem_writel(bp, HRB, -1);
- macb_or_gem_writel(bp, HRT, -1);
- cfg |= MACB_BIT(NCFGR_MTI);
- } else if (!netdev_mc_empty(dev)) {
- /* Enable specific multicasts */
- macb_sethashtable(dev);
- cfg |= MACB_BIT(NCFGR_MTI);
- } else if (dev->flags & (~IFF_ALLMULTI)) {
- /* Disable all multicast mode */
- macb_or_gem_writel(bp, HRB, 0);
- macb_or_gem_writel(bp, HRT, 0);
- cfg &= ~MACB_BIT(NCFGR_MTI);
- }
-
- macb_writel(bp, NCFGR, cfg);
-}
-
-static int macb_open(struct net_device *dev)
-{
- struct macb *bp = netdev_priv(dev);
- size_t bufsz = dev->mtu + ETH_HLEN + ETH_FCS_LEN + NET_IP_ALIGN;
- int err;
-
- netdev_dbg(bp->dev, "open\n");
-
- /* carrier starts down */
- netif_carrier_off(dev);
-
- /* if the phy is not yet register, retry later*/
- if (!dev->phydev)
- return -EAGAIN;
-
- /* RX buffers initialization */
- macb_init_rx_buffer_size(bp, bufsz);
-
- err = macb_alloc_consistent(bp);
- if (err) {
- netdev_err(dev, "Unable to allocate DMA memory (error %d)\n",
- err);
- return err;
- }
-
- napi_enable(&bp->napi);
-
- bp->macbgem_ops.mog_init_rings(bp);
- macb_init_hw(bp);
-
- /* schedule a link state check */
- phy_start(dev->phydev);
-
- netif_tx_start_all_queues(dev);
-
- if (bp->ptp_info)
- bp->ptp_info->ptp_init(dev);
-
- return 0;
-}
-
-static int macb_close(struct net_device *dev)
-{
- struct macb *bp = netdev_priv(dev);
- unsigned long flags;
-
- netif_tx_stop_all_queues(dev);
- napi_disable(&bp->napi);
-
- if (dev->phydev)
- phy_stop(dev->phydev);
-
- spin_lock_irqsave(&bp->lock, flags);
- macb_reset_hw(bp);
- netif_carrier_off(dev);
- spin_unlock_irqrestore(&bp->lock, flags);
-
- macb_free_consistent(bp);
-
- if (bp->ptp_info)
- bp->ptp_info->ptp_remove(dev);
-
- return 0;
-}
-
-static int macb_change_mtu(struct net_device *dev, int new_mtu)
-{
- if (netif_running(dev))
- return -EBUSY;
-
- dev->mtu = new_mtu;
-
- return 0;
-}
-
-static void gem_update_stats(struct macb *bp)
-{
- unsigned int i;
- u32 *p = &bp->hw_stats.gem.tx_octets_31_0;
-
- for (i = 0; i < GEM_STATS_LEN; ++i, ++p) {
- u32 offset = gem_statistics[i].offset;
- u64 val = bp->macb_reg_readl(bp, offset);
-
- bp->ethtool_stats[i] += val;
- *p += val;
-
- if (offset == GEM_OCTTXL || offset == GEM_OCTRXL) {
- /* Add GEM_OCTTXH, GEM_OCTRXH */
- val = bp->macb_reg_readl(bp, offset + 4);
- bp->ethtool_stats[i] += ((u64)val) << 32;
- *(++p) += val;
- }
- }
-}
-
-static struct net_device_stats *gem_get_stats(struct macb *bp)
-{
- struct gem_stats *hwstat = &bp->hw_stats.gem;
- struct net_device_stats *nstat = &bp->dev->stats;
-
- gem_update_stats(bp);
-
- nstat->rx_errors = (hwstat->rx_frame_check_sequence_errors +
- hwstat->rx_alignment_errors +
- hwstat->rx_resource_errors +
- hwstat->rx_overruns +
- hwstat->rx_oversize_frames +
- hwstat->rx_jabbers +
- hwstat->rx_undersized_frames +
- hwstat->rx_length_field_frame_errors);
- nstat->tx_errors = (hwstat->tx_late_collisions +
- hwstat->tx_excessive_collisions +
- hwstat->tx_underrun +
- hwstat->tx_carrier_sense_errors);
- nstat->multicast = hwstat->rx_multicast_frames;
- nstat->collisions = (hwstat->tx_single_collision_frames +
- hwstat->tx_multiple_collision_frames +
- hwstat->tx_excessive_collisions);
- nstat->rx_length_errors = (hwstat->rx_oversize_frames +
- hwstat->rx_jabbers +
- hwstat->rx_undersized_frames +
- hwstat->rx_length_field_frame_errors);
- nstat->rx_over_errors = hwstat->rx_resource_errors;
- nstat->rx_crc_errors = hwstat->rx_frame_check_sequence_errors;
- nstat->rx_frame_errors = hwstat->rx_alignment_errors;
- nstat->rx_fifo_errors = hwstat->rx_overruns;
- nstat->tx_aborted_errors = hwstat->tx_excessive_collisions;
- nstat->tx_carrier_errors = hwstat->tx_carrier_sense_errors;
- nstat->tx_fifo_errors = hwstat->tx_underrun;
-
- return nstat;
-}
-
-static void gem_get_ethtool_stats(struct net_device *dev,
- struct ethtool_stats *stats, u64 *data)
-{
- struct macb *bp;
-
- bp = netdev_priv(dev);
- gem_update_stats(bp);
- memcpy(data, &bp->ethtool_stats, sizeof(u64) * GEM_STATS_LEN);
-}
-
-static int gem_get_sset_count(struct net_device *dev, int sset)
-{
- switch (sset) {
- case ETH_SS_STATS:
- return GEM_STATS_LEN;
- default:
- return -EOPNOTSUPP;
- }
-}
-
-static void gem_get_ethtool_strings(struct net_device *dev, u32 sset, u8 *p)
-{
- unsigned int i;
-
- switch (sset) {
- case ETH_SS_STATS:
- for (i = 0; i < GEM_STATS_LEN; i++, p += ETH_GSTRING_LEN)
- memcpy(p, gem_statistics[i].stat_string,
- ETH_GSTRING_LEN);
- break;
- }
-}
-
-static struct net_device_stats *macb_get_stats(struct net_device *dev)
-{
- struct macb *bp = netdev_priv(dev);
- struct net_device_stats *nstat = &bp->dev->stats;
- struct macb_stats *hwstat = &bp->hw_stats.macb;
-
- if (macb_is_gem(bp))
- return gem_get_stats(bp);
-
- /* read stats from hardware */
- macb_update_stats(bp);
-
- /* Convert HW stats into netdevice stats */
- nstat->rx_errors = (hwstat->rx_fcs_errors +
- hwstat->rx_align_errors +
- hwstat->rx_resource_errors +
- hwstat->rx_overruns +
- hwstat->rx_oversize_pkts +
- hwstat->rx_jabbers +
- hwstat->rx_undersize_pkts +
- hwstat->rx_length_mismatch);
- nstat->tx_errors = (hwstat->tx_late_cols +
- hwstat->tx_excessive_cols +
- hwstat->tx_underruns +
- hwstat->tx_carrier_errors +
- hwstat->sqe_test_errors);
- nstat->collisions = (hwstat->tx_single_cols +
- hwstat->tx_multiple_cols +
- hwstat->tx_excessive_cols);
- nstat->rx_length_errors = (hwstat->rx_oversize_pkts +
- hwstat->rx_jabbers +
- hwstat->rx_undersize_pkts +
- hwstat->rx_length_mismatch);
- nstat->rx_over_errors = hwstat->rx_resource_errors +
- hwstat->rx_overruns;
- nstat->rx_crc_errors = hwstat->rx_fcs_errors;
- nstat->rx_frame_errors = hwstat->rx_align_errors;
- nstat->rx_fifo_errors = hwstat->rx_overruns;
- /* XXX: What does "missed" mean? */
- nstat->tx_aborted_errors = hwstat->tx_excessive_cols;
- nstat->tx_carrier_errors = hwstat->tx_carrier_errors;
- nstat->tx_fifo_errors = hwstat->tx_underruns;
- /* Don't know about heartbeat or window errors... */
-
- return nstat;
-}
-
-static int macb_get_regs_len(struct net_device *netdev)
-{
- return MACB_GREGS_NBR * sizeof(u32);
-}
-
-static void macb_get_regs(struct net_device *dev, struct ethtool_regs *regs,
- void *p)
-{
- struct macb *bp = netdev_priv(dev);
- unsigned int tail, head;
- u32 *regs_buff = p;
-
- regs->version = (macb_readl(bp, MID) & ((1 << MACB_REV_SIZE) - 1))
- | MACB_GREGS_VERSION;
-
- tail = macb_tx_ring_wrap(bp, bp->queues[0].tx_tail);
- head = macb_tx_ring_wrap(bp, bp->queues[0].tx_head);
-
- regs_buff[0] = macb_readl(bp, NCR);
- regs_buff[1] = macb_or_gem_readl(bp, NCFGR);
- regs_buff[2] = macb_readl(bp, NSR);
- regs_buff[3] = macb_readl(bp, TSR);
- regs_buff[4] = macb_readl(bp, RBQP);
- regs_buff[5] = macb_readl(bp, TBQP);
- regs_buff[6] = macb_readl(bp, RSR);
- regs_buff[7] = macb_readl(bp, IMR);
-
- regs_buff[8] = tail;
- regs_buff[9] = head;
- regs_buff[10] = macb_tx_dma(&bp->queues[0], tail);
- regs_buff[11] = macb_tx_dma(&bp->queues[0], head);
-
- if (!(bp->caps & MACB_CAPS_USRIO_DISABLED))
- regs_buff[12] = macb_or_gem_readl(bp, USRIO);
- if (macb_is_gem(bp))
- regs_buff[13] = gem_readl(bp, DMACFG);
-}
-
-static void macb_get_wol(struct net_device *netdev, struct ethtool_wolinfo *wol)
-{
- struct macb *bp = netdev_priv(netdev);
-
- wol->supported = 0;
- wol->wolopts = 0;
-
- if (bp->wol & MACB_WOL_HAS_MAGIC_PACKET) {
- wol->supported = WAKE_MAGIC;
-
- if (bp->wol & MACB_WOL_ENABLED)
- wol->wolopts |= WAKE_MAGIC;
- }
-}
-
-static int macb_set_wol(struct net_device *netdev, struct ethtool_wolinfo *wol)
-{
- struct macb *bp = netdev_priv(netdev);
-
- if (!(bp->wol & MACB_WOL_HAS_MAGIC_PACKET) ||
- (wol->wolopts & ~WAKE_MAGIC))
- return -EOPNOTSUPP;
-
- if (wol->wolopts & WAKE_MAGIC)
- bp->wol |= MACB_WOL_ENABLED;
- else
- bp->wol &= ~MACB_WOL_ENABLED;
-
- device_set_wakeup_enable(&bp->pdev->dev, bp->wol & MACB_WOL_ENABLED);
-
- return 0;
-}
-
-static void macb_get_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
-{
- struct macb *bp = netdev_priv(netdev);
-
- ring->rx_max_pending = MAX_RX_RING_SIZE;
- ring->tx_max_pending = MAX_TX_RING_SIZE;
-
- ring->rx_pending = bp->rx_ring_size;
- ring->tx_pending = bp->tx_ring_size;
-}
-
-static int macb_set_ringparam(struct net_device *netdev,
- struct ethtool_ringparam *ring)
-{
- struct macb *bp = netdev_priv(netdev);
- u32 new_rx_size, new_tx_size;
- unsigned int reset = 0;
-
- if ((ring->rx_mini_pending) || (ring->rx_jumbo_pending))
- return -EINVAL;
-
- new_rx_size = clamp_t(u32, ring->rx_pending,
- MIN_RX_RING_SIZE, MAX_RX_RING_SIZE);
- new_rx_size = roundup_pow_of_two(new_rx_size);
-
- new_tx_size = clamp_t(u32, ring->tx_pending,
- MIN_TX_RING_SIZE, MAX_TX_RING_SIZE);
- new_tx_size = roundup_pow_of_two(new_tx_size);
-
- if ((new_tx_size == bp->tx_ring_size) &&
- (new_rx_size == bp->rx_ring_size)) {
- /* nothing to do */
- return 0;
- }
-
- if (netif_running(bp->dev)) {
- reset = 1;
- macb_close(bp->dev);
- }
-
- bp->rx_ring_size = new_rx_size;
- bp->tx_ring_size = new_tx_size;
-
- if (reset)
- macb_open(bp->dev);
-
- return 0;
-}
-
-static int macb_get_ts_info(struct net_device *netdev,
- struct ethtool_ts_info *info)
-{
- struct macb *bp = netdev_priv(netdev);
-
- if (bp->ptp_info)
- return bp->ptp_info->get_ts_info(netdev, info);
-
- return ethtool_op_get_ts_info(netdev, info);
-}
-
-static const struct ethtool_ops macb_ethtool_ops = {
- .get_regs_len = macb_get_regs_len,
- .get_regs = macb_get_regs,
- .get_link = ethtool_op_get_link,
- .get_ts_info = ethtool_op_get_ts_info,
- .get_wol = macb_get_wol,
- .set_wol = macb_set_wol,
- .get_link_ksettings = phy_ethtool_get_link_ksettings,
- .set_link_ksettings = phy_ethtool_set_link_ksettings,
- .get_ringparam = macb_get_ringparam,
- .set_ringparam = macb_set_ringparam,
-};
-
-static const struct ethtool_ops gem_ethtool_ops = {
- .get_regs_len = macb_get_regs_len,
- .get_regs = macb_get_regs,
- .get_link = ethtool_op_get_link,
- .get_ts_info = macb_get_ts_info,
- .get_ethtool_stats = gem_get_ethtool_stats,
- .get_strings = gem_get_ethtool_strings,
- .get_sset_count = gem_get_sset_count,
- .get_link_ksettings = phy_ethtool_get_link_ksettings,
- .set_link_ksettings = phy_ethtool_set_link_ksettings,
- .get_ringparam = macb_get_ringparam,
- .set_ringparam = macb_set_ringparam,
-};
-
-static int macb_ioctl(struct net_device *dev, struct ifreq *rq, int cmd)
-{
- struct phy_device *phydev = dev->phydev;
- struct macb *bp = netdev_priv(dev);
-
- if (!netif_running(dev))
- return -EINVAL;
-
- if (!phydev)
- return -ENODEV;
-
- if (!bp->ptp_info)
- return phy_mii_ioctl(phydev, rq, cmd);
-
- switch (cmd) {
- case SIOCSHWTSTAMP:
- return bp->ptp_info->set_hwtst(dev, rq, cmd);
- case SIOCGHWTSTAMP:
- return bp->ptp_info->get_hwtst(dev, rq);
- default:
- return phy_mii_ioctl(phydev, rq, cmd);
- }
-}
-
-static int macb_set_features(struct net_device *netdev,
- netdev_features_t features)
-{
- struct macb *bp = netdev_priv(netdev);
- netdev_features_t changed = features ^ netdev->features;
-
- /* TX checksum offload */
- if ((changed & NETIF_F_HW_CSUM) && macb_is_gem(bp)) {
- u32 dmacfg;
-
- dmacfg = gem_readl(bp, DMACFG);
- if (features & NETIF_F_HW_CSUM)
- dmacfg |= GEM_BIT(TXCOEN);
- else
- dmacfg &= ~GEM_BIT(TXCOEN);
- gem_writel(bp, DMACFG, dmacfg);
- }
-
- /* RX checksum offload */
- if ((changed & NETIF_F_RXCSUM) && macb_is_gem(bp)) {
- u32 netcfg;
-
- netcfg = gem_readl(bp, NCFGR);
- if (features & NETIF_F_RXCSUM &&
- !(netdev->flags & IFF_PROMISC))
- netcfg |= GEM_BIT(RXCOEN);
- else
- netcfg &= ~GEM_BIT(RXCOEN);
- gem_writel(bp, NCFGR, netcfg);
- }
-
- return 0;
-}
-
-static const struct net_device_ops macb_netdev_ops = {
- .ndo_open = macb_open,
- .ndo_stop = macb_close,
- .ndo_start_xmit = macb_start_xmit,
- .ndo_set_rx_mode = macb_set_rx_mode,
- .ndo_get_stats = macb_get_stats,
- .ndo_do_ioctl = macb_ioctl,
- .ndo_validate_addr = eth_validate_addr,
- .ndo_change_mtu = macb_change_mtu,
- .ndo_set_mac_address = eth_mac_addr,
-#ifdef CONFIG_NET_POLL_CONTROLLER
- .ndo_poll_controller = macb_poll_controller,
-#endif
- .ndo_set_features = macb_set_features,
- .ndo_features_check = macb_features_check,
-};
-
-/* Configure peripheral capabilities according to device tree
- * and integration options used
- */
-static void macb_configure_caps(struct macb *bp,
- const struct macb_config *dt_conf)
-{
- u32 dcfg;
-
- if (dt_conf)
- bp->caps = dt_conf->caps;
-
- if (hw_is_gem(bp->regs, bp->native_io)) {
- bp->caps |= MACB_CAPS_MACB_IS_GEM;
-
- dcfg = gem_readl(bp, DCFG1);
- if (GEM_BFEXT(IRQCOR, dcfg) == 0)
- bp->caps |= MACB_CAPS_ISR_CLEAR_ON_WRITE;
- dcfg = gem_readl(bp, DCFG2);
- if ((dcfg & (GEM_BIT(RX_PKT_BUFF) | GEM_BIT(TX_PKT_BUFF))) == 0)
- bp->caps |= MACB_CAPS_FIFO_MODE;
- if (IS_ENABLED(CONFIG_MACB_USE_HWSTAMP) && gem_has_ptp(bp)) {
- if (!GEM_BFEXT(TSU, gem_readl(bp, DCFG5)))
- pr_err("GEM doesn't support hardware ptp.\n");
- else
- bp->hw_dma_cap |= HW_DMA_CAP_PTP;
- }
- }
-
- dev_dbg(&bp->pdev->dev, "Cadence caps 0x%08x\n", bp->caps);
-}
-
-static void macb_probe_queues(void __iomem *mem,
- bool native_io,
- unsigned int *queue_mask,
- unsigned int *num_queues)
-{
- unsigned int hw_q;
-
- *queue_mask = 0x1;
- *num_queues = 1;
-
- /* is it macb or gem ?
- *
- * We need to read directly from the hardware here because
- * we are early in the probe process and don't have the
- * MACB_CAPS_MACB_IS_GEM flag positioned
- */
- if (!hw_is_gem(mem, native_io))
- return;
-
- /* bit 0 is never set but queue 0 always exists */
- *queue_mask = readl_relaxed(mem + GEM_DCFG6) & 0xff;
-
- *queue_mask |= 0x1;
-
- for (hw_q = 1; hw_q < MACB_MAX_QUEUES; ++hw_q)
- if (*queue_mask & (1 << hw_q))
- (*num_queues)++;
-}
-
-static int macb_clk_init(struct platform_device *pdev, struct clk **pclk,
- struct clk **hclk, struct clk **tx_clk,
- struct clk **rx_clk)
-{
- struct macb_platform_data *pdata;
- int err;
-
- pdata = dev_get_platdata(&pdev->dev);
- if (pdata) {
- *pclk = pdata->pclk;
- *hclk = pdata->hclk;
- } else {
- *pclk = devm_clk_get(&pdev->dev, "pclk");
- *hclk = devm_clk_get(&pdev->dev, "hclk");
- }
-
- if (IS_ERR(*pclk)) {
- err = PTR_ERR(*pclk);
- dev_err(&pdev->dev, "failed to get macb_clk (%u)\n", err);
- return err;
- }
-
- if (IS_ERR(*hclk)) {
- err = PTR_ERR(*hclk);
- dev_err(&pdev->dev, "failed to get hclk (%u)\n", err);
- return err;
- }
-
- *tx_clk = devm_clk_get(&pdev->dev, "tx_clk");
- if (IS_ERR(*tx_clk))
- *tx_clk = NULL;
-
- *rx_clk = devm_clk_get(&pdev->dev, "rx_clk");
- if (IS_ERR(*rx_clk))
- *rx_clk = NULL;
-
- err = clk_prepare_enable(*pclk);
- if (err) {
- dev_err(&pdev->dev, "failed to enable pclk (%u)\n", err);
- return err;
- }
-
- err = clk_prepare_enable(*hclk);
- if (err) {
- dev_err(&pdev->dev, "failed to enable hclk (%u)\n", err);
- goto err_disable_pclk;
- }
-
- err = clk_prepare_enable(*tx_clk);
- if (err) {
- dev_err(&pdev->dev, "failed to enable tx_clk (%u)\n", err);
- goto err_disable_hclk;
- }
-
- err = clk_prepare_enable(*rx_clk);
- if (err) {
- dev_err(&pdev->dev, "failed to enable rx_clk (%u)\n", err);
- goto err_disable_txclk;
- }
-
- return 0;
-
-err_disable_txclk:
- clk_disable_unprepare(*tx_clk);
-
-err_disable_hclk:
- clk_disable_unprepare(*hclk);
-
-err_disable_pclk:
- clk_disable_unprepare(*pclk);
-
- return err;
-}
-
-static int macb_init(struct platform_device *pdev)
-{
- struct net_device *dev = platform_get_drvdata(pdev);
- unsigned int hw_q, q;
- struct macb *bp = netdev_priv(dev);
- struct macb_queue *queue;
- int err;
- u32 val;
-
- bp->tx_ring_size = DEFAULT_TX_RING_SIZE;
- bp->rx_ring_size = DEFAULT_RX_RING_SIZE;
-
- /* set the queue register mapping once for all: queue0 has a special
- * register mapping but we don't want to test the queue index then
- * compute the corresponding register offset at run time.
- */
- for (hw_q = 0, q = 0; hw_q < MACB_MAX_QUEUES; ++hw_q) {
- if (!(bp->queue_mask & (1 << hw_q)))
- continue;
-
- queue = &bp->queues[q];
- queue->bp = bp;
- if (hw_q) {
- queue->ISR = GEM_ISR(hw_q - 1);
- queue->IER = GEM_IER(hw_q - 1);
- queue->IDR = GEM_IDR(hw_q - 1);
- queue->IMR = GEM_IMR(hw_q - 1);
- queue->TBQP = GEM_TBQP(hw_q - 1);
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (bp->hw_dma_cap & HW_DMA_CAP_64B)
- queue->TBQPH = GEM_TBQPH(hw_q - 1);
-#endif
- } else {
- /* queue0 uses legacy registers */
- queue->ISR = MACB_ISR;
- queue->IER = MACB_IER;
- queue->IDR = MACB_IDR;
- queue->IMR = MACB_IMR;
- queue->TBQP = MACB_TBQP;
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (bp->hw_dma_cap & HW_DMA_CAP_64B)
- queue->TBQPH = MACB_TBQPH;
-#endif
- }
-
- /* get irq: here we use the linux queue index, not the hardware
- * queue index. the queue irq definitions in the device tree
- * must remove the optional gaps that could exist in the
- * hardware queue mask.
- */
- queue->irq = platform_get_irq(pdev, q);
- err = devm_request_irq(&pdev->dev, queue->irq, macb_interrupt,
- IRQF_SHARED, dev->name, queue);
- if (err) {
- dev_err(&pdev->dev,
- "Unable to request IRQ %d (error %d)\n",
- queue->irq, err);
- return err;
- }
-
- INIT_WORK(&queue->tx_error_task, macb_tx_error_task);
- q++;
- }
-
- dev->netdev_ops = &macb_netdev_ops;
- netif_napi_add(dev, &bp->napi, macb_poll, 64);
-
- /* setup appropriated routines according to adapter type */
- if (macb_is_gem(bp)) {
- bp->max_tx_length = GEM_MAX_TX_LEN;
- bp->macbgem_ops.mog_alloc_rx_buffers = gem_alloc_rx_buffers;
- bp->macbgem_ops.mog_free_rx_buffers = gem_free_rx_buffers;
- bp->macbgem_ops.mog_init_rings = gem_init_rings;
- bp->macbgem_ops.mog_rx = gem_rx;
- dev->ethtool_ops = &gem_ethtool_ops;
- } else {
- bp->max_tx_length = MACB_MAX_TX_LEN;
- bp->macbgem_ops.mog_alloc_rx_buffers = macb_alloc_rx_buffers;
- bp->macbgem_ops.mog_free_rx_buffers = macb_free_rx_buffers;
- bp->macbgem_ops.mog_init_rings = macb_init_rings;
- bp->macbgem_ops.mog_rx = macb_rx;
- dev->ethtool_ops = &macb_ethtool_ops;
- }
-
- /* Set features */
- dev->hw_features = NETIF_F_SG;
-
- /* Check LSO capability */
- if (GEM_BFEXT(PBUF_LSO, gem_readl(bp, DCFG6)))
- dev->hw_features |= MACB_NETIF_LSO;
-
- /* Checksum offload is only available on gem with packet buffer */
- if (macb_is_gem(bp) && !(bp->caps & MACB_CAPS_FIFO_MODE))
- dev->hw_features |= NETIF_F_HW_CSUM | NETIF_F_RXCSUM;
- if (bp->caps & MACB_CAPS_SG_DISABLED)
- dev->hw_features &= ~NETIF_F_SG;
- dev->features = dev->hw_features;
-
- if (!(bp->caps & MACB_CAPS_USRIO_DISABLED)) {
- val = 0;
- if (bp->phy_interface == PHY_INTERFACE_MODE_RGMII)
- val = GEM_BIT(RGMII);
- else if (bp->phy_interface == PHY_INTERFACE_MODE_RMII &&
- (bp->caps & MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII))
- val = MACB_BIT(RMII);
- else if (!(bp->caps & MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII))
- val = MACB_BIT(MII);
-
- if (bp->caps & MACB_CAPS_USRIO_HAS_CLKEN)
- val |= MACB_BIT(CLKEN);
-
- macb_or_gem_writel(bp, USRIO, val);
- }
-
- /* Set MII management clock divider */
- val = macb_mdc_clk_div(bp);
- val |= macb_dbw(bp);
- if (bp->phy_interface == PHY_INTERFACE_MODE_SGMII)
- val |= GEM_BIT(SGMIIEN) | GEM_BIT(PCSSEL);
- macb_writel(bp, NCFGR, val);
-
- return 0;
-}
-
-#if defined(CONFIG_OF)
-/* 1518 rounded up */
-#define AT91ETHER_MAX_RBUFF_SZ 0x600
-/* max number of receive buffers */
-#define AT91ETHER_MAX_RX_DESCR 9
-
-/* Initialize and start the Receiver and Transmit subsystems */
-static int at91ether_start(struct net_device *dev)
-{
- struct macb *lp = netdev_priv(dev);
- struct macb_dma_desc *desc;
- dma_addr_t addr;
- u32 ctl;
- int i;
-
- lp->rx_ring = dma_alloc_coherent(&lp->pdev->dev,
- (AT91ETHER_MAX_RX_DESCR *
- macb_dma_desc_get_size(lp)),
- &lp->rx_ring_dma, GFP_KERNEL);
- if (!lp->rx_ring)
- return -ENOMEM;
-
- lp->rx_buffers = dma_alloc_coherent(&lp->pdev->dev,
- AT91ETHER_MAX_RX_DESCR *
- AT91ETHER_MAX_RBUFF_SZ,
- &lp->rx_buffers_dma, GFP_KERNEL);
- if (!lp->rx_buffers) {
- dma_free_coherent(&lp->pdev->dev,
- AT91ETHER_MAX_RX_DESCR *
- macb_dma_desc_get_size(lp),
- lp->rx_ring, lp->rx_ring_dma);
- lp->rx_ring = NULL;
- return -ENOMEM;
- }
-
- addr = lp->rx_buffers_dma;
- for (i = 0; i < AT91ETHER_MAX_RX_DESCR; i++) {
- desc = macb_rx_desc(lp, i);
- macb_set_addr(lp, desc, addr);
- desc->ctrl = 0;
- addr += AT91ETHER_MAX_RBUFF_SZ;
- }
-
- /* Set the Wrap bit on the last descriptor */
- desc->addr |= MACB_BIT(RX_WRAP);
-
- /* Reset buffer index */
- lp->rx_tail = 0;
-
- /* Program address of descriptor list in Rx Buffer Queue register */
- macb_writel(lp, RBQP, lp->rx_ring_dma);
-
- /* Enable Receive and Transmit */
- ctl = macb_readl(lp, NCR);
- macb_writel(lp, NCR, ctl | MACB_BIT(RE) | MACB_BIT(TE));
-
- return 0;
-}
-
-/* Open the ethernet interface */
-static int at91ether_open(struct net_device *dev)
-{
- struct macb *lp = netdev_priv(dev);
- u32 ctl;
- int ret;
-
- /* Clear internal statistics */
- ctl = macb_readl(lp, NCR);
- macb_writel(lp, NCR, ctl | MACB_BIT(CLRSTAT));
-
- macb_set_hwaddr(lp);
-
- ret = at91ether_start(dev);
- if (ret)
- return ret;
-
- /* Enable MAC interrupts */
- macb_writel(lp, IER, MACB_BIT(RCOMP) |
- MACB_BIT(RXUBR) |
- MACB_BIT(ISR_TUND) |
- MACB_BIT(ISR_RLE) |
- MACB_BIT(TCOMP) |
- MACB_BIT(ISR_ROVR) |
- MACB_BIT(HRESP));
-
- /* schedule a link state check */
- phy_start(dev->phydev);
-
- netif_start_queue(dev);
-
- return 0;
-}
-
-/* Close the interface */
-static int at91ether_close(struct net_device *dev)
-{
- struct macb *lp = netdev_priv(dev);
- u32 ctl;
-
- /* Disable Receiver and Transmitter */
- ctl = macb_readl(lp, NCR);
- macb_writel(lp, NCR, ctl & ~(MACB_BIT(TE) | MACB_BIT(RE)));
-
- /* Disable MAC interrupts */
- macb_writel(lp, IDR, MACB_BIT(RCOMP) |
- MACB_BIT(RXUBR) |
- MACB_BIT(ISR_TUND) |
- MACB_BIT(ISR_RLE) |
- MACB_BIT(TCOMP) |
- MACB_BIT(ISR_ROVR) |
- MACB_BIT(HRESP));
-
- netif_stop_queue(dev);
-
- dma_free_coherent(&lp->pdev->dev,
- AT91ETHER_MAX_RX_DESCR *
- macb_dma_desc_get_size(lp),
- lp->rx_ring, lp->rx_ring_dma);
- lp->rx_ring = NULL;
-
- dma_free_coherent(&lp->pdev->dev,
- AT91ETHER_MAX_RX_DESCR * AT91ETHER_MAX_RBUFF_SZ,
- lp->rx_buffers, lp->rx_buffers_dma);
- lp->rx_buffers = NULL;
-
- return 0;
-}
-
-/* Transmit packet */
-static int at91ether_start_xmit(struct sk_buff *skb, struct net_device *dev)
-{
- struct macb *lp = netdev_priv(dev);
-
- if (macb_readl(lp, TSR) & MACB_BIT(RM9200_BNQ)) {
- netif_stop_queue(dev);
-
- /* Store packet information (to free when Tx completed) */
- lp->skb = skb;
- lp->skb_length = skb->len;
- lp->skb_physaddr = dma_map_single(NULL, skb->data, skb->len,
- DMA_TO_DEVICE);
- if (dma_mapping_error(NULL, lp->skb_physaddr)) {
- dev_kfree_skb_any(skb);
- dev->stats.tx_dropped++;
- netdev_err(dev, "%s: DMA mapping error\n", __func__);
- return NETDEV_TX_OK;
- }
-
- /* Set address of the data in the Transmit Address register */
- macb_writel(lp, TAR, lp->skb_physaddr);
- /* Set length of the packet in the Transmit Control register */
- macb_writel(lp, TCR, skb->len);
-
- } else {
- netdev_err(dev, "%s called, but device is busy!\n", __func__);
- return NETDEV_TX_BUSY;
- }
-
- return NETDEV_TX_OK;
-}
-
-/* Extract received frame from buffer descriptors and sent to upper layers.
- * (Called from interrupt context)
- */
-static void at91ether_rx(struct net_device *dev)
-{
- struct macb *lp = netdev_priv(dev);
- struct macb_dma_desc *desc;
- unsigned char *p_recv;
- struct sk_buff *skb;
- unsigned int pktlen;
-
- desc = macb_rx_desc(lp, lp->rx_tail);
- while (desc->addr & MACB_BIT(RX_USED)) {
- p_recv = lp->rx_buffers + lp->rx_tail * AT91ETHER_MAX_RBUFF_SZ;
- pktlen = MACB_BF(RX_FRMLEN, desc->ctrl);
- skb = netdev_alloc_skb(dev, pktlen + 2);
- if (skb) {
- skb_reserve(skb, 2);
- memcpy(skb_put(skb, pktlen), p_recv, pktlen);
-
- skb->protocol = eth_type_trans(skb, dev);
- dev->stats.rx_packets++;
- dev->stats.rx_bytes += pktlen;
- netif_rx(skb);
- } else {
- dev->stats.rx_dropped++;
- }
-
- if (desc->ctrl & MACB_BIT(RX_MHASH_MATCH))
- dev->stats.multicast++;
-
- /* reset ownership bit */
- desc->addr &= ~MACB_BIT(RX_USED);
-
- /* wrap after last buffer */
- if (lp->rx_tail == AT91ETHER_MAX_RX_DESCR - 1)
- lp->rx_tail = 0;
- else
- lp->rx_tail++;
-
- desc = macb_rx_desc(lp, lp->rx_tail);
- }
-}
-
-/* MAC interrupt handler */
-static irqreturn_t at91ether_interrupt(int irq, void *dev_id)
-{
- struct net_device *dev = dev_id;
- struct macb *lp = netdev_priv(dev);
- u32 intstatus, ctl;
-
- /* MAC Interrupt Status register indicates what interrupts are pending.
- * It is automatically cleared once read.
- */
- intstatus = macb_readl(lp, ISR);
-
- /* Receive complete */
- if (intstatus & MACB_BIT(RCOMP))
- at91ether_rx(dev);
-
- /* Transmit complete */
- if (intstatus & MACB_BIT(TCOMP)) {
- /* The TCOM bit is set even if the transmission failed */
- if (intstatus & (MACB_BIT(ISR_TUND) | MACB_BIT(ISR_RLE)))
- dev->stats.tx_errors++;
-
- if (lp->skb) {
- dev_kfree_skb_irq(lp->skb);
- lp->skb = NULL;
- dma_unmap_single(NULL, lp->skb_physaddr,
- lp->skb_length, DMA_TO_DEVICE);
- dev->stats.tx_packets++;
- dev->stats.tx_bytes += lp->skb_length;
- }
- netif_wake_queue(dev);
- }
-
- /* Work-around for EMAC Errata section 41.3.1 */
- if (intstatus & MACB_BIT(RXUBR)) {
- ctl = macb_readl(lp, NCR);
- macb_writel(lp, NCR, ctl & ~MACB_BIT(RE));
- wmb();
- macb_writel(lp, NCR, ctl | MACB_BIT(RE));
- }
-
- if (intstatus & MACB_BIT(ISR_ROVR))
- netdev_err(dev, "ROVR error\n");
-
- return IRQ_HANDLED;
-}
-
-#ifdef CONFIG_NET_POLL_CONTROLLER
-static void at91ether_poll_controller(struct net_device *dev)
-{
- unsigned long flags;
-
- local_irq_save(flags);
- at91ether_interrupt(dev->irq, dev);
- local_irq_restore(flags);
-}
-#endif
-
-static const struct net_device_ops at91ether_netdev_ops = {
- .ndo_open = at91ether_open,
- .ndo_stop = at91ether_close,
- .ndo_start_xmit = at91ether_start_xmit,
- .ndo_get_stats = macb_get_stats,
- .ndo_set_rx_mode = macb_set_rx_mode,
- .ndo_set_mac_address = eth_mac_addr,
- .ndo_do_ioctl = macb_ioctl,
- .ndo_validate_addr = eth_validate_addr,
-#ifdef CONFIG_NET_POLL_CONTROLLER
- .ndo_poll_controller = at91ether_poll_controller,
-#endif
-};
-
-static int at91ether_clk_init(struct platform_device *pdev, struct clk **pclk,
- struct clk **hclk, struct clk **tx_clk,
- struct clk **rx_clk)
-{
- int err;
-
- *hclk = NULL;
- *tx_clk = NULL;
- *rx_clk = NULL;
-
- *pclk = devm_clk_get(&pdev->dev, "ether_clk");
- if (IS_ERR(*pclk))
- return PTR_ERR(*pclk);
-
- err = clk_prepare_enable(*pclk);
- if (err) {
- dev_err(&pdev->dev, "failed to enable pclk (%u)\n", err);
- return err;
- }
-
- return 0;
-}
-
-static int at91ether_init(struct platform_device *pdev)
-{
- struct net_device *dev = platform_get_drvdata(pdev);
- struct macb *bp = netdev_priv(dev);
- int err;
- u32 reg;
-
- dev->netdev_ops = &at91ether_netdev_ops;
- dev->ethtool_ops = &macb_ethtool_ops;
-
- err = devm_request_irq(&pdev->dev, dev->irq, at91ether_interrupt,
- 0, dev->name, dev);
- if (err)
- return err;
-
- macb_writel(bp, NCR, 0);
-
- reg = MACB_BF(CLK, MACB_CLK_DIV32) | MACB_BIT(BIG);
- if (bp->phy_interface == PHY_INTERFACE_MODE_RMII)
- reg |= MACB_BIT(RM9200_RMII);
-
- macb_writel(bp, NCFGR, reg);
-
- return 0;
-}
-
-static const struct macb_config at91sam9260_config = {
- .caps = MACB_CAPS_USRIO_HAS_CLKEN | MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII,
- .clk_init = macb_clk_init,
- .init = macb_init,
-};
-
-static const struct macb_config pc302gem_config = {
- .caps = MACB_CAPS_SG_DISABLED | MACB_CAPS_GIGABIT_MODE_AVAILABLE,
- .dma_burst_length = 16,
- .clk_init = macb_clk_init,
- .init = macb_init,
-};
-
-static const struct macb_config sama5d2_config = {
- .caps = MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII,
- .dma_burst_length = 16,
- .clk_init = macb_clk_init,
- .init = macb_init,
-};
-
-static const struct macb_config sama5d3_config = {
- .caps = MACB_CAPS_SG_DISABLED | MACB_CAPS_GIGABIT_MODE_AVAILABLE
- | MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII,
- .dma_burst_length = 16,
- .clk_init = macb_clk_init,
- .init = macb_init,
-};
-
-static const struct macb_config sama5d4_config = {
- .caps = MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII,
- .dma_burst_length = 4,
- .clk_init = macb_clk_init,
- .init = macb_init,
-};
-
-static const struct macb_config emac_config = {
- .clk_init = at91ether_clk_init,
- .init = at91ether_init,
-};
-
-static const struct macb_config np4_config = {
- .caps = MACB_CAPS_USRIO_DISABLED,
- .clk_init = macb_clk_init,
- .init = macb_init,
-};
-
-static const struct macb_config zynqmp_config = {
- .caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE | MACB_CAPS_JUMBO,
- .dma_burst_length = 16,
- .clk_init = macb_clk_init,
- .init = macb_init,
- .jumbo_max_len = 10240,
-};
-
-static const struct macb_config zynq_config = {
- .caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE | MACB_CAPS_NO_GIGABIT_HALF,
- .dma_burst_length = 16,
- .clk_init = macb_clk_init,
- .init = macb_init,
-};
-
-static const struct of_device_id macb_dt_ids[] = {
- { .compatible = "cdns,at32ap7000-macb" },
- { .compatible = "cdns,at91sam9260-macb", .data = &at91sam9260_config },
- { .compatible = "cdns,macb" },
- { .compatible = "cdns,np4-macb", .data = &np4_config },
- { .compatible = "cdns,pc302-gem", .data = &pc302gem_config },
- { .compatible = "cdns,gem", .data = &pc302gem_config },
- { .compatible = "atmel,sama5d2-gem", .data = &sama5d2_config },
- { .compatible = "atmel,sama5d3-gem", .data = &sama5d3_config },
- { .compatible = "atmel,sama5d4-gem", .data = &sama5d4_config },
- { .compatible = "cdns,at91rm9200-emac", .data = &emac_config },
- { .compatible = "cdns,emac", .data = &emac_config },
- { .compatible = "cdns,zynqmp-gem", .data = &zynqmp_config},
- { .compatible = "cdns,zynq-gem", .data = &zynq_config },
- { /* sentinel */ }
-};
-MODULE_DEVICE_TABLE(of, macb_dt_ids);
-#endif /* CONFIG_OF */
-
-static const struct macb_config default_gem_config = {
- .caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE | MACB_CAPS_JUMBO,
- .dma_burst_length = 16,
- .clk_init = macb_clk_init,
- .init = macb_init,
- .jumbo_max_len = 10240,
-};
-
-static int macb_probe(struct platform_device *pdev)
-{
- const struct macb_config *macb_config = &default_gem_config;
- int (*clk_init)(struct platform_device *, struct clk **,
- struct clk **, struct clk **, struct clk **)
- = macb_config->clk_init;
- int (*init)(struct platform_device *) = macb_config->init;
- struct device_node *np = pdev->dev.of_node;
- struct device_node *phy_node;
- struct clk *pclk, *hclk = NULL, *tx_clk = NULL, *rx_clk = NULL;
- unsigned int queue_mask, num_queues;
- struct macb_platform_data *pdata;
- bool native_io;
- struct phy_device *phydev;
- struct net_device *dev;
- struct resource *regs;
- void __iomem *mem;
- const char *mac;
- struct macb *bp;
- int err;
-
- regs = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- mem = devm_ioremap_resource(&pdev->dev, regs);
- if (IS_ERR(mem))
- return PTR_ERR(mem);
-
- if (np) {
- const struct of_device_id *match;
-
- match = of_match_node(macb_dt_ids, np);
- if (match && match->data) {
- macb_config = match->data;
- clk_init = macb_config->clk_init;
- init = macb_config->init;
- }
- }
-
- err = clk_init(pdev, &pclk, &hclk, &tx_clk, &rx_clk);
- if (err)
- return err;
-
- native_io = hw_is_native_io(mem);
-
- macb_probe_queues(mem, native_io, &queue_mask, &num_queues);
- dev = alloc_etherdev_mq(sizeof(*bp), num_queues);
- if (!dev) {
- err = -ENOMEM;
- goto err_disable_clocks;
- }
-
- dev->base_addr = regs->start;
-
- SET_NETDEV_DEV(dev, &pdev->dev);
-
- bp = netdev_priv(dev);
- bp->pdev = pdev;
- bp->dev = dev;
- bp->regs = mem;
- bp->native_io = native_io;
- if (native_io) {
- bp->macb_reg_readl = hw_readl_native;
- bp->macb_reg_writel = hw_writel_native;
- } else {
- bp->macb_reg_readl = hw_readl;
- bp->macb_reg_writel = hw_writel;
- }
- bp->num_queues = num_queues;
- bp->queue_mask = queue_mask;
- if (macb_config)
- bp->dma_burst_length = macb_config->dma_burst_length;
- bp->pclk = pclk;
- bp->hclk = hclk;
- bp->tx_clk = tx_clk;
- bp->rx_clk = rx_clk;
- if (macb_config)
- bp->jumbo_max_len = macb_config->jumbo_max_len;
-
- bp->wol = 0;
- if (of_get_property(np, "magic-packet", NULL))
- bp->wol |= MACB_WOL_HAS_MAGIC_PACKET;
- device_init_wakeup(&pdev->dev, bp->wol & MACB_WOL_HAS_MAGIC_PACKET);
-
- spin_lock_init(&bp->lock);
-
- /* setup capabilities */
- macb_configure_caps(bp, macb_config);
-
-#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
- if (GEM_BFEXT(DAW64, gem_readl(bp, DCFG6))) {
- dma_set_mask(&pdev->dev, DMA_BIT_MASK(44));
- bp->hw_dma_cap |= HW_DMA_CAP_64B;
- }
-#endif
- platform_set_drvdata(pdev, dev);
-
- dev->irq = platform_get_irq(pdev, 0);
- if (dev->irq < 0) {
- err = dev->irq;
- goto err_out_free_netdev;
- }
-
- /* MTU range: 68 - 1500 or 10240 */
- dev->min_mtu = GEM_MTU_MIN_SIZE;
- if (bp->caps & MACB_CAPS_JUMBO)
- dev->max_mtu = gem_readl(bp, JML) - ETH_HLEN - ETH_FCS_LEN;
- else
- dev->max_mtu = ETH_DATA_LEN;
-
- mac = of_get_mac_address(np);
- if (mac)
- ether_addr_copy(bp->dev->dev_addr, mac);
- else
- macb_get_hwaddr(bp);
-
- /* Power up the PHY if there is a GPIO reset */
- phy_node = of_get_next_available_child(np, NULL);
- if (phy_node) {
- int gpio = of_get_named_gpio(phy_node, "reset-gpios", 0);
-
- if (gpio_is_valid(gpio)) {
- bp->reset_gpio = gpio_to_desc(gpio);
- gpiod_direction_output(bp->reset_gpio, 1);
- }
- }
- of_node_put(phy_node);
-
- err = of_get_phy_mode(np);
- if (err < 0) {
- pdata = dev_get_platdata(&pdev->dev);
- if (pdata && pdata->is_rmii)
- bp->phy_interface = PHY_INTERFACE_MODE_RMII;
- else
- bp->phy_interface = PHY_INTERFACE_MODE_MII;
- } else {
- bp->phy_interface = err;
- }
-
- /* IP specific init */
- err = init(pdev);
- if (err)
- goto err_out_free_netdev;
-
- err = macb_mii_init(bp);
- if (err)
- goto err_out_free_netdev;
-
- phydev = dev->phydev;
-
- netif_carrier_off(dev);
-
- err = register_netdev(dev);
- if (err) {
- dev_err(&pdev->dev, "Cannot register net device, aborting.\n");
- goto err_out_unregister_mdio;
- }
-
- phy_attached_info(phydev);
-
- netdev_info(dev, "Cadence %s rev 0x%08x at 0x%08lx irq %d (%pM)\n",
- macb_is_gem(bp) ? "GEM" : "MACB", macb_readl(bp, MID),
- dev->base_addr, dev->irq, dev->dev_addr);
-
- return 0;
-
-err_out_unregister_mdio:
- phy_disconnect(dev->phydev);
- mdiobus_unregister(bp->mii_bus);
- mdiobus_free(bp->mii_bus);
-
- /* Shutdown the PHY if there is a GPIO reset */
- if (bp->reset_gpio)
- gpiod_set_value(bp->reset_gpio, 0);
-
-err_out_free_netdev:
- free_netdev(dev);
-
-err_disable_clocks:
- clk_disable_unprepare(tx_clk);
- clk_disable_unprepare(hclk);
- clk_disable_unprepare(pclk);
- clk_disable_unprepare(rx_clk);
-
- return err;
-}
-
-static int macb_remove(struct platform_device *pdev)
-{
- struct net_device *dev;
- struct macb *bp;
-
- dev = platform_get_drvdata(pdev);
-
- if (dev) {
- bp = netdev_priv(dev);
- if (dev->phydev)
- phy_disconnect(dev->phydev);
- mdiobus_unregister(bp->mii_bus);
- dev->phydev = NULL;
- mdiobus_free(bp->mii_bus);
-
- /* Shutdown the PHY if there is a GPIO reset */
- if (bp->reset_gpio)
- gpiod_set_value(bp->reset_gpio, 0);
-
- unregister_netdev(dev);
- clk_disable_unprepare(bp->tx_clk);
- clk_disable_unprepare(bp->hclk);
- clk_disable_unprepare(bp->pclk);
- clk_disable_unprepare(bp->rx_clk);
- free_netdev(dev);
- }
-
- return 0;
-}
-
-static int __maybe_unused macb_suspend(struct device *dev)
-{
- struct platform_device *pdev = to_platform_device(dev);
- struct net_device *netdev = platform_get_drvdata(pdev);
- struct macb *bp = netdev_priv(netdev);
-
- netif_carrier_off(netdev);
- netif_device_detach(netdev);
-
- if (bp->wol & MACB_WOL_ENABLED) {
- macb_writel(bp, IER, MACB_BIT(WOL));
- macb_writel(bp, WOL, MACB_BIT(MAG));
- enable_irq_wake(bp->queues[0].irq);
- } else {
- clk_disable_unprepare(bp->tx_clk);
- clk_disable_unprepare(bp->hclk);
- clk_disable_unprepare(bp->pclk);
- clk_disable_unprepare(bp->rx_clk);
- }
-
- return 0;
-}
-
-static int __maybe_unused macb_resume(struct device *dev)
-{
- struct platform_device *pdev = to_platform_device(dev);
- struct net_device *netdev = platform_get_drvdata(pdev);
- struct macb *bp = netdev_priv(netdev);
-
- if (bp->wol & MACB_WOL_ENABLED) {
- macb_writel(bp, IDR, MACB_BIT(WOL));
- macb_writel(bp, WOL, 0);
- disable_irq_wake(bp->queues[0].irq);
- } else {
- clk_prepare_enable(bp->pclk);
- clk_prepare_enable(bp->hclk);
- clk_prepare_enable(bp->tx_clk);
- clk_prepare_enable(bp->rx_clk);
- }
-
- netif_device_attach(netdev);
-
- return 0;
-}
-
-static SIMPLE_DEV_PM_OPS(macb_pm_ops, macb_suspend, macb_resume);
-
-static struct platform_driver macb_driver = {
- .probe = macb_probe,
- .remove = macb_remove,
- .driver = {
- .name = "macb",
- .of_match_table = of_match_ptr(macb_dt_ids),
- .pm = &macb_pm_ops,
- },
-};
-
-module_platform_driver(macb_driver);
-
-MODULE_LICENSE("GPL");
-MODULE_DESCRIPTION("Cadence MACB/GEM Ethernet driver");
-MODULE_AUTHOR("Haavard Skinnemoen (Atmel)");
-MODULE_ALIAS("platform:macb");
diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
new file mode 100644
index 0000000..3151429
--- /dev/null
+++ b/drivers/net/ethernet/cadence/macb_main.c
@@ -0,0 +1,3568 @@
+/*
+ * Cadence MACB/GEM Ethernet Controller driver
+ *
+ * Copyright (C) 2004-2006 Atmel Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#include <linux/clk.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/kernel.h>
+#include <linux/types.h>
+#include <linux/circ_buf.h>
+#include <linux/slab.h>
+#include <linux/init.h>
+#include <linux/io.h>
+#include <linux/gpio.h>
+#include <linux/gpio/consumer.h>
+#include <linux/interrupt.h>
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/dma-mapping.h>
+#include <linux/platform_data/macb.h>
+#include <linux/platform_device.h>
+#include <linux/phy.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/of_gpio.h>
+#include <linux/of_mdio.h>
+#include <linux/of_net.h>
+#include <linux/ip.h>
+#include <linux/udp.h>
+#include <linux/tcp.h>
+#include "macb.h"
+
+#define MACB_RX_BUFFER_SIZE 128
+#define RX_BUFFER_MULTIPLE 64 /* bytes */
+
+#define DEFAULT_RX_RING_SIZE 512 /* must be power of 2 */
+#define MIN_RX_RING_SIZE 64
+#define MAX_RX_RING_SIZE 8192
+#define RX_RING_BYTES(bp) (macb_dma_desc_get_size(bp) \
+ * (bp)->rx_ring_size)
+
+#define DEFAULT_TX_RING_SIZE 512 /* must be power of 2 */
+#define MIN_TX_RING_SIZE 64
+#define MAX_TX_RING_SIZE 4096
+#define TX_RING_BYTES(bp) (macb_dma_desc_get_size(bp) \
+ * (bp)->tx_ring_size)
+
+/* level of occupied TX descriptors under which we wake up TX process */
+#define MACB_TX_WAKEUP_THRESH(bp) (3 * (bp)->tx_ring_size / 4)
+
+#define MACB_RX_INT_FLAGS (MACB_BIT(RCOMP) | MACB_BIT(RXUBR) \
+ | MACB_BIT(ISR_ROVR))
+#define MACB_TX_ERR_FLAGS (MACB_BIT(ISR_TUND) \
+ | MACB_BIT(ISR_RLE) \
+ | MACB_BIT(TXERR))
+#define MACB_TX_INT_FLAGS (MACB_TX_ERR_FLAGS | MACB_BIT(TCOMP))
+
+/* Max length of transmit frame must be a multiple of 8 bytes */
+#define MACB_TX_LEN_ALIGN 8
+#define MACB_MAX_TX_LEN ((unsigned int)((1 << MACB_TX_FRMLEN_SIZE) - 1) & ~((unsigned int)(MACB_TX_LEN_ALIGN - 1)))
+#define GEM_MAX_TX_LEN ((unsigned int)((1 << GEM_TX_FRMLEN_SIZE) - 1) & ~((unsigned int)(MACB_TX_LEN_ALIGN - 1)))
+
+#define GEM_MTU_MIN_SIZE ETH_MIN_MTU
+#define MACB_NETIF_LSO (NETIF_F_TSO | NETIF_F_UFO)
+
+#define MACB_WOL_HAS_MAGIC_PACKET (0x1 << 0)
+#define MACB_WOL_ENABLED (0x1 << 1)
+
+/* Graceful stop timeouts in us. We should allow up to
+ * 1 frame time (10 Mbits/s, full-duplex, ignoring collisions)
+ */
+#define MACB_HALT_TIMEOUT 1230
+
+/* DMA buffer descriptor might be different size
+ * depends on hardware configuration:
+ *
+ * 1. dma address width 32 bits:
+ * word 1: 32 bit address of Data Buffer
+ * word 2: control
+ *
+ * 2. dma address width 64 bits:
+ * word 1: 32 bit address of Data Buffer
+ * word 2: control
+ * word 3: upper 32 bit address of Data Buffer
+ * word 4: unused
+ *
+ * 3. dma address width 32 bits with hardware timestamping:
+ * word 1: 32 bit address of Data Buffer
+ * word 2: control
+ * word 3: timestamp word 1
+ * word 4: timestamp word 2
+ *
+ * 4. dma address width 64 bits with hardware timestamping:
+ * word 1: 32 bit address of Data Buffer
+ * word 2: control
+ * word 3: upper 32 bit address of Data Buffer
+ * word 4: unused
+ * word 5: timestamp word 1
+ * word 6: timestamp word 2
+ */
+static unsigned int macb_dma_desc_get_size(struct macb *bp)
+{
+#ifdef MACB_EXT_DESC
+ unsigned int desc_size;
+
+ switch (bp->hw_dma_cap) {
+ case HW_DMA_CAP_64B:
+ desc_size = sizeof(struct macb_dma_desc)
+ + sizeof(struct macb_dma_desc_64);
+ break;
+ case HW_DMA_CAP_PTP:
+ desc_size = sizeof(struct macb_dma_desc)
+ + sizeof(struct macb_dma_desc_ptp);
+ break;
+ case HW_DMA_CAP_64B_PTP:
+ desc_size = sizeof(struct macb_dma_desc)
+ + sizeof(struct macb_dma_desc_64)
+ + sizeof(struct macb_dma_desc_ptp);
+ break;
+ default:
+ desc_size = sizeof(struct macb_dma_desc);
+ }
+ return desc_size;
+#endif
+ return sizeof(struct macb_dma_desc);
+}
+
+static unsigned int macb_adj_dma_desc_idx(struct macb *bp, unsigned int desc_idx)
+{
+#ifdef MACB_EXT_DESC
+ switch (bp->hw_dma_cap) {
+ case HW_DMA_CAP_64B:
+ case HW_DMA_CAP_PTP:
+ desc_idx <<= 1;
+ break;
+ case HW_DMA_CAP_64B_PTP:
+ desc_idx *= 3;
+ break;
+ default:
+ break;
+ }
+ return desc_idx;
+#endif
+ return desc_idx;
+}
+
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+static struct macb_dma_desc_64 *macb_64b_desc(struct macb *bp, struct macb_dma_desc *desc)
+{
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B)
+ return (struct macb_dma_desc_64 *)((void *)desc + sizeof(struct macb_dma_desc));
+ return NULL;
+}
+#endif
+
+/* Ring buffer accessors */
+static unsigned int macb_tx_ring_wrap(struct macb *bp, unsigned int index)
+{
+ return index & (bp->tx_ring_size - 1);
+}
+
+static struct macb_dma_desc *macb_tx_desc(struct macb_queue *queue,
+ unsigned int index)
+{
+ index = macb_tx_ring_wrap(queue->bp, index);
+ index = macb_adj_dma_desc_idx(queue->bp, index);
+ return &queue->tx_ring[index];
+}
+
+static struct macb_tx_skb *macb_tx_skb(struct macb_queue *queue,
+ unsigned int index)
+{
+ return &queue->tx_skb[macb_tx_ring_wrap(queue->bp, index)];
+}
+
+static dma_addr_t macb_tx_dma(struct macb_queue *queue, unsigned int index)
+{
+ dma_addr_t offset;
+
+ offset = macb_tx_ring_wrap(queue->bp, index) *
+ macb_dma_desc_get_size(queue->bp);
+
+ return queue->tx_ring_dma + offset;
+}
+
+static unsigned int macb_rx_ring_wrap(struct macb *bp, unsigned int index)
+{
+ return index & (bp->rx_ring_size - 1);
+}
+
+static struct macb_dma_desc *macb_rx_desc(struct macb *bp, unsigned int index)
+{
+ index = macb_rx_ring_wrap(bp, index);
+ index = macb_adj_dma_desc_idx(bp, index);
+ return &bp->rx_ring[index];
+}
+
+static void *macb_rx_buffer(struct macb *bp, unsigned int index)
+{
+ return bp->rx_buffers + bp->rx_buffer_size *
+ macb_rx_ring_wrap(bp, index);
+}
+
+/* I/O accessors */
+static u32 hw_readl_native(struct macb *bp, int offset)
+{
+ return __raw_readl(bp->regs + offset);
+}
+
+static void hw_writel_native(struct macb *bp, int offset, u32 value)
+{
+ __raw_writel(value, bp->regs + offset);
+}
+
+static u32 hw_readl(struct macb *bp, int offset)
+{
+ return readl_relaxed(bp->regs + offset);
+}
+
+static void hw_writel(struct macb *bp, int offset, u32 value)
+{
+ writel_relaxed(value, bp->regs + offset);
+}
+
+/* Find the CPU endianness by using the loopback bit of NCR register. When the
+ * CPU is in big endian we need to program swapped mode for management
+ * descriptor access.
+ */
+static bool hw_is_native_io(void __iomem *addr)
+{
+ u32 value = MACB_BIT(LLB);
+
+ __raw_writel(value, addr + MACB_NCR);
+ value = __raw_readl(addr + MACB_NCR);
+
+ /* Write 0 back to disable everything */
+ __raw_writel(0, addr + MACB_NCR);
+
+ return value == MACB_BIT(LLB);
+}
+
+static bool hw_is_gem(void __iomem *addr, bool native_io)
+{
+ u32 id;
+
+ if (native_io)
+ id = __raw_readl(addr + MACB_MID);
+ else
+ id = readl_relaxed(addr + MACB_MID);
+
+ return MACB_BFEXT(IDNUM, id) >= 0x2;
+}
+
+static void macb_set_hwaddr(struct macb *bp)
+{
+ u32 bottom;
+ u16 top;
+
+ bottom = cpu_to_le32(*((u32 *)bp->dev->dev_addr));
+ macb_or_gem_writel(bp, SA1B, bottom);
+ top = cpu_to_le16(*((u16 *)(bp->dev->dev_addr + 4)));
+ macb_or_gem_writel(bp, SA1T, top);
+
+ /* Clear unused address register sets */
+ macb_or_gem_writel(bp, SA2B, 0);
+ macb_or_gem_writel(bp, SA2T, 0);
+ macb_or_gem_writel(bp, SA3B, 0);
+ macb_or_gem_writel(bp, SA3T, 0);
+ macb_or_gem_writel(bp, SA4B, 0);
+ macb_or_gem_writel(bp, SA4T, 0);
+}
+
+static void macb_get_hwaddr(struct macb *bp)
+{
+ struct macb_platform_data *pdata;
+ u32 bottom;
+ u16 top;
+ u8 addr[6];
+ int i;
+
+ pdata = dev_get_platdata(&bp->pdev->dev);
+
+ /* Check all 4 address register for valid address */
+ for (i = 0; i < 4; i++) {
+ bottom = macb_or_gem_readl(bp, SA1B + i * 8);
+ top = macb_or_gem_readl(bp, SA1T + i * 8);
+
+ if (pdata && pdata->rev_eth_addr) {
+ addr[5] = bottom & 0xff;
+ addr[4] = (bottom >> 8) & 0xff;
+ addr[3] = (bottom >> 16) & 0xff;
+ addr[2] = (bottom >> 24) & 0xff;
+ addr[1] = top & 0xff;
+ addr[0] = (top & 0xff00) >> 8;
+ } else {
+ addr[0] = bottom & 0xff;
+ addr[1] = (bottom >> 8) & 0xff;
+ addr[2] = (bottom >> 16) & 0xff;
+ addr[3] = (bottom >> 24) & 0xff;
+ addr[4] = top & 0xff;
+ addr[5] = (top >> 8) & 0xff;
+ }
+
+ if (is_valid_ether_addr(addr)) {
+ memcpy(bp->dev->dev_addr, addr, sizeof(addr));
+ return;
+ }
+ }
+
+ dev_info(&bp->pdev->dev, "invalid hw address, using random\n");
+ eth_hw_addr_random(bp->dev);
+}
+
+static int macb_mdio_read(struct mii_bus *bus, int mii_id, int regnum)
+{
+ struct macb *bp = bus->priv;
+ int value;
+
+ macb_writel(bp, MAN, (MACB_BF(SOF, MACB_MAN_SOF)
+ | MACB_BF(RW, MACB_MAN_READ)
+ | MACB_BF(PHYA, mii_id)
+ | MACB_BF(REGA, regnum)
+ | MACB_BF(CODE, MACB_MAN_CODE)));
+
+ /* wait for end of transfer */
+ while (!MACB_BFEXT(IDLE, macb_readl(bp, NSR)))
+ cpu_relax();
+
+ value = MACB_BFEXT(DATA, macb_readl(bp, MAN));
+
+ return value;
+}
+
+static int macb_mdio_write(struct mii_bus *bus, int mii_id, int regnum,
+ u16 value)
+{
+ struct macb *bp = bus->priv;
+
+ macb_writel(bp, MAN, (MACB_BF(SOF, MACB_MAN_SOF)
+ | MACB_BF(RW, MACB_MAN_WRITE)
+ | MACB_BF(PHYA, mii_id)
+ | MACB_BF(REGA, regnum)
+ | MACB_BF(CODE, MACB_MAN_CODE)
+ | MACB_BF(DATA, value)));
+
+ /* wait for end of transfer */
+ while (!MACB_BFEXT(IDLE, macb_readl(bp, NSR)))
+ cpu_relax();
+
+ return 0;
+}
+
+/**
+ * macb_set_tx_clk() - Set a clock to a new frequency
+ * @clk Pointer to the clock to change
+ * @rate New frequency in Hz
+ * @dev Pointer to the struct net_device
+ */
+static void macb_set_tx_clk(struct clk *clk, int speed, struct net_device *dev)
+{
+ long ferr, rate, rate_rounded;
+
+ if (!clk)
+ return;
+
+ switch (speed) {
+ case SPEED_10:
+ rate = 2500000;
+ break;
+ case SPEED_100:
+ rate = 25000000;
+ break;
+ case SPEED_1000:
+ rate = 125000000;
+ break;
+ default:
+ return;
+ }
+
+ rate_rounded = clk_round_rate(clk, rate);
+ if (rate_rounded < 0)
+ return;
+
+ /* RGMII allows 50 ppm frequency error. Test and warn if this limit
+ * is not satisfied.
+ */
+ ferr = abs(rate_rounded - rate);
+ ferr = DIV_ROUND_UP(ferr, rate / 100000);
+ if (ferr > 5)
+ netdev_warn(dev, "unable to generate target frequency: %ld Hz\n",
+ rate);
+
+ if (clk_set_rate(clk, rate_rounded))
+ netdev_err(dev, "adjusting tx_clk failed.\n");
+}
+
+static void macb_handle_link_change(struct net_device *dev)
+{
+ struct macb *bp = netdev_priv(dev);
+ struct phy_device *phydev = dev->phydev;
+ unsigned long flags;
+ int status_change = 0;
+
+ spin_lock_irqsave(&bp->lock, flags);
+
+ if (phydev->link) {
+ if ((bp->speed != phydev->speed) ||
+ (bp->duplex != phydev->duplex)) {
+ u32 reg;
+
+ reg = macb_readl(bp, NCFGR);
+ reg &= ~(MACB_BIT(SPD) | MACB_BIT(FD));
+ if (macb_is_gem(bp))
+ reg &= ~GEM_BIT(GBE);
+
+ if (phydev->duplex)
+ reg |= MACB_BIT(FD);
+ if (phydev->speed == SPEED_100)
+ reg |= MACB_BIT(SPD);
+ if (phydev->speed == SPEED_1000 &&
+ bp->caps & MACB_CAPS_GIGABIT_MODE_AVAILABLE)
+ reg |= GEM_BIT(GBE);
+
+ macb_or_gem_writel(bp, NCFGR, reg);
+
+ bp->speed = phydev->speed;
+ bp->duplex = phydev->duplex;
+ status_change = 1;
+ }
+ }
+
+ if (phydev->link != bp->link) {
+ if (!phydev->link) {
+ bp->speed = 0;
+ bp->duplex = -1;
+ }
+ bp->link = phydev->link;
+
+ status_change = 1;
+ }
+
+ spin_unlock_irqrestore(&bp->lock, flags);
+
+ if (status_change) {
+ if (phydev->link) {
+ /* Update the TX clock rate if and only if the link is
+ * up and there has been a link change.
+ */
+ macb_set_tx_clk(bp->tx_clk, phydev->speed, dev);
+
+ netif_carrier_on(dev);
+ netdev_info(dev, "link up (%d/%s)\n",
+ phydev->speed,
+ phydev->duplex == DUPLEX_FULL ?
+ "Full" : "Half");
+ } else {
+ netif_carrier_off(dev);
+ netdev_info(dev, "link down\n");
+ }
+ }
+}
+
+/* based on au1000_eth. c*/
+static int macb_mii_probe(struct net_device *dev)
+{
+ struct macb *bp = netdev_priv(dev);
+ struct macb_platform_data *pdata;
+ struct phy_device *phydev;
+ int phy_irq;
+ int ret;
+
+ phydev = phy_find_first(bp->mii_bus);
+ if (!phydev) {
+ netdev_err(dev, "no PHY found\n");
+ return -ENXIO;
+ }
+
+ pdata = dev_get_platdata(&bp->pdev->dev);
+ if (pdata) {
+ if (gpio_is_valid(pdata->phy_irq_pin)) {
+ ret = devm_gpio_request(&bp->pdev->dev,
+ pdata->phy_irq_pin, "phy int");
+ if (!ret) {
+ phy_irq = gpio_to_irq(pdata->phy_irq_pin);
+ phydev->irq = (phy_irq < 0) ? PHY_POLL : phy_irq;
+ }
+ } else {
+ phydev->irq = PHY_POLL;
+ }
+ }
+
+ /* attach the mac to the phy */
+ ret = phy_connect_direct(dev, phydev, &macb_handle_link_change,
+ bp->phy_interface);
+ if (ret) {
+ netdev_err(dev, "Could not attach to PHY\n");
+ return ret;
+ }
+
+ /* mask with MAC supported features */
+ if (macb_is_gem(bp) && bp->caps & MACB_CAPS_GIGABIT_MODE_AVAILABLE)
+ phydev->supported &= PHY_GBIT_FEATURES;
+ else
+ phydev->supported &= PHY_BASIC_FEATURES;
+
+ if (bp->caps & MACB_CAPS_NO_GIGABIT_HALF)
+ phydev->supported &= ~SUPPORTED_1000baseT_Half;
+
+ phydev->advertising = phydev->supported;
+
+ bp->link = 0;
+ bp->speed = 0;
+ bp->duplex = -1;
+
+ return 0;
+}
+
+static int macb_mii_init(struct macb *bp)
+{
+ struct macb_platform_data *pdata;
+ struct device_node *np;
+ int err = -ENXIO, i;
+
+ /* Enable management port */
+ macb_writel(bp, NCR, MACB_BIT(MPE));
+
+ bp->mii_bus = mdiobus_alloc();
+ if (!bp->mii_bus) {
+ err = -ENOMEM;
+ goto err_out;
+ }
+
+ bp->mii_bus->name = "MACB_mii_bus";
+ bp->mii_bus->read = &macb_mdio_read;
+ bp->mii_bus->write = &macb_mdio_write;
+ snprintf(bp->mii_bus->id, MII_BUS_ID_SIZE, "%s-%x",
+ bp->pdev->name, bp->pdev->id);
+ bp->mii_bus->priv = bp;
+ bp->mii_bus->parent = &bp->pdev->dev;
+ pdata = dev_get_platdata(&bp->pdev->dev);
+
+ dev_set_drvdata(&bp->dev->dev, bp->mii_bus);
+
+ np = bp->pdev->dev.of_node;
+ if (np) {
+ /* try dt phy registration */
+ err = of_mdiobus_register(bp->mii_bus, np);
+
+ /* fallback to standard phy registration if no phy were
+ * found during dt phy registration
+ */
+ if (!err && !phy_find_first(bp->mii_bus)) {
+ for (i = 0; i < PHY_MAX_ADDR; i++) {
+ struct phy_device *phydev;
+
+ phydev = mdiobus_scan(bp->mii_bus, i);
+ if (IS_ERR(phydev) &&
+ PTR_ERR(phydev) != -ENODEV) {
+ err = PTR_ERR(phydev);
+ break;
+ }
+ }
+
+ if (err)
+ goto err_out_unregister_bus;
+ }
+ } else {
+ for (i = 0; i < PHY_MAX_ADDR; i++)
+ bp->mii_bus->irq[i] = PHY_POLL;
+
+ if (pdata)
+ bp->mii_bus->phy_mask = pdata->phy_mask;
+
+ err = mdiobus_register(bp->mii_bus);
+ }
+
+ if (err)
+ goto err_out_free_mdiobus;
+
+ err = macb_mii_probe(bp->dev);
+ if (err)
+ goto err_out_unregister_bus;
+
+ return 0;
+
+err_out_unregister_bus:
+ mdiobus_unregister(bp->mii_bus);
+err_out_free_mdiobus:
+ mdiobus_free(bp->mii_bus);
+err_out:
+ return err;
+}
+
+static void macb_update_stats(struct macb *bp)
+{
+ u32 *p = &bp->hw_stats.macb.rx_pause_frames;
+ u32 *end = &bp->hw_stats.macb.tx_pause_frames + 1;
+ int offset = MACB_PFR;
+
+ WARN_ON((unsigned long)(end - p - 1) != (MACB_TPF - MACB_PFR) / 4);
+
+ for (; p < end; p++, offset += 4)
+ *p += bp->macb_reg_readl(bp, offset);
+}
+
+static int macb_halt_tx(struct macb *bp)
+{
+ unsigned long halt_time, timeout;
+ u32 status;
+
+ macb_writel(bp, NCR, macb_readl(bp, NCR) | MACB_BIT(THALT));
+
+ timeout = jiffies + usecs_to_jiffies(MACB_HALT_TIMEOUT);
+ do {
+ halt_time = jiffies;
+ status = macb_readl(bp, TSR);
+ if (!(status & MACB_BIT(TGO)))
+ return 0;
+
+ usleep_range(10, 250);
+ } while (time_before(halt_time, timeout));
+
+ return -ETIMEDOUT;
+}
+
+static void macb_tx_unmap(struct macb *bp, struct macb_tx_skb *tx_skb)
+{
+ if (tx_skb->mapping) {
+ if (tx_skb->mapped_as_page)
+ dma_unmap_page(&bp->pdev->dev, tx_skb->mapping,
+ tx_skb->size, DMA_TO_DEVICE);
+ else
+ dma_unmap_single(&bp->pdev->dev, tx_skb->mapping,
+ tx_skb->size, DMA_TO_DEVICE);
+ tx_skb->mapping = 0;
+ }
+
+ if (tx_skb->skb) {
+ dev_kfree_skb_any(tx_skb->skb);
+ tx_skb->skb = NULL;
+ }
+}
+
+static void macb_set_addr(struct macb *bp, struct macb_dma_desc *desc, dma_addr_t addr)
+{
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ struct macb_dma_desc_64 *desc_64;
+
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B) {
+ desc_64 = macb_64b_desc(bp, desc);
+ desc_64->addrh = upper_32_bits(addr);
+ }
+#endif
+ desc->addr = lower_32_bits(addr);
+}
+
+static dma_addr_t macb_get_addr(struct macb *bp, struct macb_dma_desc *desc)
+{
+ dma_addr_t addr = 0;
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ struct macb_dma_desc_64 *desc_64;
+
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B) {
+ desc_64 = macb_64b_desc(bp, desc);
+ addr = ((u64)(desc_64->addrh) << 32);
+ }
+#endif
+ addr |= MACB_BF(RX_WADDR, MACB_BFEXT(RX_WADDR, desc->addr));
+ return addr;
+}
+
+static void macb_tx_error_task(struct work_struct *work)
+{
+ struct macb_queue *queue = container_of(work, struct macb_queue,
+ tx_error_task);
+ struct macb *bp = queue->bp;
+ struct macb_tx_skb *tx_skb;
+ struct macb_dma_desc *desc;
+ struct sk_buff *skb;
+ unsigned int tail;
+ unsigned long flags;
+
+ netdev_vdbg(bp->dev, "macb_tx_error_task: q = %u, t = %u, h = %u\n",
+ (unsigned int)(queue - bp->queues),
+ queue->tx_tail, queue->tx_head);
+
+ /* Prevent the queue IRQ handlers from running: each of them may call
+ * macb_tx_interrupt(), which in turn may call netif_wake_subqueue().
+ * As explained below, we have to halt the transmission before updating
+ * TBQP registers so we call netif_tx_stop_all_queues() to notify the
+ * network engine about the macb/gem being halted.
+ */
+ spin_lock_irqsave(&bp->lock, flags);
+
+ /* Make sure nobody is trying to queue up new packets */
+ netif_tx_stop_all_queues(bp->dev);
+
+ /* Stop transmission now
+ * (in case we have just queued new packets)
+ * macb/gem must be halted to write TBQP register
+ */
+ if (macb_halt_tx(bp))
+ /* Just complain for now, reinitializing TX path can be good */
+ netdev_err(bp->dev, "BUG: halt tx timed out\n");
+
+ /* Treat frames in TX queue including the ones that caused the error.
+ * Free transmit buffers in upper layer.
+ */
+ for (tail = queue->tx_tail; tail != queue->tx_head; tail++) {
+ u32 ctrl;
+
+ desc = macb_tx_desc(queue, tail);
+ ctrl = desc->ctrl;
+ tx_skb = macb_tx_skb(queue, tail);
+ skb = tx_skb->skb;
+
+ if (ctrl & MACB_BIT(TX_USED)) {
+ /* skb is set for the last buffer of the frame */
+ while (!skb) {
+ macb_tx_unmap(bp, tx_skb);
+ tail++;
+ tx_skb = macb_tx_skb(queue, tail);
+ skb = tx_skb->skb;
+ }
+
+ /* ctrl still refers to the first buffer descriptor
+ * since it's the only one written back by the hardware
+ */
+ if (!(ctrl & MACB_BIT(TX_BUF_EXHAUSTED))) {
+ netdev_vdbg(bp->dev, "txerr skb %u (data %p) TX complete\n",
+ macb_tx_ring_wrap(bp, tail),
+ skb->data);
+ bp->dev->stats.tx_packets++;
+ bp->dev->stats.tx_bytes += skb->len;
+ }
+ } else {
+ /* "Buffers exhausted mid-frame" errors may only happen
+ * if the driver is buggy, so complain loudly about
+ * those. Statistics are updated by hardware.
+ */
+ if (ctrl & MACB_BIT(TX_BUF_EXHAUSTED))
+ netdev_err(bp->dev,
+ "BUG: TX buffers exhausted mid-frame\n");
+
+ desc->ctrl = ctrl | MACB_BIT(TX_USED);
+ }
+
+ macb_tx_unmap(bp, tx_skb);
+ }
+
+ /* Set end of TX queue */
+ desc = macb_tx_desc(queue, 0);
+ macb_set_addr(bp, desc, 0);
+ desc->ctrl = MACB_BIT(TX_USED);
+
+ /* Make descriptor updates visible to hardware */
+ wmb();
+
+ /* Reinitialize the TX desc queue */
+ queue_writel(queue, TBQP, lower_32_bits(queue->tx_ring_dma));
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B)
+ queue_writel(queue, TBQPH, upper_32_bits(queue->tx_ring_dma));
+#endif
+ /* Make TX ring reflect state of hardware */
+ queue->tx_head = 0;
+ queue->tx_tail = 0;
+
+ /* Housework before enabling TX IRQ */
+ macb_writel(bp, TSR, macb_readl(bp, TSR));
+ queue_writel(queue, IER, MACB_TX_INT_FLAGS);
+
+ /* Now we are ready to start transmission again */
+ netif_tx_start_all_queues(bp->dev);
+ macb_writel(bp, NCR, macb_readl(bp, NCR) | MACB_BIT(TSTART));
+
+ spin_unlock_irqrestore(&bp->lock, flags);
+}
+
+static void macb_tx_interrupt(struct macb_queue *queue)
+{
+ unsigned int tail;
+ unsigned int head;
+ u32 status;
+ struct macb *bp = queue->bp;
+ u16 queue_index = queue - bp->queues;
+
+ status = macb_readl(bp, TSR);
+ macb_writel(bp, TSR, status);
+
+ if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
+ queue_writel(queue, ISR, MACB_BIT(TCOMP));
+
+ netdev_vdbg(bp->dev, "macb_tx_interrupt status = 0x%03lx\n",
+ (unsigned long)status);
+
+ head = queue->tx_head;
+ for (tail = queue->tx_tail; tail != head; tail++) {
+ struct macb_tx_skb *tx_skb;
+ struct sk_buff *skb;
+ struct macb_dma_desc *desc;
+ u32 ctrl;
+
+ desc = macb_tx_desc(queue, tail);
+
+ /* Make hw descriptor updates visible to CPU */
+ rmb();
+
+ ctrl = desc->ctrl;
+
+ /* TX_USED bit is only set by hardware on the very first buffer
+ * descriptor of the transmitted frame.
+ */
+ if (!(ctrl & MACB_BIT(TX_USED)))
+ break;
+
+ /* Process all buffers of the current transmitted frame */
+ for (;; tail++) {
+ tx_skb = macb_tx_skb(queue, tail);
+ skb = tx_skb->skb;
+
+ /* First, update TX stats if needed */
+ if (skb) {
+ netdev_vdbg(bp->dev, "skb %u (data %p) TX complete\n",
+ macb_tx_ring_wrap(bp, tail),
+ skb->data);
+ bp->dev->stats.tx_packets++;
+ bp->dev->stats.tx_bytes += skb->len;
+ }
+
+ /* Now we can safely release resources */
+ macb_tx_unmap(bp, tx_skb);
+
+ /* skb is set only for the last buffer of the frame.
+ * WARNING: at this point skb has been freed by
+ * macb_tx_unmap().
+ */
+ if (skb)
+ break;
+ }
+ }
+
+ queue->tx_tail = tail;
+ if (__netif_subqueue_stopped(bp->dev, queue_index) &&
+ CIRC_CNT(queue->tx_head, queue->tx_tail,
+ bp->tx_ring_size) <= MACB_TX_WAKEUP_THRESH(bp))
+ netif_wake_subqueue(bp->dev, queue_index);
+}
+
+static void gem_rx_refill(struct macb *bp)
+{
+ unsigned int entry;
+ struct sk_buff *skb;
+ dma_addr_t paddr;
+ struct macb_dma_desc *desc;
+
+ while (CIRC_SPACE(bp->rx_prepared_head, bp->rx_tail,
+ bp->rx_ring_size) > 0) {
+ entry = macb_rx_ring_wrap(bp, bp->rx_prepared_head);
+
+ /* Make hw descriptor updates visible to CPU */
+ rmb();
+
+ bp->rx_prepared_head++;
+ desc = macb_rx_desc(bp, entry);
+
+ if (!bp->rx_skbuff[entry]) {
+ /* allocate sk_buff for this free entry in ring */
+ skb = netdev_alloc_skb(bp->dev, bp->rx_buffer_size);
+ if (unlikely(!skb)) {
+ netdev_err(bp->dev,
+ "Unable to allocate sk_buff\n");
+ break;
+ }
+
+ /* now fill corresponding descriptor entry */
+ paddr = dma_map_single(&bp->pdev->dev, skb->data,
+ bp->rx_buffer_size,
+ DMA_FROM_DEVICE);
+ if (dma_mapping_error(&bp->pdev->dev, paddr)) {
+ dev_kfree_skb(skb);
+ break;
+ }
+
+ bp->rx_skbuff[entry] = skb;
+
+ if (entry == bp->rx_ring_size - 1)
+ paddr |= MACB_BIT(RX_WRAP);
+ macb_set_addr(bp, desc, paddr);
+ desc->ctrl = 0;
+
+ /* properly align Ethernet header */
+ skb_reserve(skb, NET_IP_ALIGN);
+ } else {
+ desc->addr &= ~MACB_BIT(RX_USED);
+ desc->ctrl = 0;
+ }
+ }
+
+ /* Make descriptor updates visible to hardware */
+ wmb();
+
+ netdev_vdbg(bp->dev, "rx ring: prepared head %d, tail %d\n",
+ bp->rx_prepared_head, bp->rx_tail);
+}
+
+/* Mark DMA descriptors from begin up to and not including end as unused */
+static void discard_partial_frame(struct macb *bp, unsigned int begin,
+ unsigned int end)
+{
+ unsigned int frag;
+
+ for (frag = begin; frag != end; frag++) {
+ struct macb_dma_desc *desc = macb_rx_desc(bp, frag);
+
+ desc->addr &= ~MACB_BIT(RX_USED);
+ }
+
+ /* Make descriptor updates visible to hardware */
+ wmb();
+
+ /* When this happens, the hardware stats registers for
+ * whatever caused this is updated, so we don't have to record
+ * anything.
+ */
+}
+
+static int gem_rx(struct macb *bp, int budget)
+{
+ unsigned int len;
+ unsigned int entry;
+ struct sk_buff *skb;
+ struct macb_dma_desc *desc;
+ int count = 0;
+
+ while (count < budget) {
+ u32 ctrl;
+ dma_addr_t addr;
+ bool rxused;
+
+ entry = macb_rx_ring_wrap(bp, bp->rx_tail);
+ desc = macb_rx_desc(bp, entry);
+
+ /* Make hw descriptor updates visible to CPU */
+ rmb();
+
+ rxused = (desc->addr & MACB_BIT(RX_USED)) ? true : false;
+ addr = macb_get_addr(bp, desc);
+ ctrl = desc->ctrl;
+
+ if (!rxused)
+ break;
+
+ bp->rx_tail++;
+ count++;
+
+ if (!(ctrl & MACB_BIT(RX_SOF) && ctrl & MACB_BIT(RX_EOF))) {
+ netdev_err(bp->dev,
+ "not whole frame pointed by descriptor\n");
+ bp->dev->stats.rx_dropped++;
+ break;
+ }
+ skb = bp->rx_skbuff[entry];
+ if (unlikely(!skb)) {
+ netdev_err(bp->dev,
+ "inconsistent Rx descriptor chain\n");
+ bp->dev->stats.rx_dropped++;
+ break;
+ }
+ /* now everything is ready for receiving packet */
+ bp->rx_skbuff[entry] = NULL;
+ len = ctrl & bp->rx_frm_len_mask;
+
+ netdev_vdbg(bp->dev, "gem_rx %u (len %u)\n", entry, len);
+
+ skb_put(skb, len);
+ dma_unmap_single(&bp->pdev->dev, addr,
+ bp->rx_buffer_size, DMA_FROM_DEVICE);
+
+ skb->protocol = eth_type_trans(skb, bp->dev);
+ skb_checksum_none_assert(skb);
+ if (bp->dev->features & NETIF_F_RXCSUM &&
+ !(bp->dev->flags & IFF_PROMISC) &&
+ GEM_BFEXT(RX_CSUM, ctrl) & GEM_RX_CSUM_CHECKED_MASK)
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+
+ bp->dev->stats.rx_packets++;
+ bp->dev->stats.rx_bytes += skb->len;
+
+#if defined(DEBUG) && defined(VERBOSE_DEBUG)
+ netdev_vdbg(bp->dev, "received skb of length %u, csum: %08x\n",
+ skb->len, skb->csum);
+ print_hex_dump(KERN_DEBUG, " mac: ", DUMP_PREFIX_ADDRESS, 16, 1,
+ skb_mac_header(skb), 16, true);
+ print_hex_dump(KERN_DEBUG, "data: ", DUMP_PREFIX_ADDRESS, 16, 1,
+ skb->data, 32, true);
+#endif
+
+ netif_receive_skb(skb);
+ }
+
+ gem_rx_refill(bp);
+
+ return count;
+}
+
+static int macb_rx_frame(struct macb *bp, unsigned int first_frag,
+ unsigned int last_frag)
+{
+ unsigned int len;
+ unsigned int frag;
+ unsigned int offset;
+ struct sk_buff *skb;
+ struct macb_dma_desc *desc;
+
+ desc = macb_rx_desc(bp, last_frag);
+ len = desc->ctrl & bp->rx_frm_len_mask;
+
+ netdev_vdbg(bp->dev, "macb_rx_frame frags %u - %u (len %u)\n",
+ macb_rx_ring_wrap(bp, first_frag),
+ macb_rx_ring_wrap(bp, last_frag), len);
+
+ /* The ethernet header starts NET_IP_ALIGN bytes into the
+ * first buffer. Since the header is 14 bytes, this makes the
+ * payload word-aligned.
+ *
+ * Instead of calling skb_reserve(NET_IP_ALIGN), we just copy
+ * the two padding bytes into the skb so that we avoid hitting
+ * the slowpath in memcpy(), and pull them off afterwards.
+ */
+ skb = netdev_alloc_skb(bp->dev, len + NET_IP_ALIGN);
+ if (!skb) {
+ bp->dev->stats.rx_dropped++;
+ for (frag = first_frag; ; frag++) {
+ desc = macb_rx_desc(bp, frag);
+ desc->addr &= ~MACB_BIT(RX_USED);
+ if (frag == last_frag)
+ break;
+ }
+
+ /* Make descriptor updates visible to hardware */
+ wmb();
+
+ return 1;
+ }
+
+ offset = 0;
+ len += NET_IP_ALIGN;
+ skb_checksum_none_assert(skb);
+ skb_put(skb, len);
+
+ for (frag = first_frag; ; frag++) {
+ unsigned int frag_len = bp->rx_buffer_size;
+
+ if (offset + frag_len > len) {
+ if (unlikely(frag != last_frag)) {
+ dev_kfree_skb_any(skb);
+ return -1;
+ }
+ frag_len = len - offset;
+ }
+ skb_copy_to_linear_data_offset(skb, offset,
+ macb_rx_buffer(bp, frag),
+ frag_len);
+ offset += bp->rx_buffer_size;
+ desc = macb_rx_desc(bp, frag);
+ desc->addr &= ~MACB_BIT(RX_USED);
+
+ if (frag == last_frag)
+ break;
+ }
+
+ /* Make descriptor updates visible to hardware */
+ wmb();
+
+ __skb_pull(skb, NET_IP_ALIGN);
+ skb->protocol = eth_type_trans(skb, bp->dev);
+
+ bp->dev->stats.rx_packets++;
+ bp->dev->stats.rx_bytes += skb->len;
+ netdev_vdbg(bp->dev, "received skb of length %u, csum: %08x\n",
+ skb->len, skb->csum);
+ netif_receive_skb(skb);
+
+ return 0;
+}
+
+static inline void macb_init_rx_ring(struct macb *bp)
+{
+ dma_addr_t addr;
+ struct macb_dma_desc *desc = NULL;
+ int i;
+
+ addr = bp->rx_buffers_dma;
+ for (i = 0; i < bp->rx_ring_size; i++) {
+ desc = macb_rx_desc(bp, i);
+ macb_set_addr(bp, desc, addr);
+ desc->ctrl = 0;
+ addr += bp->rx_buffer_size;
+ }
+ desc->addr |= MACB_BIT(RX_WRAP);
+ bp->rx_tail = 0;
+}
+
+static int macb_rx(struct macb *bp, int budget)
+{
+ bool reset_rx_queue = false;
+ int received = 0;
+ unsigned int tail;
+ int first_frag = -1;
+
+ for (tail = bp->rx_tail; budget > 0; tail++) {
+ struct macb_dma_desc *desc = macb_rx_desc(bp, tail);
+ u32 ctrl;
+
+ /* Make hw descriptor updates visible to CPU */
+ rmb();
+
+ ctrl = desc->ctrl;
+
+ if (!(desc->addr & MACB_BIT(RX_USED)))
+ break;
+
+ if (ctrl & MACB_BIT(RX_SOF)) {
+ if (first_frag != -1)
+ discard_partial_frame(bp, first_frag, tail);
+ first_frag = tail;
+ }
+
+ if (ctrl & MACB_BIT(RX_EOF)) {
+ int dropped;
+
+ if (unlikely(first_frag == -1)) {
+ reset_rx_queue = true;
+ continue;
+ }
+
+ dropped = macb_rx_frame(bp, first_frag, tail);
+ first_frag = -1;
+ if (unlikely(dropped < 0)) {
+ reset_rx_queue = true;
+ continue;
+ }
+ if (!dropped) {
+ received++;
+ budget--;
+ }
+ }
+ }
+
+ if (unlikely(reset_rx_queue)) {
+ unsigned long flags;
+ u32 ctrl;
+
+ netdev_err(bp->dev, "RX queue corruption: reset it\n");
+
+ spin_lock_irqsave(&bp->lock, flags);
+
+ ctrl = macb_readl(bp, NCR);
+ macb_writel(bp, NCR, ctrl & ~MACB_BIT(RE));
+
+ macb_init_rx_ring(bp);
+ macb_writel(bp, RBQP, bp->rx_ring_dma);
+
+ macb_writel(bp, NCR, ctrl | MACB_BIT(RE));
+
+ spin_unlock_irqrestore(&bp->lock, flags);
+ return received;
+ }
+
+ if (first_frag != -1)
+ bp->rx_tail = first_frag;
+ else
+ bp->rx_tail = tail;
+
+ return received;
+}
+
+static int macb_poll(struct napi_struct *napi, int budget)
+{
+ struct macb *bp = container_of(napi, struct macb, napi);
+ int work_done;
+ u32 status;
+
+ status = macb_readl(bp, RSR);
+ macb_writel(bp, RSR, status);
+
+ work_done = 0;
+
+ netdev_vdbg(bp->dev, "poll: status = %08lx, budget = %d\n",
+ (unsigned long)status, budget);
+
+ work_done = bp->macbgem_ops.mog_rx(bp, budget);
+ if (work_done < budget) {
+ napi_complete_done(napi, work_done);
+
+ /* Packets received while interrupts were disabled */
+ status = macb_readl(bp, RSR);
+ if (status) {
+ if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
+ macb_writel(bp, ISR, MACB_BIT(RCOMP));
+ napi_reschedule(napi);
+ } else {
+ macb_writel(bp, IER, MACB_RX_INT_FLAGS);
+ }
+ }
+
+ /* TODO: Handle errors */
+
+ return work_done;
+}
+
+static irqreturn_t macb_interrupt(int irq, void *dev_id)
+{
+ struct macb_queue *queue = dev_id;
+ struct macb *bp = queue->bp;
+ struct net_device *dev = bp->dev;
+ u32 status, ctrl;
+
+ status = queue_readl(queue, ISR);
+
+ if (unlikely(!status))
+ return IRQ_NONE;
+
+ spin_lock(&bp->lock);
+
+ while (status) {
+ /* close possible race with dev_close */
+ if (unlikely(!netif_running(dev))) {
+ queue_writel(queue, IDR, -1);
+ if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
+ queue_writel(queue, ISR, -1);
+ break;
+ }
+
+ netdev_vdbg(bp->dev, "queue = %u, isr = 0x%08lx\n",
+ (unsigned int)(queue - bp->queues),
+ (unsigned long)status);
+
+ if (status & MACB_RX_INT_FLAGS) {
+ /* There's no point taking any more interrupts
+ * until we have processed the buffers. The
+ * scheduling call may fail if the poll routine
+ * is already scheduled, so disable interrupts
+ * now.
+ */
+ queue_writel(queue, IDR, MACB_RX_INT_FLAGS);
+ if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
+ queue_writel(queue, ISR, MACB_BIT(RCOMP));
+
+ if (napi_schedule_prep(&bp->napi)) {
+ netdev_vdbg(bp->dev, "scheduling RX softirq\n");
+ __napi_schedule(&bp->napi);
+ }
+ }
+
+ if (unlikely(status & (MACB_TX_ERR_FLAGS))) {
+ queue_writel(queue, IDR, MACB_TX_INT_FLAGS);
+ schedule_work(&queue->tx_error_task);
+
+ if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
+ queue_writel(queue, ISR, MACB_TX_ERR_FLAGS);
+
+ break;
+ }
+
+ if (status & MACB_BIT(TCOMP))
+ macb_tx_interrupt(queue);
+
+ /* Link change detection isn't possible with RMII, so we'll
+ * add that if/when we get our hands on a full-blown MII PHY.
+ */
+
+ /* There is a hardware issue under heavy load where DMA can
+ * stop, this causes endless "used buffer descriptor read"
+ * interrupts but it can be cleared by re-enabling RX. See
+ * the at91 manual, section 41.3.1 or the Zynq manual
+ * section 16.7.4 for details.
+ */
+ if (status & MACB_BIT(RXUBR)) {
+ ctrl = macb_readl(bp, NCR);
+ macb_writel(bp, NCR, ctrl & ~MACB_BIT(RE));
+ wmb();
+ macb_writel(bp, NCR, ctrl | MACB_BIT(RE));
+
+ if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
+ queue_writel(queue, ISR, MACB_BIT(RXUBR));
+ }
+
+ if (status & MACB_BIT(ISR_ROVR)) {
+ /* We missed at least one packet */
+ if (macb_is_gem(bp))
+ bp->hw_stats.gem.rx_overruns++;
+ else
+ bp->hw_stats.macb.rx_overruns++;
+
+ if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
+ queue_writel(queue, ISR, MACB_BIT(ISR_ROVR));
+ }
+
+ if (status & MACB_BIT(HRESP)) {
+ /* TODO: Reset the hardware, and maybe move the
+ * netdev_err to a lower-priority context as well
+ * (work queue?)
+ */
+ netdev_err(dev, "DMA bus error: HRESP not OK\n");
+
+ if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
+ queue_writel(queue, ISR, MACB_BIT(HRESP));
+ }
+
+ status = queue_readl(queue, ISR);
+ }
+
+ spin_unlock(&bp->lock);
+
+ return IRQ_HANDLED;
+}
+
+#ifdef CONFIG_NET_POLL_CONTROLLER
+/* Polling receive - used by netconsole and other diagnostic tools
+ * to allow network i/o with interrupts disabled.
+ */
+static void macb_poll_controller(struct net_device *dev)
+{
+ struct macb *bp = netdev_priv(dev);
+ struct macb_queue *queue;
+ unsigned long flags;
+ unsigned int q;
+
+ local_irq_save(flags);
+ for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue)
+ macb_interrupt(dev->irq, queue);
+ local_irq_restore(flags);
+}
+#endif
+
+static unsigned int macb_tx_map(struct macb *bp,
+ struct macb_queue *queue,
+ struct sk_buff *skb,
+ unsigned int hdrlen)
+{
+ dma_addr_t mapping;
+ unsigned int len, entry, i, tx_head = queue->tx_head;
+ struct macb_tx_skb *tx_skb = NULL;
+ struct macb_dma_desc *desc;
+ unsigned int offset, size, count = 0;
+ unsigned int f, nr_frags = skb_shinfo(skb)->nr_frags;
+ unsigned int eof = 1, mss_mfs = 0;
+ u32 ctrl, lso_ctrl = 0, seq_ctrl = 0;
+
+ /* LSO */
+ if (skb_shinfo(skb)->gso_size != 0) {
+ if (ip_hdr(skb)->protocol == IPPROTO_UDP)
+ /* UDP - UFO */
+ lso_ctrl = MACB_LSO_UFO_ENABLE;
+ else
+ /* TCP - TSO */
+ lso_ctrl = MACB_LSO_TSO_ENABLE;
+ }
+
+ /* First, map non-paged data */
+ len = skb_headlen(skb);
+
+ /* first buffer length */
+ size = hdrlen;
+
+ offset = 0;
+ while (len) {
+ entry = macb_tx_ring_wrap(bp, tx_head);
+ tx_skb = &queue->tx_skb[entry];
+
+ mapping = dma_map_single(&bp->pdev->dev,
+ skb->data + offset,
+ size, DMA_TO_DEVICE);
+ if (dma_mapping_error(&bp->pdev->dev, mapping))
+ goto dma_error;
+
+ /* Save info to properly release resources */
+ tx_skb->skb = NULL;
+ tx_skb->mapping = mapping;
+ tx_skb->size = size;
+ tx_skb->mapped_as_page = false;
+
+ len -= size;
+ offset += size;
+ count++;
+ tx_head++;
+
+ size = min(len, bp->max_tx_length);
+ }
+
+ /* Then, map paged data from fragments */
+ for (f = 0; f < nr_frags; f++) {
+ const skb_frag_t *frag = &skb_shinfo(skb)->frags[f];
+
+ len = skb_frag_size(frag);
+ offset = 0;
+ while (len) {
+ size = min(len, bp->max_tx_length);
+ entry = macb_tx_ring_wrap(bp, tx_head);
+ tx_skb = &queue->tx_skb[entry];
+
+ mapping = skb_frag_dma_map(&bp->pdev->dev, frag,
+ offset, size, DMA_TO_DEVICE);
+ if (dma_mapping_error(&bp->pdev->dev, mapping))
+ goto dma_error;
+
+ /* Save info to properly release resources */
+ tx_skb->skb = NULL;
+ tx_skb->mapping = mapping;
+ tx_skb->size = size;
+ tx_skb->mapped_as_page = true;
+
+ len -= size;
+ offset += size;
+ count++;
+ tx_head++;
+ }
+ }
+
+ /* Should never happen */
+ if (unlikely(!tx_skb)) {
+ netdev_err(bp->dev, "BUG! empty skb!\n");
+ return 0;
+ }
+
+ /* This is the last buffer of the frame: save socket buffer */
+ tx_skb->skb = skb;
+
+ /* Update TX ring: update buffer descriptors in reverse order
+ * to avoid race condition
+ */
+
+ /* Set 'TX_USED' bit in buffer descriptor at tx_head position
+ * to set the end of TX queue
+ */
+ i = tx_head;
+ entry = macb_tx_ring_wrap(bp, i);
+ ctrl = MACB_BIT(TX_USED);
+ desc = macb_tx_desc(queue, entry);
+ desc->ctrl = ctrl;
+
+ if (lso_ctrl) {
+ if (lso_ctrl == MACB_LSO_UFO_ENABLE)
+ /* include header and FCS in value given to h/w */
+ mss_mfs = skb_shinfo(skb)->gso_size +
+ skb_transport_offset(skb) +
+ ETH_FCS_LEN;
+ else /* TSO */ {
+ mss_mfs = skb_shinfo(skb)->gso_size;
+ /* TCP Sequence Number Source Select
+ * can be set only for TSO
+ */
+ seq_ctrl = 0;
+ }
+ }
+
+ do {
+ i--;
+ entry = macb_tx_ring_wrap(bp, i);
+ tx_skb = &queue->tx_skb[entry];
+ desc = macb_tx_desc(queue, entry);
+
+ ctrl = (u32)tx_skb->size;
+ if (eof) {
+ ctrl |= MACB_BIT(TX_LAST);
+ eof = 0;
+ }
+ if (unlikely(entry == (bp->tx_ring_size - 1)))
+ ctrl |= MACB_BIT(TX_WRAP);
+
+ /* First descriptor is header descriptor */
+ if (i == queue->tx_head) {
+ ctrl |= MACB_BF(TX_LSO, lso_ctrl);
+ ctrl |= MACB_BF(TX_TCP_SEQ_SRC, seq_ctrl);
+ } else
+ /* Only set MSS/MFS on payload descriptors
+ * (second or later descriptor)
+ */
+ ctrl |= MACB_BF(MSS_MFS, mss_mfs);
+
+ /* Set TX buffer descriptor */
+ macb_set_addr(bp, desc, tx_skb->mapping);
+ /* desc->addr must be visible to hardware before clearing
+ * 'TX_USED' bit in desc->ctrl.
+ */
+ wmb();
+ desc->ctrl = ctrl;
+ } while (i != queue->tx_head);
+
+ queue->tx_head = tx_head;
+
+ return count;
+
+dma_error:
+ netdev_err(bp->dev, "TX DMA map failed\n");
+
+ for (i = queue->tx_head; i != tx_head; i++) {
+ tx_skb = macb_tx_skb(queue, i);
+
+ macb_tx_unmap(bp, tx_skb);
+ }
+
+ return 0;
+}
+
+static netdev_features_t macb_features_check(struct sk_buff *skb,
+ struct net_device *dev,
+ netdev_features_t features)
+{
+ unsigned int nr_frags, f;
+ unsigned int hdrlen;
+
+ /* Validate LSO compatibility */
+
+ /* there is only one buffer */
+ if (!skb_is_nonlinear(skb))
+ return features;
+
+ /* length of header */
+ hdrlen = skb_transport_offset(skb);
+ if (ip_hdr(skb)->protocol == IPPROTO_TCP)
+ hdrlen += tcp_hdrlen(skb);
+
+ /* For LSO:
+ * When software supplies two or more payload buffers all payload buffers
+ * apart from the last must be a multiple of 8 bytes in size.
+ */
+ if (!IS_ALIGNED(skb_headlen(skb) - hdrlen, MACB_TX_LEN_ALIGN))
+ return features & ~MACB_NETIF_LSO;
+
+ nr_frags = skb_shinfo(skb)->nr_frags;
+ /* No need to check last fragment */
+ nr_frags--;
+ for (f = 0; f < nr_frags; f++) {
+ const skb_frag_t *frag = &skb_shinfo(skb)->frags[f];
+
+ if (!IS_ALIGNED(skb_frag_size(frag), MACB_TX_LEN_ALIGN))
+ return features & ~MACB_NETIF_LSO;
+ }
+ return features;
+}
+
+static inline int macb_clear_csum(struct sk_buff *skb)
+{
+ /* no change for packets without checksum offloading */
+ if (skb->ip_summed != CHECKSUM_PARTIAL)
+ return 0;
+
+ /* make sure we can modify the header */
+ if (unlikely(skb_cow_head(skb, 0)))
+ return -1;
+
+ /* initialize checksum field
+ * This is required - at least for Zynq, which otherwise calculates
+ * wrong UDP header checksums for UDP packets with UDP data len <=2
+ */
+ *(__sum16 *)(skb_checksum_start(skb) + skb->csum_offset) = 0;
+ return 0;
+}
+
+static int macb_start_xmit(struct sk_buff *skb, struct net_device *dev)
+{
+ u16 queue_index = skb_get_queue_mapping(skb);
+ struct macb *bp = netdev_priv(dev);
+ struct macb_queue *queue = &bp->queues[queue_index];
+ unsigned long flags;
+ unsigned int desc_cnt, nr_frags, frag_size, f;
+ unsigned int hdrlen;
+ bool is_lso, is_udp = 0;
+
+ is_lso = (skb_shinfo(skb)->gso_size != 0);
+
+ if (is_lso) {
+ is_udp = !!(ip_hdr(skb)->protocol == IPPROTO_UDP);
+
+ /* length of headers */
+ if (is_udp)
+ /* only queue eth + ip headers separately for UDP */
+ hdrlen = skb_transport_offset(skb);
+ else
+ hdrlen = skb_transport_offset(skb) + tcp_hdrlen(skb);
+ if (skb_headlen(skb) < hdrlen) {
+ netdev_err(bp->dev, "Error - LSO headers fragmented!!!\n");
+ /* if this is required, would need to copy to single buffer */
+ return NETDEV_TX_BUSY;
+ }
+ } else
+ hdrlen = min(skb_headlen(skb), bp->max_tx_length);
+
+#if defined(DEBUG) && defined(VERBOSE_DEBUG)
+ netdev_vdbg(bp->dev,
+ "start_xmit: queue %hu len %u head %p data %p tail %p end %p\n",
+ queue_index, skb->len, skb->head, skb->data,
+ skb_tail_pointer(skb), skb_end_pointer(skb));
+ print_hex_dump(KERN_DEBUG, "data: ", DUMP_PREFIX_OFFSET, 16, 1,
+ skb->data, 16, true);
+#endif
+
+ /* Count how many TX buffer descriptors are needed to send this
+ * socket buffer: skb fragments of jumbo frames may need to be
+ * split into many buffer descriptors.
+ */
+ if (is_lso && (skb_headlen(skb) > hdrlen))
+ /* extra header descriptor if also payload in first buffer */
+ desc_cnt = DIV_ROUND_UP((skb_headlen(skb) - hdrlen), bp->max_tx_length) + 1;
+ else
+ desc_cnt = DIV_ROUND_UP(skb_headlen(skb), bp->max_tx_length);
+ nr_frags = skb_shinfo(skb)->nr_frags;
+ for (f = 0; f < nr_frags; f++) {
+ frag_size = skb_frag_size(&skb_shinfo(skb)->frags[f]);
+ desc_cnt += DIV_ROUND_UP(frag_size, bp->max_tx_length);
+ }
+
+ spin_lock_irqsave(&bp->lock, flags);
+
+ /* This is a hard error, log it. */
+ if (CIRC_SPACE(queue->tx_head, queue->tx_tail,
+ bp->tx_ring_size) < desc_cnt) {
+ netif_stop_subqueue(dev, queue_index);
+ spin_unlock_irqrestore(&bp->lock, flags);
+ netdev_dbg(bp->dev, "tx_head = %u, tx_tail = %u\n",
+ queue->tx_head, queue->tx_tail);
+ return NETDEV_TX_BUSY;
+ }
+
+ if (macb_clear_csum(skb)) {
+ dev_kfree_skb_any(skb);
+ goto unlock;
+ }
+
+ /* Map socket buffer for DMA transfer */
+ if (!macb_tx_map(bp, queue, skb, hdrlen)) {
+ dev_kfree_skb_any(skb);
+ goto unlock;
+ }
+
+ /* Make newly initialized descriptor visible to hardware */
+ wmb();
+
+ skb_tx_timestamp(skb);
+
+ macb_writel(bp, NCR, macb_readl(bp, NCR) | MACB_BIT(TSTART));
+
+ if (CIRC_SPACE(queue->tx_head, queue->tx_tail, bp->tx_ring_size) < 1)
+ netif_stop_subqueue(dev, queue_index);
+
+unlock:
+ spin_unlock_irqrestore(&bp->lock, flags);
+
+ return NETDEV_TX_OK;
+}
+
+static void macb_init_rx_buffer_size(struct macb *bp, size_t size)
+{
+ if (!macb_is_gem(bp)) {
+ bp->rx_buffer_size = MACB_RX_BUFFER_SIZE;
+ } else {
+ bp->rx_buffer_size = size;
+
+ if (bp->rx_buffer_size % RX_BUFFER_MULTIPLE) {
+ netdev_dbg(bp->dev,
+ "RX buffer must be multiple of %d bytes, expanding\n",
+ RX_BUFFER_MULTIPLE);
+ bp->rx_buffer_size =
+ roundup(bp->rx_buffer_size, RX_BUFFER_MULTIPLE);
+ }
+ }
+
+ netdev_dbg(bp->dev, "mtu [%u] rx_buffer_size [%zu]\n",
+ bp->dev->mtu, bp->rx_buffer_size);
+}
+
+static void gem_free_rx_buffers(struct macb *bp)
+{
+ struct sk_buff *skb;
+ struct macb_dma_desc *desc;
+ dma_addr_t addr;
+ int i;
+
+ if (!bp->rx_skbuff)
+ return;
+
+ for (i = 0; i < bp->rx_ring_size; i++) {
+ skb = bp->rx_skbuff[i];
+
+ if (!skb)
+ continue;
+
+ desc = macb_rx_desc(bp, i);
+ addr = macb_get_addr(bp, desc);
+
+ dma_unmap_single(&bp->pdev->dev, addr, bp->rx_buffer_size,
+ DMA_FROM_DEVICE);
+ dev_kfree_skb_any(skb);
+ skb = NULL;
+ }
+
+ kfree(bp->rx_skbuff);
+ bp->rx_skbuff = NULL;
+}
+
+static void macb_free_rx_buffers(struct macb *bp)
+{
+ if (bp->rx_buffers) {
+ dma_free_coherent(&bp->pdev->dev,
+ bp->rx_ring_size * bp->rx_buffer_size,
+ bp->rx_buffers, bp->rx_buffers_dma);
+ bp->rx_buffers = NULL;
+ }
+}
+
+static void macb_free_consistent(struct macb *bp)
+{
+ struct macb_queue *queue;
+ unsigned int q;
+
+ bp->macbgem_ops.mog_free_rx_buffers(bp);
+ if (bp->rx_ring) {
+ dma_free_coherent(&bp->pdev->dev, RX_RING_BYTES(bp),
+ bp->rx_ring, bp->rx_ring_dma);
+ bp->rx_ring = NULL;
+ }
+
+ for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ kfree(queue->tx_skb);
+ queue->tx_skb = NULL;
+ if (queue->tx_ring) {
+ dma_free_coherent(&bp->pdev->dev, TX_RING_BYTES(bp),
+ queue->tx_ring, queue->tx_ring_dma);
+ queue->tx_ring = NULL;
+ }
+ }
+}
+
+static int gem_alloc_rx_buffers(struct macb *bp)
+{
+ int size;
+
+ size = bp->rx_ring_size * sizeof(struct sk_buff *);
+ bp->rx_skbuff = kzalloc(size, GFP_KERNEL);
+ if (!bp->rx_skbuff)
+ return -ENOMEM;
+ else
+ netdev_dbg(bp->dev,
+ "Allocated %d RX struct sk_buff entries at %p\n",
+ bp->rx_ring_size, bp->rx_skbuff);
+ return 0;
+}
+
+static int macb_alloc_rx_buffers(struct macb *bp)
+{
+ int size;
+
+ size = bp->rx_ring_size * bp->rx_buffer_size;
+ bp->rx_buffers = dma_alloc_coherent(&bp->pdev->dev, size,
+ &bp->rx_buffers_dma, GFP_KERNEL);
+ if (!bp->rx_buffers)
+ return -ENOMEM;
+
+ netdev_dbg(bp->dev,
+ "Allocated RX buffers of %d bytes at %08lx (mapped %p)\n",
+ size, (unsigned long)bp->rx_buffers_dma, bp->rx_buffers);
+ return 0;
+}
+
+static int macb_alloc_consistent(struct macb *bp)
+{
+ struct macb_queue *queue;
+ unsigned int q;
+ int size;
+
+ for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ size = TX_RING_BYTES(bp);
+ queue->tx_ring = dma_alloc_coherent(&bp->pdev->dev, size,
+ &queue->tx_ring_dma,
+ GFP_KERNEL);
+ if (!queue->tx_ring)
+ goto out_err;
+ netdev_dbg(bp->dev,
+ "Allocated TX ring for queue %u of %d bytes at %08lx (mapped %p)\n",
+ q, size, (unsigned long)queue->tx_ring_dma,
+ queue->tx_ring);
+
+ size = bp->tx_ring_size * sizeof(struct macb_tx_skb);
+ queue->tx_skb = kmalloc(size, GFP_KERNEL);
+ if (!queue->tx_skb)
+ goto out_err;
+ }
+
+ size = RX_RING_BYTES(bp);
+ bp->rx_ring = dma_alloc_coherent(&bp->pdev->dev, size,
+ &bp->rx_ring_dma, GFP_KERNEL);
+ if (!bp->rx_ring)
+ goto out_err;
+ netdev_dbg(bp->dev,
+ "Allocated RX ring of %d bytes at %08lx (mapped %p)\n",
+ size, (unsigned long)bp->rx_ring_dma, bp->rx_ring);
+
+ if (bp->macbgem_ops.mog_alloc_rx_buffers(bp))
+ goto out_err;
+
+ return 0;
+
+out_err:
+ macb_free_consistent(bp);
+ return -ENOMEM;
+}
+
+static void gem_init_rings(struct macb *bp)
+{
+ struct macb_queue *queue;
+ struct macb_dma_desc *desc = NULL;
+ unsigned int q;
+ int i;
+
+ for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ for (i = 0; i < bp->tx_ring_size; i++) {
+ desc = macb_tx_desc(queue, i);
+ macb_set_addr(bp, desc, 0);
+ desc->ctrl = MACB_BIT(TX_USED);
+ }
+ desc->ctrl |= MACB_BIT(TX_WRAP);
+ queue->tx_head = 0;
+ queue->tx_tail = 0;
+ }
+
+ bp->rx_tail = 0;
+ bp->rx_prepared_head = 0;
+
+ gem_rx_refill(bp);
+}
+
+static void macb_init_rings(struct macb *bp)
+{
+ int i;
+ struct macb_dma_desc *desc = NULL;
+
+ macb_init_rx_ring(bp);
+
+ for (i = 0; i < bp->tx_ring_size; i++) {
+ desc = macb_tx_desc(&bp->queues[0], i);
+ macb_set_addr(bp, desc, 0);
+ desc->ctrl = MACB_BIT(TX_USED);
+ }
+ bp->queues[0].tx_head = 0;
+ bp->queues[0].tx_tail = 0;
+ desc->ctrl |= MACB_BIT(TX_WRAP);
+}
+
+static void macb_reset_hw(struct macb *bp)
+{
+ struct macb_queue *queue;
+ unsigned int q;
+
+ /* Disable RX and TX (XXX: Should we halt the transmission
+ * more gracefully?)
+ */
+ macb_writel(bp, NCR, 0);
+
+ /* Clear the stats registers (XXX: Update stats first?) */
+ macb_writel(bp, NCR, MACB_BIT(CLRSTAT));
+
+ /* Clear all status flags */
+ macb_writel(bp, TSR, -1);
+ macb_writel(bp, RSR, -1);
+
+ /* Disable all interrupts */
+ for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ queue_writel(queue, IDR, -1);
+ queue_readl(queue, ISR);
+ if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
+ queue_writel(queue, ISR, -1);
+ }
+}
+
+static u32 gem_mdc_clk_div(struct macb *bp)
+{
+ u32 config;
+ unsigned long pclk_hz = clk_get_rate(bp->pclk);
+
+ if (pclk_hz <= 20000000)
+ config = GEM_BF(CLK, GEM_CLK_DIV8);
+ else if (pclk_hz <= 40000000)
+ config = GEM_BF(CLK, GEM_CLK_DIV16);
+ else if (pclk_hz <= 80000000)
+ config = GEM_BF(CLK, GEM_CLK_DIV32);
+ else if (pclk_hz <= 120000000)
+ config = GEM_BF(CLK, GEM_CLK_DIV48);
+ else if (pclk_hz <= 160000000)
+ config = GEM_BF(CLK, GEM_CLK_DIV64);
+ else
+ config = GEM_BF(CLK, GEM_CLK_DIV96);
+
+ return config;
+}
+
+static u32 macb_mdc_clk_div(struct macb *bp)
+{
+ u32 config;
+ unsigned long pclk_hz;
+
+ if (macb_is_gem(bp))
+ return gem_mdc_clk_div(bp);
+
+ pclk_hz = clk_get_rate(bp->pclk);
+ if (pclk_hz <= 20000000)
+ config = MACB_BF(CLK, MACB_CLK_DIV8);
+ else if (pclk_hz <= 40000000)
+ config = MACB_BF(CLK, MACB_CLK_DIV16);
+ else if (pclk_hz <= 80000000)
+ config = MACB_BF(CLK, MACB_CLK_DIV32);
+ else
+ config = MACB_BF(CLK, MACB_CLK_DIV64);
+
+ return config;
+}
+
+/* Get the DMA bus width field of the network configuration register that we
+ * should program. We find the width from decoding the design configuration
+ * register to find the maximum supported data bus width.
+ */
+static u32 macb_dbw(struct macb *bp)
+{
+ if (!macb_is_gem(bp))
+ return 0;
+
+ switch (GEM_BFEXT(DBWDEF, gem_readl(bp, DCFG1))) {
+ case 4:
+ return GEM_BF(DBW, GEM_DBW128);
+ case 2:
+ return GEM_BF(DBW, GEM_DBW64);
+ case 1:
+ default:
+ return GEM_BF(DBW, GEM_DBW32);
+ }
+}
+
+/* Configure the receive DMA engine
+ * - use the correct receive buffer size
+ * - set best burst length for DMA operations
+ * (if not supported by FIFO, it will fallback to default)
+ * - set both rx/tx packet buffers to full memory size
+ * These are configurable parameters for GEM.
+ */
+static void macb_configure_dma(struct macb *bp)
+{
+ u32 dmacfg;
+
+ if (macb_is_gem(bp)) {
+ dmacfg = gem_readl(bp, DMACFG) & ~GEM_BF(RXBS, -1L);
+ dmacfg |= GEM_BF(RXBS, bp->rx_buffer_size / RX_BUFFER_MULTIPLE);
+ if (bp->dma_burst_length)
+ dmacfg = GEM_BFINS(FBLDO, bp->dma_burst_length, dmacfg);
+ dmacfg |= GEM_BIT(TXPBMS) | GEM_BF(RXBMS, -1L);
+ dmacfg &= ~GEM_BIT(ENDIA_PKT);
+
+ if (bp->native_io)
+ dmacfg &= ~GEM_BIT(ENDIA_DESC);
+ else
+ dmacfg |= GEM_BIT(ENDIA_DESC); /* CPU in big endian */
+
+ if (bp->dev->features & NETIF_F_HW_CSUM)
+ dmacfg |= GEM_BIT(TXCOEN);
+ else
+ dmacfg &= ~GEM_BIT(TXCOEN);
+
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B)
+ dmacfg |= GEM_BIT(ADDR64);
+#endif
+#ifdef CONFIG_MACB_USE_HWSTAMP
+ if (bp->hw_dma_cap & HW_DMA_CAP_PTP)
+ dmacfg |= GEM_BIT(RXEXT) | GEM_BIT(TXEXT);
+#endif
+ netdev_dbg(bp->dev, "Cadence configure DMA with 0x%08x\n",
+ dmacfg);
+ gem_writel(bp, DMACFG, dmacfg);
+ }
+}
+
+static void macb_init_hw(struct macb *bp)
+{
+ struct macb_queue *queue;
+ unsigned int q;
+
+ u32 config;
+
+ macb_reset_hw(bp);
+ macb_set_hwaddr(bp);
+
+ config = macb_mdc_clk_div(bp);
+ if (bp->phy_interface == PHY_INTERFACE_MODE_SGMII)
+ config |= GEM_BIT(SGMIIEN) | GEM_BIT(PCSSEL);
+ config |= MACB_BF(RBOF, NET_IP_ALIGN); /* Make eth data aligned */
+ config |= MACB_BIT(PAE); /* PAuse Enable */
+ config |= MACB_BIT(DRFCS); /* Discard Rx FCS */
+ if (bp->caps & MACB_CAPS_JUMBO)
+ config |= MACB_BIT(JFRAME); /* Enable jumbo frames */
+ else
+ config |= MACB_BIT(BIG); /* Receive oversized frames */
+ if (bp->dev->flags & IFF_PROMISC)
+ config |= MACB_BIT(CAF); /* Copy All Frames */
+ else if (macb_is_gem(bp) && bp->dev->features & NETIF_F_RXCSUM)
+ config |= GEM_BIT(RXCOEN);
+ if (!(bp->dev->flags & IFF_BROADCAST))
+ config |= MACB_BIT(NBC); /* No BroadCast */
+ config |= macb_dbw(bp);
+ macb_writel(bp, NCFGR, config);
+ if ((bp->caps & MACB_CAPS_JUMBO) && bp->jumbo_max_len)
+ gem_writel(bp, JML, bp->jumbo_max_len);
+ bp->speed = SPEED_10;
+ bp->duplex = DUPLEX_HALF;
+ bp->rx_frm_len_mask = MACB_RX_FRMLEN_MASK;
+ if (bp->caps & MACB_CAPS_JUMBO)
+ bp->rx_frm_len_mask = MACB_RX_JFRMLEN_MASK;
+
+ macb_configure_dma(bp);
+
+ /* Initialize TX and RX buffers */
+ macb_writel(bp, RBQP, lower_32_bits(bp->rx_ring_dma));
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B)
+ macb_writel(bp, RBQPH, upper_32_bits(bp->rx_ring_dma));
+#endif
+ for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ queue_writel(queue, TBQP, lower_32_bits(queue->tx_ring_dma));
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B)
+ queue_writel(queue, TBQPH, upper_32_bits(queue->tx_ring_dma));
+#endif
+
+ /* Enable interrupts */
+ queue_writel(queue, IER,
+ MACB_RX_INT_FLAGS |
+ MACB_TX_INT_FLAGS |
+ MACB_BIT(HRESP));
+ }
+
+ /* Enable TX and RX */
+ macb_writel(bp, NCR, MACB_BIT(RE) | MACB_BIT(TE) | MACB_BIT(MPE));
+}
+
+/* The hash address register is 64 bits long and takes up two
+ * locations in the memory map. The least significant bits are stored
+ * in EMAC_HSL and the most significant bits in EMAC_HSH.
+ *
+ * The unicast hash enable and the multicast hash enable bits in the
+ * network configuration register enable the reception of hash matched
+ * frames. The destination address is reduced to a 6 bit index into
+ * the 64 bit hash register using the following hash function. The
+ * hash function is an exclusive or of every sixth bit of the
+ * destination address.
+ *
+ * hi[5] = da[5] ^ da[11] ^ da[17] ^ da[23] ^ da[29] ^ da[35] ^ da[41] ^ da[47]
+ * hi[4] = da[4] ^ da[10] ^ da[16] ^ da[22] ^ da[28] ^ da[34] ^ da[40] ^ da[46]
+ * hi[3] = da[3] ^ da[09] ^ da[15] ^ da[21] ^ da[27] ^ da[33] ^ da[39] ^ da[45]
+ * hi[2] = da[2] ^ da[08] ^ da[14] ^ da[20] ^ da[26] ^ da[32] ^ da[38] ^ da[44]
+ * hi[1] = da[1] ^ da[07] ^ da[13] ^ da[19] ^ da[25] ^ da[31] ^ da[37] ^ da[43]
+ * hi[0] = da[0] ^ da[06] ^ da[12] ^ da[18] ^ da[24] ^ da[30] ^ da[36] ^ da[42]
+ *
+ * da[0] represents the least significant bit of the first byte
+ * received, that is, the multicast/unicast indicator, and da[47]
+ * represents the most significant bit of the last byte received. If
+ * the hash index, hi[n], points to a bit that is set in the hash
+ * register then the frame will be matched according to whether the
+ * frame is multicast or unicast. A multicast match will be signalled
+ * if the multicast hash enable bit is set, da[0] is 1 and the hash
+ * index points to a bit set in the hash register. A unicast match
+ * will be signalled if the unicast hash enable bit is set, da[0] is 0
+ * and the hash index points to a bit set in the hash register. To
+ * receive all multicast frames, the hash register should be set with
+ * all ones and the multicast hash enable bit should be set in the
+ * network configuration register.
+ */
+
+static inline int hash_bit_value(int bitnr, __u8 *addr)
+{
+ if (addr[bitnr / 8] & (1 << (bitnr % 8)))
+ return 1;
+ return 0;
+}
+
+/* Return the hash index value for the specified address. */
+static int hash_get_index(__u8 *addr)
+{
+ int i, j, bitval;
+ int hash_index = 0;
+
+ for (j = 0; j < 6; j++) {
+ for (i = 0, bitval = 0; i < 8; i++)
+ bitval ^= hash_bit_value(i * 6 + j, addr);
+
+ hash_index |= (bitval << j);
+ }
+
+ return hash_index;
+}
+
+/* Add multicast addresses to the internal multicast-hash table. */
+static void macb_sethashtable(struct net_device *dev)
+{
+ struct netdev_hw_addr *ha;
+ unsigned long mc_filter[2];
+ unsigned int bitnr;
+ struct macb *bp = netdev_priv(dev);
+
+ mc_filter[0] = 0;
+ mc_filter[1] = 0;
+
+ netdev_for_each_mc_addr(ha, dev) {
+ bitnr = hash_get_index(ha->addr);
+ mc_filter[bitnr >> 5] |= 1 << (bitnr & 31);
+ }
+
+ macb_or_gem_writel(bp, HRB, mc_filter[0]);
+ macb_or_gem_writel(bp, HRT, mc_filter[1]);
+}
+
+/* Enable/Disable promiscuous and multicast modes. */
+static void macb_set_rx_mode(struct net_device *dev)
+{
+ unsigned long cfg;
+ struct macb *bp = netdev_priv(dev);
+
+ cfg = macb_readl(bp, NCFGR);
+
+ if (dev->flags & IFF_PROMISC) {
+ /* Enable promiscuous mode */
+ cfg |= MACB_BIT(CAF);
+
+ /* Disable RX checksum offload */
+ if (macb_is_gem(bp))
+ cfg &= ~GEM_BIT(RXCOEN);
+ } else {
+ /* Disable promiscuous mode */
+ cfg &= ~MACB_BIT(CAF);
+
+ /* Enable RX checksum offload only if requested */
+ if (macb_is_gem(bp) && dev->features & NETIF_F_RXCSUM)
+ cfg |= GEM_BIT(RXCOEN);
+ }
+
+ if (dev->flags & IFF_ALLMULTI) {
+ /* Enable all multicast mode */
+ macb_or_gem_writel(bp, HRB, -1);
+ macb_or_gem_writel(bp, HRT, -1);
+ cfg |= MACB_BIT(NCFGR_MTI);
+ } else if (!netdev_mc_empty(dev)) {
+ /* Enable specific multicasts */
+ macb_sethashtable(dev);
+ cfg |= MACB_BIT(NCFGR_MTI);
+ } else if (dev->flags & (~IFF_ALLMULTI)) {
+ /* Disable all multicast mode */
+ macb_or_gem_writel(bp, HRB, 0);
+ macb_or_gem_writel(bp, HRT, 0);
+ cfg &= ~MACB_BIT(NCFGR_MTI);
+ }
+
+ macb_writel(bp, NCFGR, cfg);
+}
+
+static int macb_open(struct net_device *dev)
+{
+ struct macb *bp = netdev_priv(dev);
+ size_t bufsz = dev->mtu + ETH_HLEN + ETH_FCS_LEN + NET_IP_ALIGN;
+ int err;
+
+ netdev_dbg(bp->dev, "open\n");
+
+ /* carrier starts down */
+ netif_carrier_off(dev);
+
+ /* if the phy is not yet register, retry later*/
+ if (!dev->phydev)
+ return -EAGAIN;
+
+ /* RX buffers initialization */
+ macb_init_rx_buffer_size(bp, bufsz);
+
+ err = macb_alloc_consistent(bp);
+ if (err) {
+ netdev_err(dev, "Unable to allocate DMA memory (error %d)\n",
+ err);
+ return err;
+ }
+
+ napi_enable(&bp->napi);
+
+ bp->macbgem_ops.mog_init_rings(bp);
+ macb_init_hw(bp);
+
+ /* schedule a link state check */
+ phy_start(dev->phydev);
+
+ netif_tx_start_all_queues(dev);
+
+ if (bp->ptp_info)
+ bp->ptp_info->ptp_init(dev);
+
+ return 0;
+}
+
+static int macb_close(struct net_device *dev)
+{
+ struct macb *bp = netdev_priv(dev);
+ unsigned long flags;
+
+ netif_tx_stop_all_queues(dev);
+ napi_disable(&bp->napi);
+
+ if (dev->phydev)
+ phy_stop(dev->phydev);
+
+ spin_lock_irqsave(&bp->lock, flags);
+ macb_reset_hw(bp);
+ netif_carrier_off(dev);
+ spin_unlock_irqrestore(&bp->lock, flags);
+
+ macb_free_consistent(bp);
+
+ if (bp->ptp_info)
+ bp->ptp_info->ptp_remove(dev);
+
+ return 0;
+}
+
+static int macb_change_mtu(struct net_device *dev, int new_mtu)
+{
+ if (netif_running(dev))
+ return -EBUSY;
+
+ dev->mtu = new_mtu;
+
+ return 0;
+}
+
+static void gem_update_stats(struct macb *bp)
+{
+ unsigned int i;
+ u32 *p = &bp->hw_stats.gem.tx_octets_31_0;
+
+ for (i = 0; i < GEM_STATS_LEN; ++i, ++p) {
+ u32 offset = gem_statistics[i].offset;
+ u64 val = bp->macb_reg_readl(bp, offset);
+
+ bp->ethtool_stats[i] += val;
+ *p += val;
+
+ if (offset == GEM_OCTTXL || offset == GEM_OCTRXL) {
+ /* Add GEM_OCTTXH, GEM_OCTRXH */
+ val = bp->macb_reg_readl(bp, offset + 4);
+ bp->ethtool_stats[i] += ((u64)val) << 32;
+ *(++p) += val;
+ }
+ }
+}
+
+static struct net_device_stats *gem_get_stats(struct macb *bp)
+{
+ struct gem_stats *hwstat = &bp->hw_stats.gem;
+ struct net_device_stats *nstat = &bp->dev->stats;
+
+ gem_update_stats(bp);
+
+ nstat->rx_errors = (hwstat->rx_frame_check_sequence_errors +
+ hwstat->rx_alignment_errors +
+ hwstat->rx_resource_errors +
+ hwstat->rx_overruns +
+ hwstat->rx_oversize_frames +
+ hwstat->rx_jabbers +
+ hwstat->rx_undersized_frames +
+ hwstat->rx_length_field_frame_errors);
+ nstat->tx_errors = (hwstat->tx_late_collisions +
+ hwstat->tx_excessive_collisions +
+ hwstat->tx_underrun +
+ hwstat->tx_carrier_sense_errors);
+ nstat->multicast = hwstat->rx_multicast_frames;
+ nstat->collisions = (hwstat->tx_single_collision_frames +
+ hwstat->tx_multiple_collision_frames +
+ hwstat->tx_excessive_collisions);
+ nstat->rx_length_errors = (hwstat->rx_oversize_frames +
+ hwstat->rx_jabbers +
+ hwstat->rx_undersized_frames +
+ hwstat->rx_length_field_frame_errors);
+ nstat->rx_over_errors = hwstat->rx_resource_errors;
+ nstat->rx_crc_errors = hwstat->rx_frame_check_sequence_errors;
+ nstat->rx_frame_errors = hwstat->rx_alignment_errors;
+ nstat->rx_fifo_errors = hwstat->rx_overruns;
+ nstat->tx_aborted_errors = hwstat->tx_excessive_collisions;
+ nstat->tx_carrier_errors = hwstat->tx_carrier_sense_errors;
+ nstat->tx_fifo_errors = hwstat->tx_underrun;
+
+ return nstat;
+}
+
+static void gem_get_ethtool_stats(struct net_device *dev,
+ struct ethtool_stats *stats, u64 *data)
+{
+ struct macb *bp;
+
+ bp = netdev_priv(dev);
+ gem_update_stats(bp);
+ memcpy(data, &bp->ethtool_stats, sizeof(u64) * GEM_STATS_LEN);
+}
+
+static int gem_get_sset_count(struct net_device *dev, int sset)
+{
+ switch (sset) {
+ case ETH_SS_STATS:
+ return GEM_STATS_LEN;
+ default:
+ return -EOPNOTSUPP;
+ }
+}
+
+static void gem_get_ethtool_strings(struct net_device *dev, u32 sset, u8 *p)
+{
+ unsigned int i;
+
+ switch (sset) {
+ case ETH_SS_STATS:
+ for (i = 0; i < GEM_STATS_LEN; i++, p += ETH_GSTRING_LEN)
+ memcpy(p, gem_statistics[i].stat_string,
+ ETH_GSTRING_LEN);
+ break;
+ }
+}
+
+static struct net_device_stats *macb_get_stats(struct net_device *dev)
+{
+ struct macb *bp = netdev_priv(dev);
+ struct net_device_stats *nstat = &bp->dev->stats;
+ struct macb_stats *hwstat = &bp->hw_stats.macb;
+
+ if (macb_is_gem(bp))
+ return gem_get_stats(bp);
+
+ /* read stats from hardware */
+ macb_update_stats(bp);
+
+ /* Convert HW stats into netdevice stats */
+ nstat->rx_errors = (hwstat->rx_fcs_errors +
+ hwstat->rx_align_errors +
+ hwstat->rx_resource_errors +
+ hwstat->rx_overruns +
+ hwstat->rx_oversize_pkts +
+ hwstat->rx_jabbers +
+ hwstat->rx_undersize_pkts +
+ hwstat->rx_length_mismatch);
+ nstat->tx_errors = (hwstat->tx_late_cols +
+ hwstat->tx_excessive_cols +
+ hwstat->tx_underruns +
+ hwstat->tx_carrier_errors +
+ hwstat->sqe_test_errors);
+ nstat->collisions = (hwstat->tx_single_cols +
+ hwstat->tx_multiple_cols +
+ hwstat->tx_excessive_cols);
+ nstat->rx_length_errors = (hwstat->rx_oversize_pkts +
+ hwstat->rx_jabbers +
+ hwstat->rx_undersize_pkts +
+ hwstat->rx_length_mismatch);
+ nstat->rx_over_errors = hwstat->rx_resource_errors +
+ hwstat->rx_overruns;
+ nstat->rx_crc_errors = hwstat->rx_fcs_errors;
+ nstat->rx_frame_errors = hwstat->rx_align_errors;
+ nstat->rx_fifo_errors = hwstat->rx_overruns;
+ /* XXX: What does "missed" mean? */
+ nstat->tx_aborted_errors = hwstat->tx_excessive_cols;
+ nstat->tx_carrier_errors = hwstat->tx_carrier_errors;
+ nstat->tx_fifo_errors = hwstat->tx_underruns;
+ /* Don't know about heartbeat or window errors... */
+
+ return nstat;
+}
+
+static int macb_get_regs_len(struct net_device *netdev)
+{
+ return MACB_GREGS_NBR * sizeof(u32);
+}
+
+static void macb_get_regs(struct net_device *dev, struct ethtool_regs *regs,
+ void *p)
+{
+ struct macb *bp = netdev_priv(dev);
+ unsigned int tail, head;
+ u32 *regs_buff = p;
+
+ regs->version = (macb_readl(bp, MID) & ((1 << MACB_REV_SIZE) - 1))
+ | MACB_GREGS_VERSION;
+
+ tail = macb_tx_ring_wrap(bp, bp->queues[0].tx_tail);
+ head = macb_tx_ring_wrap(bp, bp->queues[0].tx_head);
+
+ regs_buff[0] = macb_readl(bp, NCR);
+ regs_buff[1] = macb_or_gem_readl(bp, NCFGR);
+ regs_buff[2] = macb_readl(bp, NSR);
+ regs_buff[3] = macb_readl(bp, TSR);
+ regs_buff[4] = macb_readl(bp, RBQP);
+ regs_buff[5] = macb_readl(bp, TBQP);
+ regs_buff[6] = macb_readl(bp, RSR);
+ regs_buff[7] = macb_readl(bp, IMR);
+
+ regs_buff[8] = tail;
+ regs_buff[9] = head;
+ regs_buff[10] = macb_tx_dma(&bp->queues[0], tail);
+ regs_buff[11] = macb_tx_dma(&bp->queues[0], head);
+
+ if (!(bp->caps & MACB_CAPS_USRIO_DISABLED))
+ regs_buff[12] = macb_or_gem_readl(bp, USRIO);
+ if (macb_is_gem(bp))
+ regs_buff[13] = gem_readl(bp, DMACFG);
+}
+
+static void macb_get_wol(struct net_device *netdev, struct ethtool_wolinfo *wol)
+{
+ struct macb *bp = netdev_priv(netdev);
+
+ wol->supported = 0;
+ wol->wolopts = 0;
+
+ if (bp->wol & MACB_WOL_HAS_MAGIC_PACKET) {
+ wol->supported = WAKE_MAGIC;
+
+ if (bp->wol & MACB_WOL_ENABLED)
+ wol->wolopts |= WAKE_MAGIC;
+ }
+}
+
+static int macb_set_wol(struct net_device *netdev, struct ethtool_wolinfo *wol)
+{
+ struct macb *bp = netdev_priv(netdev);
+
+ if (!(bp->wol & MACB_WOL_HAS_MAGIC_PACKET) ||
+ (wol->wolopts & ~WAKE_MAGIC))
+ return -EOPNOTSUPP;
+
+ if (wol->wolopts & WAKE_MAGIC)
+ bp->wol |= MACB_WOL_ENABLED;
+ else
+ bp->wol &= ~MACB_WOL_ENABLED;
+
+ device_set_wakeup_enable(&bp->pdev->dev, bp->wol & MACB_WOL_ENABLED);
+
+ return 0;
+}
+
+static void macb_get_ringparam(struct net_device *netdev,
+ struct ethtool_ringparam *ring)
+{
+ struct macb *bp = netdev_priv(netdev);
+
+ ring->rx_max_pending = MAX_RX_RING_SIZE;
+ ring->tx_max_pending = MAX_TX_RING_SIZE;
+
+ ring->rx_pending = bp->rx_ring_size;
+ ring->tx_pending = bp->tx_ring_size;
+}
+
+static int macb_set_ringparam(struct net_device *netdev,
+ struct ethtool_ringparam *ring)
+{
+ struct macb *bp = netdev_priv(netdev);
+ u32 new_rx_size, new_tx_size;
+ unsigned int reset = 0;
+
+ if ((ring->rx_mini_pending) || (ring->rx_jumbo_pending))
+ return -EINVAL;
+
+ new_rx_size = clamp_t(u32, ring->rx_pending,
+ MIN_RX_RING_SIZE, MAX_RX_RING_SIZE);
+ new_rx_size = roundup_pow_of_two(new_rx_size);
+
+ new_tx_size = clamp_t(u32, ring->tx_pending,
+ MIN_TX_RING_SIZE, MAX_TX_RING_SIZE);
+ new_tx_size = roundup_pow_of_two(new_tx_size);
+
+ if ((new_tx_size == bp->tx_ring_size) &&
+ (new_rx_size == bp->rx_ring_size)) {
+ /* nothing to do */
+ return 0;
+ }
+
+ if (netif_running(bp->dev)) {
+ reset = 1;
+ macb_close(bp->dev);
+ }
+
+ bp->rx_ring_size = new_rx_size;
+ bp->tx_ring_size = new_tx_size;
+
+ if (reset)
+ macb_open(bp->dev);
+
+ return 0;
+}
+
+static int macb_get_ts_info(struct net_device *netdev,
+ struct ethtool_ts_info *info)
+{
+ struct macb *bp = netdev_priv(netdev);
+
+ if (bp->ptp_info)
+ return bp->ptp_info->get_ts_info(netdev, info);
+
+ return ethtool_op_get_ts_info(netdev, info);
+}
+
+static const struct ethtool_ops macb_ethtool_ops = {
+ .get_regs_len = macb_get_regs_len,
+ .get_regs = macb_get_regs,
+ .get_link = ethtool_op_get_link,
+ .get_ts_info = ethtool_op_get_ts_info,
+ .get_wol = macb_get_wol,
+ .set_wol = macb_set_wol,
+ .get_link_ksettings = phy_ethtool_get_link_ksettings,
+ .set_link_ksettings = phy_ethtool_set_link_ksettings,
+ .get_ringparam = macb_get_ringparam,
+ .set_ringparam = macb_set_ringparam,
+};
+
+static const struct ethtool_ops gem_ethtool_ops = {
+ .get_regs_len = macb_get_regs_len,
+ .get_regs = macb_get_regs,
+ .get_link = ethtool_op_get_link,
+ .get_ts_info = macb_get_ts_info,
+ .get_ethtool_stats = gem_get_ethtool_stats,
+ .get_strings = gem_get_ethtool_strings,
+ .get_sset_count = gem_get_sset_count,
+ .get_link_ksettings = phy_ethtool_get_link_ksettings,
+ .set_link_ksettings = phy_ethtool_set_link_ksettings,
+ .get_ringparam = macb_get_ringparam,
+ .set_ringparam = macb_set_ringparam,
+};
+
+static int macb_ioctl(struct net_device *dev, struct ifreq *rq, int cmd)
+{
+ struct phy_device *phydev = dev->phydev;
+ struct macb *bp = netdev_priv(dev);
+
+ if (!netif_running(dev))
+ return -EINVAL;
+
+ if (!phydev)
+ return -ENODEV;
+
+ if (!bp->ptp_info)
+ return phy_mii_ioctl(phydev, rq, cmd);
+
+ switch (cmd) {
+ case SIOCSHWTSTAMP:
+ return bp->ptp_info->set_hwtst(dev, rq, cmd);
+ case SIOCGHWTSTAMP:
+ return bp->ptp_info->get_hwtst(dev, rq);
+ default:
+ return phy_mii_ioctl(phydev, rq, cmd);
+ }
+}
+
+static int macb_set_features(struct net_device *netdev,
+ netdev_features_t features)
+{
+ struct macb *bp = netdev_priv(netdev);
+ netdev_features_t changed = features ^ netdev->features;
+
+ /* TX checksum offload */
+ if ((changed & NETIF_F_HW_CSUM) && macb_is_gem(bp)) {
+ u32 dmacfg;
+
+ dmacfg = gem_readl(bp, DMACFG);
+ if (features & NETIF_F_HW_CSUM)
+ dmacfg |= GEM_BIT(TXCOEN);
+ else
+ dmacfg &= ~GEM_BIT(TXCOEN);
+ gem_writel(bp, DMACFG, dmacfg);
+ }
+
+ /* RX checksum offload */
+ if ((changed & NETIF_F_RXCSUM) && macb_is_gem(bp)) {
+ u32 netcfg;
+
+ netcfg = gem_readl(bp, NCFGR);
+ if (features & NETIF_F_RXCSUM &&
+ !(netdev->flags & IFF_PROMISC))
+ netcfg |= GEM_BIT(RXCOEN);
+ else
+ netcfg &= ~GEM_BIT(RXCOEN);
+ gem_writel(bp, NCFGR, netcfg);
+ }
+
+ return 0;
+}
+
+static const struct net_device_ops macb_netdev_ops = {
+ .ndo_open = macb_open,
+ .ndo_stop = macb_close,
+ .ndo_start_xmit = macb_start_xmit,
+ .ndo_set_rx_mode = macb_set_rx_mode,
+ .ndo_get_stats = macb_get_stats,
+ .ndo_do_ioctl = macb_ioctl,
+ .ndo_validate_addr = eth_validate_addr,
+ .ndo_change_mtu = macb_change_mtu,
+ .ndo_set_mac_address = eth_mac_addr,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = macb_poll_controller,
+#endif
+ .ndo_set_features = macb_set_features,
+ .ndo_features_check = macb_features_check,
+};
+
+/* Configure peripheral capabilities according to device tree
+ * and integration options used
+ */
+static void macb_configure_caps(struct macb *bp,
+ const struct macb_config *dt_conf)
+{
+ u32 dcfg;
+
+ if (dt_conf)
+ bp->caps = dt_conf->caps;
+
+ if (hw_is_gem(bp->regs, bp->native_io)) {
+ bp->caps |= MACB_CAPS_MACB_IS_GEM;
+
+ dcfg = gem_readl(bp, DCFG1);
+ if (GEM_BFEXT(IRQCOR, dcfg) == 0)
+ bp->caps |= MACB_CAPS_ISR_CLEAR_ON_WRITE;
+ dcfg = gem_readl(bp, DCFG2);
+ if ((dcfg & (GEM_BIT(RX_PKT_BUFF) | GEM_BIT(TX_PKT_BUFF))) == 0)
+ bp->caps |= MACB_CAPS_FIFO_MODE;
+ if (IS_ENABLED(CONFIG_MACB_USE_HWSTAMP) && gem_has_ptp(bp)) {
+ if (!GEM_BFEXT(TSU, gem_readl(bp, DCFG5)))
+ pr_err("GEM doesn't support hardware ptp.\n");
+ else
+ bp->hw_dma_cap |= HW_DMA_CAP_PTP;
+ }
+ }
+
+ dev_dbg(&bp->pdev->dev, "Cadence caps 0x%08x\n", bp->caps);
+}
+
+static void macb_probe_queues(void __iomem *mem,
+ bool native_io,
+ unsigned int *queue_mask,
+ unsigned int *num_queues)
+{
+ unsigned int hw_q;
+
+ *queue_mask = 0x1;
+ *num_queues = 1;
+
+ /* is it macb or gem ?
+ *
+ * We need to read directly from the hardware here because
+ * we are early in the probe process and don't have the
+ * MACB_CAPS_MACB_IS_GEM flag positioned
+ */
+ if (!hw_is_gem(mem, native_io))
+ return;
+
+ /* bit 0 is never set but queue 0 always exists */
+ *queue_mask = readl_relaxed(mem + GEM_DCFG6) & 0xff;
+
+ *queue_mask |= 0x1;
+
+ for (hw_q = 1; hw_q < MACB_MAX_QUEUES; ++hw_q)
+ if (*queue_mask & (1 << hw_q))
+ (*num_queues)++;
+}
+
+static int macb_clk_init(struct platform_device *pdev, struct clk **pclk,
+ struct clk **hclk, struct clk **tx_clk,
+ struct clk **rx_clk)
+{
+ struct macb_platform_data *pdata;
+ int err;
+
+ pdata = dev_get_platdata(&pdev->dev);
+ if (pdata) {
+ *pclk = pdata->pclk;
+ *hclk = pdata->hclk;
+ } else {
+ *pclk = devm_clk_get(&pdev->dev, "pclk");
+ *hclk = devm_clk_get(&pdev->dev, "hclk");
+ }
+
+ if (IS_ERR(*pclk)) {
+ err = PTR_ERR(*pclk);
+ dev_err(&pdev->dev, "failed to get macb_clk (%u)\n", err);
+ return err;
+ }
+
+ if (IS_ERR(*hclk)) {
+ err = PTR_ERR(*hclk);
+ dev_err(&pdev->dev, "failed to get hclk (%u)\n", err);
+ return err;
+ }
+
+ *tx_clk = devm_clk_get(&pdev->dev, "tx_clk");
+ if (IS_ERR(*tx_clk))
+ *tx_clk = NULL;
+
+ *rx_clk = devm_clk_get(&pdev->dev, "rx_clk");
+ if (IS_ERR(*rx_clk))
+ *rx_clk = NULL;
+
+ err = clk_prepare_enable(*pclk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable pclk (%u)\n", err);
+ return err;
+ }
+
+ err = clk_prepare_enable(*hclk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable hclk (%u)\n", err);
+ goto err_disable_pclk;
+ }
+
+ err = clk_prepare_enable(*tx_clk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable tx_clk (%u)\n", err);
+ goto err_disable_hclk;
+ }
+
+ err = clk_prepare_enable(*rx_clk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable rx_clk (%u)\n", err);
+ goto err_disable_txclk;
+ }
+
+ return 0;
+
+err_disable_txclk:
+ clk_disable_unprepare(*tx_clk);
+
+err_disable_hclk:
+ clk_disable_unprepare(*hclk);
+
+err_disable_pclk:
+ clk_disable_unprepare(*pclk);
+
+ return err;
+}
+
+static int macb_init(struct platform_device *pdev)
+{
+ struct net_device *dev = platform_get_drvdata(pdev);
+ unsigned int hw_q, q;
+ struct macb *bp = netdev_priv(dev);
+ struct macb_queue *queue;
+ int err;
+ u32 val;
+
+ bp->tx_ring_size = DEFAULT_TX_RING_SIZE;
+ bp->rx_ring_size = DEFAULT_RX_RING_SIZE;
+
+ /* set the queue register mapping once for all: queue0 has a special
+ * register mapping but we don't want to test the queue index then
+ * compute the corresponding register offset at run time.
+ */
+ for (hw_q = 0, q = 0; hw_q < MACB_MAX_QUEUES; ++hw_q) {
+ if (!(bp->queue_mask & (1 << hw_q)))
+ continue;
+
+ queue = &bp->queues[q];
+ queue->bp = bp;
+ if (hw_q) {
+ queue->ISR = GEM_ISR(hw_q - 1);
+ queue->IER = GEM_IER(hw_q - 1);
+ queue->IDR = GEM_IDR(hw_q - 1);
+ queue->IMR = GEM_IMR(hw_q - 1);
+ queue->TBQP = GEM_TBQP(hw_q - 1);
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B)
+ queue->TBQPH = GEM_TBQPH(hw_q - 1);
+#endif
+ } else {
+ /* queue0 uses legacy registers */
+ queue->ISR = MACB_ISR;
+ queue->IER = MACB_IER;
+ queue->IDR = MACB_IDR;
+ queue->IMR = MACB_IMR;
+ queue->TBQP = MACB_TBQP;
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ if (bp->hw_dma_cap & HW_DMA_CAP_64B)
+ queue->TBQPH = MACB_TBQPH;
+#endif
+ }
+
+ /* get irq: here we use the linux queue index, not the hardware
+ * queue index. the queue irq definitions in the device tree
+ * must remove the optional gaps that could exist in the
+ * hardware queue mask.
+ */
+ queue->irq = platform_get_irq(pdev, q);
+ err = devm_request_irq(&pdev->dev, queue->irq, macb_interrupt,
+ IRQF_SHARED, dev->name, queue);
+ if (err) {
+ dev_err(&pdev->dev,
+ "Unable to request IRQ %d (error %d)\n",
+ queue->irq, err);
+ return err;
+ }
+
+ INIT_WORK(&queue->tx_error_task, macb_tx_error_task);
+ q++;
+ }
+
+ dev->netdev_ops = &macb_netdev_ops;
+ netif_napi_add(dev, &bp->napi, macb_poll, 64);
+
+ /* setup appropriated routines according to adapter type */
+ if (macb_is_gem(bp)) {
+ bp->max_tx_length = GEM_MAX_TX_LEN;
+ bp->macbgem_ops.mog_alloc_rx_buffers = gem_alloc_rx_buffers;
+ bp->macbgem_ops.mog_free_rx_buffers = gem_free_rx_buffers;
+ bp->macbgem_ops.mog_init_rings = gem_init_rings;
+ bp->macbgem_ops.mog_rx = gem_rx;
+ dev->ethtool_ops = &gem_ethtool_ops;
+ } else {
+ bp->max_tx_length = MACB_MAX_TX_LEN;
+ bp->macbgem_ops.mog_alloc_rx_buffers = macb_alloc_rx_buffers;
+ bp->macbgem_ops.mog_free_rx_buffers = macb_free_rx_buffers;
+ bp->macbgem_ops.mog_init_rings = macb_init_rings;
+ bp->macbgem_ops.mog_rx = macb_rx;
+ dev->ethtool_ops = &macb_ethtool_ops;
+ }
+
+ /* Set features */
+ dev->hw_features = NETIF_F_SG;
+
+ /* Check LSO capability */
+ if (GEM_BFEXT(PBUF_LSO, gem_readl(bp, DCFG6)))
+ dev->hw_features |= MACB_NETIF_LSO;
+
+ /* Checksum offload is only available on gem with packet buffer */
+ if (macb_is_gem(bp) && !(bp->caps & MACB_CAPS_FIFO_MODE))
+ dev->hw_features |= NETIF_F_HW_CSUM | NETIF_F_RXCSUM;
+ if (bp->caps & MACB_CAPS_SG_DISABLED)
+ dev->hw_features &= ~NETIF_F_SG;
+ dev->features = dev->hw_features;
+
+ if (!(bp->caps & MACB_CAPS_USRIO_DISABLED)) {
+ val = 0;
+ if (bp->phy_interface == PHY_INTERFACE_MODE_RGMII)
+ val = GEM_BIT(RGMII);
+ else if (bp->phy_interface == PHY_INTERFACE_MODE_RMII &&
+ (bp->caps & MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII))
+ val = MACB_BIT(RMII);
+ else if (!(bp->caps & MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII))
+ val = MACB_BIT(MII);
+
+ if (bp->caps & MACB_CAPS_USRIO_HAS_CLKEN)
+ val |= MACB_BIT(CLKEN);
+
+ macb_or_gem_writel(bp, USRIO, val);
+ }
+
+ /* Set MII management clock divider */
+ val = macb_mdc_clk_div(bp);
+ val |= macb_dbw(bp);
+ if (bp->phy_interface == PHY_INTERFACE_MODE_SGMII)
+ val |= GEM_BIT(SGMIIEN) | GEM_BIT(PCSSEL);
+ macb_writel(bp, NCFGR, val);
+
+ return 0;
+}
+
+#if defined(CONFIG_OF)
+/* 1518 rounded up */
+#define AT91ETHER_MAX_RBUFF_SZ 0x600
+/* max number of receive buffers */
+#define AT91ETHER_MAX_RX_DESCR 9
+
+/* Initialize and start the Receiver and Transmit subsystems */
+static int at91ether_start(struct net_device *dev)
+{
+ struct macb *lp = netdev_priv(dev);
+ struct macb_dma_desc *desc;
+ dma_addr_t addr;
+ u32 ctl;
+ int i;
+
+ lp->rx_ring = dma_alloc_coherent(&lp->pdev->dev,
+ (AT91ETHER_MAX_RX_DESCR *
+ macb_dma_desc_get_size(lp)),
+ &lp->rx_ring_dma, GFP_KERNEL);
+ if (!lp->rx_ring)
+ return -ENOMEM;
+
+ lp->rx_buffers = dma_alloc_coherent(&lp->pdev->dev,
+ AT91ETHER_MAX_RX_DESCR *
+ AT91ETHER_MAX_RBUFF_SZ,
+ &lp->rx_buffers_dma, GFP_KERNEL);
+ if (!lp->rx_buffers) {
+ dma_free_coherent(&lp->pdev->dev,
+ AT91ETHER_MAX_RX_DESCR *
+ macb_dma_desc_get_size(lp),
+ lp->rx_ring, lp->rx_ring_dma);
+ lp->rx_ring = NULL;
+ return -ENOMEM;
+ }
+
+ addr = lp->rx_buffers_dma;
+ for (i = 0; i < AT91ETHER_MAX_RX_DESCR; i++) {
+ desc = macb_rx_desc(lp, i);
+ macb_set_addr(lp, desc, addr);
+ desc->ctrl = 0;
+ addr += AT91ETHER_MAX_RBUFF_SZ;
+ }
+
+ /* Set the Wrap bit on the last descriptor */
+ desc->addr |= MACB_BIT(RX_WRAP);
+
+ /* Reset buffer index */
+ lp->rx_tail = 0;
+
+ /* Program address of descriptor list in Rx Buffer Queue register */
+ macb_writel(lp, RBQP, lp->rx_ring_dma);
+
+ /* Enable Receive and Transmit */
+ ctl = macb_readl(lp, NCR);
+ macb_writel(lp, NCR, ctl | MACB_BIT(RE) | MACB_BIT(TE));
+
+ return 0;
+}
+
+/* Open the ethernet interface */
+static int at91ether_open(struct net_device *dev)
+{
+ struct macb *lp = netdev_priv(dev);
+ u32 ctl;
+ int ret;
+
+ /* Clear internal statistics */
+ ctl = macb_readl(lp, NCR);
+ macb_writel(lp, NCR, ctl | MACB_BIT(CLRSTAT));
+
+ macb_set_hwaddr(lp);
+
+ ret = at91ether_start(dev);
+ if (ret)
+ return ret;
+
+ /* Enable MAC interrupts */
+ macb_writel(lp, IER, MACB_BIT(RCOMP) |
+ MACB_BIT(RXUBR) |
+ MACB_BIT(ISR_TUND) |
+ MACB_BIT(ISR_RLE) |
+ MACB_BIT(TCOMP) |
+ MACB_BIT(ISR_ROVR) |
+ MACB_BIT(HRESP));
+
+ /* schedule a link state check */
+ phy_start(dev->phydev);
+
+ netif_start_queue(dev);
+
+ return 0;
+}
+
+/* Close the interface */
+static int at91ether_close(struct net_device *dev)
+{
+ struct macb *lp = netdev_priv(dev);
+ u32 ctl;
+
+ /* Disable Receiver and Transmitter */
+ ctl = macb_readl(lp, NCR);
+ macb_writel(lp, NCR, ctl & ~(MACB_BIT(TE) | MACB_BIT(RE)));
+
+ /* Disable MAC interrupts */
+ macb_writel(lp, IDR, MACB_BIT(RCOMP) |
+ MACB_BIT(RXUBR) |
+ MACB_BIT(ISR_TUND) |
+ MACB_BIT(ISR_RLE) |
+ MACB_BIT(TCOMP) |
+ MACB_BIT(ISR_ROVR) |
+ MACB_BIT(HRESP));
+
+ netif_stop_queue(dev);
+
+ dma_free_coherent(&lp->pdev->dev,
+ AT91ETHER_MAX_RX_DESCR *
+ macb_dma_desc_get_size(lp),
+ lp->rx_ring, lp->rx_ring_dma);
+ lp->rx_ring = NULL;
+
+ dma_free_coherent(&lp->pdev->dev,
+ AT91ETHER_MAX_RX_DESCR * AT91ETHER_MAX_RBUFF_SZ,
+ lp->rx_buffers, lp->rx_buffers_dma);
+ lp->rx_buffers = NULL;
+
+ return 0;
+}
+
+/* Transmit packet */
+static int at91ether_start_xmit(struct sk_buff *skb, struct net_device *dev)
+{
+ struct macb *lp = netdev_priv(dev);
+
+ if (macb_readl(lp, TSR) & MACB_BIT(RM9200_BNQ)) {
+ netif_stop_queue(dev);
+
+ /* Store packet information (to free when Tx completed) */
+ lp->skb = skb;
+ lp->skb_length = skb->len;
+ lp->skb_physaddr = dma_map_single(NULL, skb->data, skb->len,
+ DMA_TO_DEVICE);
+ if (dma_mapping_error(NULL, lp->skb_physaddr)) {
+ dev_kfree_skb_any(skb);
+ dev->stats.tx_dropped++;
+ netdev_err(dev, "%s: DMA mapping error\n", __func__);
+ return NETDEV_TX_OK;
+ }
+
+ /* Set address of the data in the Transmit Address register */
+ macb_writel(lp, TAR, lp->skb_physaddr);
+ /* Set length of the packet in the Transmit Control register */
+ macb_writel(lp, TCR, skb->len);
+
+ } else {
+ netdev_err(dev, "%s called, but device is busy!\n", __func__);
+ return NETDEV_TX_BUSY;
+ }
+
+ return NETDEV_TX_OK;
+}
+
+/* Extract received frame from buffer descriptors and sent to upper layers.
+ * (Called from interrupt context)
+ */
+static void at91ether_rx(struct net_device *dev)
+{
+ struct macb *lp = netdev_priv(dev);
+ struct macb_dma_desc *desc;
+ unsigned char *p_recv;
+ struct sk_buff *skb;
+ unsigned int pktlen;
+
+ desc = macb_rx_desc(lp, lp->rx_tail);
+ while (desc->addr & MACB_BIT(RX_USED)) {
+ p_recv = lp->rx_buffers + lp->rx_tail * AT91ETHER_MAX_RBUFF_SZ;
+ pktlen = MACB_BF(RX_FRMLEN, desc->ctrl);
+ skb = netdev_alloc_skb(dev, pktlen + 2);
+ if (skb) {
+ skb_reserve(skb, 2);
+ memcpy(skb_put(skb, pktlen), p_recv, pktlen);
+
+ skb->protocol = eth_type_trans(skb, dev);
+ dev->stats.rx_packets++;
+ dev->stats.rx_bytes += pktlen;
+ netif_rx(skb);
+ } else {
+ dev->stats.rx_dropped++;
+ }
+
+ if (desc->ctrl & MACB_BIT(RX_MHASH_MATCH))
+ dev->stats.multicast++;
+
+ /* reset ownership bit */
+ desc->addr &= ~MACB_BIT(RX_USED);
+
+ /* wrap after last buffer */
+ if (lp->rx_tail == AT91ETHER_MAX_RX_DESCR - 1)
+ lp->rx_tail = 0;
+ else
+ lp->rx_tail++;
+
+ desc = macb_rx_desc(lp, lp->rx_tail);
+ }
+}
+
+/* MAC interrupt handler */
+static irqreturn_t at91ether_interrupt(int irq, void *dev_id)
+{
+ struct net_device *dev = dev_id;
+ struct macb *lp = netdev_priv(dev);
+ u32 intstatus, ctl;
+
+ /* MAC Interrupt Status register indicates what interrupts are pending.
+ * It is automatically cleared once read.
+ */
+ intstatus = macb_readl(lp, ISR);
+
+ /* Receive complete */
+ if (intstatus & MACB_BIT(RCOMP))
+ at91ether_rx(dev);
+
+ /* Transmit complete */
+ if (intstatus & MACB_BIT(TCOMP)) {
+ /* The TCOM bit is set even if the transmission failed */
+ if (intstatus & (MACB_BIT(ISR_TUND) | MACB_BIT(ISR_RLE)))
+ dev->stats.tx_errors++;
+
+ if (lp->skb) {
+ dev_kfree_skb_irq(lp->skb);
+ lp->skb = NULL;
+ dma_unmap_single(NULL, lp->skb_physaddr,
+ lp->skb_length, DMA_TO_DEVICE);
+ dev->stats.tx_packets++;
+ dev->stats.tx_bytes += lp->skb_length;
+ }
+ netif_wake_queue(dev);
+ }
+
+ /* Work-around for EMAC Errata section 41.3.1 */
+ if (intstatus & MACB_BIT(RXUBR)) {
+ ctl = macb_readl(lp, NCR);
+ macb_writel(lp, NCR, ctl & ~MACB_BIT(RE));
+ wmb();
+ macb_writel(lp, NCR, ctl | MACB_BIT(RE));
+ }
+
+ if (intstatus & MACB_BIT(ISR_ROVR))
+ netdev_err(dev, "ROVR error\n");
+
+ return IRQ_HANDLED;
+}
+
+#ifdef CONFIG_NET_POLL_CONTROLLER
+static void at91ether_poll_controller(struct net_device *dev)
+{
+ unsigned long flags;
+
+ local_irq_save(flags);
+ at91ether_interrupt(dev->irq, dev);
+ local_irq_restore(flags);
+}
+#endif
+
+static const struct net_device_ops at91ether_netdev_ops = {
+ .ndo_open = at91ether_open,
+ .ndo_stop = at91ether_close,
+ .ndo_start_xmit = at91ether_start_xmit,
+ .ndo_get_stats = macb_get_stats,
+ .ndo_set_rx_mode = macb_set_rx_mode,
+ .ndo_set_mac_address = eth_mac_addr,
+ .ndo_do_ioctl = macb_ioctl,
+ .ndo_validate_addr = eth_validate_addr,
+#ifdef CONFIG_NET_POLL_CONTROLLER
+ .ndo_poll_controller = at91ether_poll_controller,
+#endif
+};
+
+static int at91ether_clk_init(struct platform_device *pdev, struct clk **pclk,
+ struct clk **hclk, struct clk **tx_clk,
+ struct clk **rx_clk)
+{
+ int err;
+
+ *hclk = NULL;
+ *tx_clk = NULL;
+ *rx_clk = NULL;
+
+ *pclk = devm_clk_get(&pdev->dev, "ether_clk");
+ if (IS_ERR(*pclk))
+ return PTR_ERR(*pclk);
+
+ err = clk_prepare_enable(*pclk);
+ if (err) {
+ dev_err(&pdev->dev, "failed to enable pclk (%u)\n", err);
+ return err;
+ }
+
+ return 0;
+}
+
+static int at91ether_init(struct platform_device *pdev)
+{
+ struct net_device *dev = platform_get_drvdata(pdev);
+ struct macb *bp = netdev_priv(dev);
+ int err;
+ u32 reg;
+
+ dev->netdev_ops = &at91ether_netdev_ops;
+ dev->ethtool_ops = &macb_ethtool_ops;
+
+ err = devm_request_irq(&pdev->dev, dev->irq, at91ether_interrupt,
+ 0, dev->name, dev);
+ if (err)
+ return err;
+
+ macb_writel(bp, NCR, 0);
+
+ reg = MACB_BF(CLK, MACB_CLK_DIV32) | MACB_BIT(BIG);
+ if (bp->phy_interface == PHY_INTERFACE_MODE_RMII)
+ reg |= MACB_BIT(RM9200_RMII);
+
+ macb_writel(bp, NCFGR, reg);
+
+ return 0;
+}
+
+static const struct macb_config at91sam9260_config = {
+ .caps = MACB_CAPS_USRIO_HAS_CLKEN | MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII,
+ .clk_init = macb_clk_init,
+ .init = macb_init,
+};
+
+static const struct macb_config pc302gem_config = {
+ .caps = MACB_CAPS_SG_DISABLED | MACB_CAPS_GIGABIT_MODE_AVAILABLE,
+ .dma_burst_length = 16,
+ .clk_init = macb_clk_init,
+ .init = macb_init,
+};
+
+static const struct macb_config sama5d2_config = {
+ .caps = MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII,
+ .dma_burst_length = 16,
+ .clk_init = macb_clk_init,
+ .init = macb_init,
+};
+
+static const struct macb_config sama5d3_config = {
+ .caps = MACB_CAPS_SG_DISABLED | MACB_CAPS_GIGABIT_MODE_AVAILABLE
+ | MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII,
+ .dma_burst_length = 16,
+ .clk_init = macb_clk_init,
+ .init = macb_init,
+};
+
+static const struct macb_config sama5d4_config = {
+ .caps = MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII,
+ .dma_burst_length = 4,
+ .clk_init = macb_clk_init,
+ .init = macb_init,
+};
+
+static const struct macb_config emac_config = {
+ .clk_init = at91ether_clk_init,
+ .init = at91ether_init,
+};
+
+static const struct macb_config np4_config = {
+ .caps = MACB_CAPS_USRIO_DISABLED,
+ .clk_init = macb_clk_init,
+ .init = macb_init,
+};
+
+static const struct macb_config zynqmp_config = {
+ .caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE | MACB_CAPS_JUMBO,
+ .dma_burst_length = 16,
+ .clk_init = macb_clk_init,
+ .init = macb_init,
+ .jumbo_max_len = 10240,
+};
+
+static const struct macb_config zynq_config = {
+ .caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE | MACB_CAPS_NO_GIGABIT_HALF,
+ .dma_burst_length = 16,
+ .clk_init = macb_clk_init,
+ .init = macb_init,
+};
+
+static const struct of_device_id macb_dt_ids[] = {
+ { .compatible = "cdns,at32ap7000-macb" },
+ { .compatible = "cdns,at91sam9260-macb", .data = &at91sam9260_config },
+ { .compatible = "cdns,macb" },
+ { .compatible = "cdns,np4-macb", .data = &np4_config },
+ { .compatible = "cdns,pc302-gem", .data = &pc302gem_config },
+ { .compatible = "cdns,gem", .data = &pc302gem_config },
+ { .compatible = "atmel,sama5d2-gem", .data = &sama5d2_config },
+ { .compatible = "atmel,sama5d3-gem", .data = &sama5d3_config },
+ { .compatible = "atmel,sama5d4-gem", .data = &sama5d4_config },
+ { .compatible = "cdns,at91rm9200-emac", .data = &emac_config },
+ { .compatible = "cdns,emac", .data = &emac_config },
+ { .compatible = "cdns,zynqmp-gem", .data = &zynqmp_config},
+ { .compatible = "cdns,zynq-gem", .data = &zynq_config },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, macb_dt_ids);
+#endif /* CONFIG_OF */
+
+static const struct macb_config default_gem_config = {
+ .caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE | MACB_CAPS_JUMBO,
+ .dma_burst_length = 16,
+ .clk_init = macb_clk_init,
+ .init = macb_init,
+ .jumbo_max_len = 10240,
+};
+
+static int macb_probe(struct platform_device *pdev)
+{
+ const struct macb_config *macb_config = &default_gem_config;
+ int (*clk_init)(struct platform_device *, struct clk **,
+ struct clk **, struct clk **, struct clk **)
+ = macb_config->clk_init;
+ int (*init)(struct platform_device *) = macb_config->init;
+ struct device_node *np = pdev->dev.of_node;
+ struct device_node *phy_node;
+ struct clk *pclk, *hclk = NULL, *tx_clk = NULL, *rx_clk = NULL;
+ unsigned int queue_mask, num_queues;
+ struct macb_platform_data *pdata;
+ bool native_io;
+ struct phy_device *phydev;
+ struct net_device *dev;
+ struct resource *regs;
+ void __iomem *mem;
+ const char *mac;
+ struct macb *bp;
+ int err;
+
+ regs = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ mem = devm_ioremap_resource(&pdev->dev, regs);
+ if (IS_ERR(mem))
+ return PTR_ERR(mem);
+
+ if (np) {
+ const struct of_device_id *match;
+
+ match = of_match_node(macb_dt_ids, np);
+ if (match && match->data) {
+ macb_config = match->data;
+ clk_init = macb_config->clk_init;
+ init = macb_config->init;
+ }
+ }
+
+ err = clk_init(pdev, &pclk, &hclk, &tx_clk, &rx_clk);
+ if (err)
+ return err;
+
+ native_io = hw_is_native_io(mem);
+
+ macb_probe_queues(mem, native_io, &queue_mask, &num_queues);
+ dev = alloc_etherdev_mq(sizeof(*bp), num_queues);
+ if (!dev) {
+ err = -ENOMEM;
+ goto err_disable_clocks;
+ }
+
+ dev->base_addr = regs->start;
+
+ SET_NETDEV_DEV(dev, &pdev->dev);
+
+ bp = netdev_priv(dev);
+ bp->pdev = pdev;
+ bp->dev = dev;
+ bp->regs = mem;
+ bp->native_io = native_io;
+ if (native_io) {
+ bp->macb_reg_readl = hw_readl_native;
+ bp->macb_reg_writel = hw_writel_native;
+ } else {
+ bp->macb_reg_readl = hw_readl;
+ bp->macb_reg_writel = hw_writel;
+ }
+ bp->num_queues = num_queues;
+ bp->queue_mask = queue_mask;
+ if (macb_config)
+ bp->dma_burst_length = macb_config->dma_burst_length;
+ bp->pclk = pclk;
+ bp->hclk = hclk;
+ bp->tx_clk = tx_clk;
+ bp->rx_clk = rx_clk;
+ if (macb_config)
+ bp->jumbo_max_len = macb_config->jumbo_max_len;
+
+ bp->wol = 0;
+ if (of_get_property(np, "magic-packet", NULL))
+ bp->wol |= MACB_WOL_HAS_MAGIC_PACKET;
+ device_init_wakeup(&pdev->dev, bp->wol & MACB_WOL_HAS_MAGIC_PACKET);
+
+ spin_lock_init(&bp->lock);
+
+ /* setup capabilities */
+ macb_configure_caps(bp, macb_config);
+
+#ifdef CONFIG_ARCH_DMA_ADDR_T_64BIT
+ if (GEM_BFEXT(DAW64, gem_readl(bp, DCFG6))) {
+ dma_set_mask(&pdev->dev, DMA_BIT_MASK(44));
+ bp->hw_dma_cap |= HW_DMA_CAP_64B;
+ }
+#endif
+ platform_set_drvdata(pdev, dev);
+
+ dev->irq = platform_get_irq(pdev, 0);
+ if (dev->irq < 0) {
+ err = dev->irq;
+ goto err_out_free_netdev;
+ }
+
+ /* MTU range: 68 - 1500 or 10240 */
+ dev->min_mtu = GEM_MTU_MIN_SIZE;
+ if (bp->caps & MACB_CAPS_JUMBO)
+ dev->max_mtu = gem_readl(bp, JML) - ETH_HLEN - ETH_FCS_LEN;
+ else
+ dev->max_mtu = ETH_DATA_LEN;
+
+ mac = of_get_mac_address(np);
+ if (mac)
+ ether_addr_copy(bp->dev->dev_addr, mac);
+ else
+ macb_get_hwaddr(bp);
+
+ /* Power up the PHY if there is a GPIO reset */
+ phy_node = of_get_next_available_child(np, NULL);
+ if (phy_node) {
+ int gpio = of_get_named_gpio(phy_node, "reset-gpios", 0);
+
+ if (gpio_is_valid(gpio)) {
+ bp->reset_gpio = gpio_to_desc(gpio);
+ gpiod_direction_output(bp->reset_gpio, 1);
+ }
+ }
+ of_node_put(phy_node);
+
+ err = of_get_phy_mode(np);
+ if (err < 0) {
+ pdata = dev_get_platdata(&pdev->dev);
+ if (pdata && pdata->is_rmii)
+ bp->phy_interface = PHY_INTERFACE_MODE_RMII;
+ else
+ bp->phy_interface = PHY_INTERFACE_MODE_MII;
+ } else {
+ bp->phy_interface = err;
+ }
+
+ /* IP specific init */
+ err = init(pdev);
+ if (err)
+ goto err_out_free_netdev;
+
+ err = macb_mii_init(bp);
+ if (err)
+ goto err_out_free_netdev;
+
+ phydev = dev->phydev;
+
+ netif_carrier_off(dev);
+
+ err = register_netdev(dev);
+ if (err) {
+ dev_err(&pdev->dev, "Cannot register net device, aborting.\n");
+ goto err_out_unregister_mdio;
+ }
+
+ phy_attached_info(phydev);
+
+ netdev_info(dev, "Cadence %s rev 0x%08x at 0x%08lx irq %d (%pM)\n",
+ macb_is_gem(bp) ? "GEM" : "MACB", macb_readl(bp, MID),
+ dev->base_addr, dev->irq, dev->dev_addr);
+
+ return 0;
+
+err_out_unregister_mdio:
+ phy_disconnect(dev->phydev);
+ mdiobus_unregister(bp->mii_bus);
+ mdiobus_free(bp->mii_bus);
+
+ /* Shutdown the PHY if there is a GPIO reset */
+ if (bp->reset_gpio)
+ gpiod_set_value(bp->reset_gpio, 0);
+
+err_out_free_netdev:
+ free_netdev(dev);
+
+err_disable_clocks:
+ clk_disable_unprepare(tx_clk);
+ clk_disable_unprepare(hclk);
+ clk_disable_unprepare(pclk);
+ clk_disable_unprepare(rx_clk);
+
+ return err;
+}
+
+static int macb_remove(struct platform_device *pdev)
+{
+ struct net_device *dev;
+ struct macb *bp;
+
+ dev = platform_get_drvdata(pdev);
+
+ if (dev) {
+ bp = netdev_priv(dev);
+ if (dev->phydev)
+ phy_disconnect(dev->phydev);
+ mdiobus_unregister(bp->mii_bus);
+ dev->phydev = NULL;
+ mdiobus_free(bp->mii_bus);
+
+ /* Shutdown the PHY if there is a GPIO reset */
+ if (bp->reset_gpio)
+ gpiod_set_value(bp->reset_gpio, 0);
+
+ unregister_netdev(dev);
+ clk_disable_unprepare(bp->tx_clk);
+ clk_disable_unprepare(bp->hclk);
+ clk_disable_unprepare(bp->pclk);
+ clk_disable_unprepare(bp->rx_clk);
+ free_netdev(dev);
+ }
+
+ return 0;
+}
+
+static int __maybe_unused macb_suspend(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct net_device *netdev = platform_get_drvdata(pdev);
+ struct macb *bp = netdev_priv(netdev);
+
+ netif_carrier_off(netdev);
+ netif_device_detach(netdev);
+
+ if (bp->wol & MACB_WOL_ENABLED) {
+ macb_writel(bp, IER, MACB_BIT(WOL));
+ macb_writel(bp, WOL, MACB_BIT(MAG));
+ enable_irq_wake(bp->queues[0].irq);
+ } else {
+ clk_disable_unprepare(bp->tx_clk);
+ clk_disable_unprepare(bp->hclk);
+ clk_disable_unprepare(bp->pclk);
+ clk_disable_unprepare(bp->rx_clk);
+ }
+
+ return 0;
+}
+
+static int __maybe_unused macb_resume(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct net_device *netdev = platform_get_drvdata(pdev);
+ struct macb *bp = netdev_priv(netdev);
+
+ if (bp->wol & MACB_WOL_ENABLED) {
+ macb_writel(bp, IDR, MACB_BIT(WOL));
+ macb_writel(bp, WOL, 0);
+ disable_irq_wake(bp->queues[0].irq);
+ } else {
+ clk_prepare_enable(bp->pclk);
+ clk_prepare_enable(bp->hclk);
+ clk_prepare_enable(bp->tx_clk);
+ clk_prepare_enable(bp->rx_clk);
+ }
+
+ netif_device_attach(netdev);
+
+ return 0;
+}
+
+static SIMPLE_DEV_PM_OPS(macb_pm_ops, macb_suspend, macb_resume);
+
+static struct platform_driver macb_driver = {
+ .probe = macb_probe,
+ .remove = macb_remove,
+ .driver = {
+ .name = "macb",
+ .of_match_table = of_match_ptr(macb_dt_ids),
+ .pm = &macb_pm_ops,
+ },
+};
+
+module_platform_driver(macb_driver);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("Cadence MACB/GEM Ethernet driver");
+MODULE_AUTHOR("Haavard Skinnemoen (Atmel)");
+MODULE_ALIAS("platform:macb");
--
2.4.5
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH 4/4] net: macb: Add hardware PTP support
From: Rafal Ozieblo @ 2017-05-09 9:27 UTC (permalink / raw)
To: David Miller, nicolas.ferre, Richard Cochran, netdev,
linux-kernel, devicetree, linux-arm-kernel, harini.katakam,
andrei.pistirica
Cc: Rafal Ozieblo
In-Reply-To: <1494321885-14384-1-git-send-email-rafalo@cadence.com>
This patch is based on original Harini's patch and Andrei's patch,
implemented in a separate file to ease the review/maintanance
and integration with other platforms.
This driver supports GEM-GXL:
- Register ptp clock framework
- Initialize PTP related registers
- HW time stamp on the PTP Ethernet packets are received using the
SO_TIMESTAMPING API. Time stamps are obtained from the dma buffer
descriptors
- add macb_ptp to compilation chain
Signed-off-by: Rafal Ozieblo <rafalo@cadence.com>
---
drivers/net/ethernet/cadence/Makefile | 4 +
drivers/net/ethernet/cadence/macb.h | 126 ++++++++
drivers/net/ethernet/cadence/macb_main.c | 91 +++++-
drivers/net/ethernet/cadence/macb_ptp.c | 512 +++++++++++++++++++++++++++++++
4 files changed, 727 insertions(+), 6 deletions(-)
create mode 100755 drivers/net/ethernet/cadence/macb_ptp.c
diff --git a/drivers/net/ethernet/cadence/Makefile b/drivers/net/ethernet/cadence/Makefile
index 31ea6e3..1d66ddb 100644
--- a/drivers/net/ethernet/cadence/Makefile
+++ b/drivers/net/ethernet/cadence/Makefile
@@ -3,5 +3,9 @@
#
macb-y := macb_main.o
+ifeq ($(CONFIG_MACB_USE_HWSTAMP),y)
+macb-y += macb_ptp.o
+endif
+
obj-$(CONFIG_MACB) += macb.o
obj-$(CONFIG_MACB_PCI) += macb_pci.o
diff --git a/drivers/net/ethernet/cadence/macb.h b/drivers/net/ethernet/cadence/macb.h
index 4359b08..4285daa6 100644
--- a/drivers/net/ethernet/cadence/macb.h
+++ b/drivers/net/ethernet/cadence/macb.h
@@ -11,6 +11,8 @@
#define _MACB_H
#include <linux/phy.h>
+#include <linux/ptp_clock_kernel.h>
+#include <linux/net_tstamp.h>
#if defined(CONFIG_ARCH_DMA_ADDR_T_64BIT) || defined(CONFIG_MACB_USE_HWSTAMP)
#define MACB_EXT_DESC
@@ -90,6 +92,10 @@
#define GEM_SA3T 0x009C /* Specific3 Top */
#define GEM_SA4B 0x00A0 /* Specific4 Bottom */
#define GEM_SA4T 0x00A4 /* Specific4 Top */
+#define GEM_EFTSH 0x00e8 /* PTP Event Frame Transmitted Seconds Register 47:32 */
+#define GEM_EFRSH 0x00ec /* PTP Event Frame Received Seconds Register 47:32 */
+#define GEM_PEFTSH 0x00f0 /* PTP Peer Event Frame Transmitted Seconds Register 47:32 */
+#define GEM_PEFRSH 0x00f4 /* PTP Peer Event Frame Received Seconds Register 47:32 */
#define GEM_OTX 0x0100 /* Octets transmitted */
#define GEM_OCTTXL 0x0100 /* Octets transmitted [31:0] */
#define GEM_OCTTXH 0x0104 /* Octets transmitted [47:32] */
@@ -159,6 +165,9 @@
#define GEM_DCFG6 0x0294 /* Design Config 6 */
#define GEM_DCFG7 0x0298 /* Design Config 7 */
+#define GEM_TXBDCTRL 0x04cc /* TX Buffer Descriptor control register */
+#define GEM_RXBDCTRL 0x04d0 /* RX Buffer Descriptor control register */
+
#define GEM_ISR(hw_q) (0x0400 + ((hw_q) << 2))
#define GEM_TBQP(hw_q) (0x0440 + ((hw_q) << 2))
#define GEM_TBQPH(hw_q) (0x04C8)
@@ -195,6 +204,8 @@
#define MACB_TZQ_OFFSET 12 /* Transmit zero quantum pause frame */
#define MACB_TZQ_SIZE 1
#define MACB_SRTSM_OFFSET 15
+#define MACB_OSSMODE_OFFSET 24 /* Enable One Step Synchro Mode */
+#define MACB_OSSMODE_SIZE 1
/* Bitfields in NCFGR */
#define MACB_SPD_OFFSET 0 /* Speed */
@@ -452,6 +463,52 @@
#define GEM_NSINCR_OFFSET 0
#define GEM_NSINCR_SIZE 8
+/* Bitfields in TSH */
+#define GEM_TSH_OFFSET 0 /* TSU timer value (s). MSB [47:32] of seconds timer count */
+#define GEM_TSH_SIZE 16
+
+/* Bitfields in TSL */
+#define GEM_TSL_OFFSET 0 /* TSU timer value (s). LSB [31:0] of seconds timer count */
+#define GEM_TSL_SIZE 32
+
+/* Bitfields in TN */
+#define GEM_TN_OFFSET 0 /* TSU timer value (ns) */
+#define GEM_TN_SIZE 30
+
+/* Bitfields in TXBDCTRL */
+#define GEM_TXTSMODE_OFFSET 4 /* TX Descriptor Timestamp Insertion mode */
+#define GEM_TXTSMODE_SIZE 2
+
+/* Bitfields in RXBDCTRL */
+#define GEM_RXTSMODE_OFFSET 4 /* RX Descriptor Timestamp Insertion mode */
+#define GEM_RXTSMODE_SIZE 2
+
+/* Transmit DMA buffer descriptor Word 1 */
+#define GEM_DMA_TXVALID_OFFSET 23 /* timestamp has been captured in the Buffer Descriptor */
+#define GEM_DMA_TXVALID_SIZE 1
+
+/* Receive DMA buffer descriptor Word 0 */
+#define GEM_DMA_RXVALID_OFFSET 2 /* indicates a valid timestamp in the Buffer Descriptor */
+#define GEM_DMA_RXVALID_SIZE 1
+
+/* DMA buffer descriptor Word 2 (32 bit addressing) or Word 4 (64 bit addressing) */
+#define GEM_DMA_SECL_OFFSET 30 /* Timestamp seconds[1:0] */
+#define GEM_DMA_SECL_SIZE 2
+#define GEM_DMA_NSEC_OFFSET 0 /* Timestamp nanosecs [29:0] */
+#define GEM_DMA_NSEC_SIZE 30
+
+/* DMA buffer descriptor Word 3 (32 bit addressing) or Word 5 (64 bit addressing) */
+
+/* New hardware supports 12 bit precision of timestamp in DMA buffer descriptor.
+ * Old hardware supports only 6 bit precision but it is enough for PTP.
+ * Less accuracy is used always instead of checking hardware version.
+ */
+#define GEM_DMA_SECH_OFFSET 0 /* Timestamp seconds[5:2] */
+#define GEM_DMA_SECH_SIZE 4
+#define GEM_DMA_SEC_WIDTH (GEM_DMA_SECH_SIZE + GEM_DMA_SECL_SIZE)
+#define GEM_DMA_SEC_TOP (1 << GEM_DMA_SEC_WIDTH)
+#define GEM_DMA_SEC_MASK (GEM_DMA_SEC_TOP - 1)
+
/* Bitfields in ADJ */
#define GEM_ADDSUB_OFFSET 31
#define GEM_ADDSUB_SIZE 1
@@ -527,6 +584,8 @@
#define queue_readl(queue, reg) (queue)->bp->macb_reg_readl((queue)->bp, (queue)->reg)
#define queue_writel(queue, reg, value) (queue)->bp->macb_reg_writel((queue)->bp, (queue)->reg, (value))
+#define PTP_TS_BUFFER_SIZE 128 /* must be power of 2 */
+
/* Conditional GEM/MACB macros. These perform the operation to the correct
* register dependent on whether the device is a GEM or a MACB. For registers
* and bitfields that are common across both devices, use macb_{read,write}l
@@ -574,6 +633,11 @@ struct macb_dma_desc_ptp {
u32 ts_1;
u32 ts_2;
};
+
+struct gem_tx_ts {
+ struct sk_buff *skb;
+ struct macb_dma_desc_ptp desc_ptp;
+};
#endif
/* DMA descriptor bitfields */
@@ -889,6 +953,11 @@ struct macb_config {
int jumbo_max_len;
};
+struct tsu_incr {
+ u32 sub_ns;
+ u32 ns;
+};
+
struct macb_queue {
struct macb *bp;
int irq;
@@ -905,6 +974,12 @@ struct macb_queue {
struct macb_tx_skb *tx_skb;
dma_addr_t tx_ring_dma;
struct work_struct tx_error_task;
+
+#ifdef CONFIG_MACB_USE_HWSTAMP
+ struct work_struct tx_ts_task;
+ unsigned int tx_ts_head, tx_ts_tail;
+ struct gem_tx_ts tx_timestamps[PTP_TS_BUFFER_SIZE];
+#endif
};
struct macb {
@@ -975,8 +1050,59 @@ struct macb {
#ifdef MACB_EXT_DESC
uint8_t hw_dma_cap;
#endif
+ spinlock_t tsu_clk_lock; /* gem tsu clock locking */
+ unsigned int tsu_rate;
+ struct ptp_clock *ptp_clock;
+ struct ptp_clock_info ptp_clock_info;
+ struct tsu_incr tsu_incr;
+ struct hwtstamp_config tstamp_config;
};
+#ifdef CONFIG_MACB_USE_HWSTAMP
+#define GEM_TSEC_SIZE (GEM_TSH_SIZE + GEM_TSL_SIZE)
+#define TSU_SEC_MAX_VAL (((u64)1 << GEM_TSEC_SIZE) - 1)
+#define TSU_NSEC_MAX_VAL ((1 << GEM_TN_SIZE) - 1)
+
+enum macb_bd_control {
+ TSTAMP_DISABLED,
+ TSTAMP_FRAME_PTP_EVENT_ONLY,
+ TSTAMP_ALL_PTP_FRAMES,
+ TSTAMP_ALL_FRAMES,
+};
+
+void gem_ptp_init(struct net_device *ndev);
+void gem_ptp_remove(struct net_device *ndev);
+int gem_ptp_txstamp(struct macb_queue *queue, struct sk_buff *skb, struct macb_dma_desc *des);
+void gem_ptp_rxstamp(struct macb *bp, struct sk_buff *skb, struct macb_dma_desc *desc);
+static inline int gem_ptp_do_txstamp(struct macb_queue *queue, struct sk_buff *skb, struct macb_dma_desc *desc)
+{
+ if (queue->bp->tstamp_config.tx_type == TSTAMP_DISABLED)
+ return -ENOTSUPP;
+
+ return gem_ptp_txstamp(queue, skb, desc);
+}
+
+static inline void gem_ptp_do_rxstamp(struct macb *bp, struct sk_buff *skb, struct macb_dma_desc *desc)
+{
+ if (bp->tstamp_config.rx_filter == TSTAMP_DISABLED)
+ return;
+
+ gem_ptp_rxstamp(bp, skb, desc);
+}
+int gem_get_hwtst(struct net_device *dev, struct ifreq *rq);
+int gem_set_hwtst(struct net_device *dev, struct ifreq *ifr, int cmd);
+#else
+static inline void gem_ptp_init(struct net_device *ndev) { }
+static inline void gem_ptp_remove(struct net_device *ndev) { }
+
+static inline int gem_ptp_do_txstamp(struct macb_queue *queue, struct sk_buff *skb, struct macb_dma_desc *desc)
+{
+ return -1;
+}
+
+static inline void gem_ptp_do_rxstamp(struct macb *bp, struct sk_buff *skb, struct macb_dma_desc *desc) { }
+#endif
+
static inline bool macb_is_gem(struct macb *bp)
{
return !!(bp->caps & MACB_CAPS_MACB_IS_GEM);
diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
index 3151429..87a6580 100644
--- a/drivers/net/ethernet/cadence/macb_main.c
+++ b/drivers/net/ethernet/cadence/macb_main.c
@@ -828,6 +828,12 @@ static void macb_tx_interrupt(struct macb_queue *queue)
/* First, update TX stats if needed */
if (skb) {
+ if (gem_ptp_do_txstamp(queue, skb, desc) == 0) {
+ /* skb now belongs to timestamp buffer
+ * and will be removed later
+ */
+ tx_skb->skb = NULL;
+ }
netdev_vdbg(bp->dev, "skb %u (data %p) TX complete\n",
macb_tx_ring_wrap(bp, tail),
skb->data);
@@ -994,6 +1000,8 @@ static int gem_rx(struct macb *bp, int budget)
bp->dev->stats.rx_packets++;
bp->dev->stats.rx_bytes += skb->len;
+ gem_ptp_do_rxstamp(bp, skb, desc);
+
#if defined(DEBUG) && defined(VERBOSE_DEBUG)
netdev_vdbg(bp->dev, "received skb of length %u, csum: %08x\n",
skb->len, skb->csum);
@@ -1315,7 +1323,6 @@ static irqreturn_t macb_interrupt(int irq, void *dev_id)
if (bp->caps & MACB_CAPS_ISR_CLEAR_ON_WRITE)
queue_writel(queue, ISR, MACB_BIT(HRESP));
}
-
status = queue_readl(queue, ISR);
}
@@ -1645,7 +1652,6 @@ static int macb_start_xmit(struct sk_buff *skb, struct net_device *dev)
/* Make newly initialized descriptor visible to hardware */
wmb();
-
skb_tx_timestamp(skb);
macb_writel(bp, NCR, macb_readl(bp, NCR) | MACB_BIT(TSTART));
@@ -2503,6 +2509,71 @@ static int macb_set_ringparam(struct net_device *netdev,
return 0;
}
+#ifdef CONFIG_MACB_USE_HWSTAMP
+static unsigned int gem_get_tsu_rate(struct macb *bp)
+{
+ struct clk *tsu_clk;
+ unsigned int tsu_rate;
+
+ tsu_clk = devm_clk_get(&bp->pdev->dev, "tsu_clk");
+ if (!IS_ERR(tsu_clk))
+ tsu_rate = clk_get_rate(tsu_clk);
+ /* try pclk instead */
+ else if (!IS_ERR(bp->pclk)) {
+ tsu_clk = bp->pclk;
+ tsu_rate = clk_get_rate(tsu_clk);
+ } else
+ return -ENOTSUPP;
+ return tsu_rate;
+}
+
+static s32 gem_get_ptp_max_adj(void)
+{
+ return 64E6;
+}
+
+static int gem_get_ts_info(struct net_device *dev,
+ struct ethtool_ts_info *info)
+{
+ struct macb *bp = netdev_priv(dev);
+
+ ethtool_op_get_ts_info(dev, info);
+ if ((bp->hw_dma_cap & HW_DMA_CAP_PTP) == 0)
+ return 0;
+
+ info->so_timestamping =
+ SOF_TIMESTAMPING_TX_SOFTWARE |
+ SOF_TIMESTAMPING_RX_SOFTWARE |
+ SOF_TIMESTAMPING_SOFTWARE |
+ SOF_TIMESTAMPING_TX_HARDWARE |
+ SOF_TIMESTAMPING_RX_HARDWARE |
+ SOF_TIMESTAMPING_RAW_HARDWARE;
+ info->tx_types =
+ (1 << HWTSTAMP_TX_ONESTEP_SYNC) |
+ (1 << HWTSTAMP_TX_OFF) |
+ (1 << HWTSTAMP_TX_ON);
+ info->rx_filters =
+ (1 << HWTSTAMP_FILTER_NONE) |
+ (1 << HWTSTAMP_FILTER_ALL);
+ info->phc_index = -1;
+
+ if (bp->ptp_clock)
+ info->phc_index = ptp_clock_index(bp->ptp_clock);
+
+ return 0;
+}
+
+static struct macb_ptp_info gem_ptp_info = {
+ .ptp_init = gem_ptp_init,
+ .ptp_remove = gem_ptp_remove,
+ .get_ptp_max_adj = gem_get_ptp_max_adj,
+ .get_tsu_rate = gem_get_tsu_rate,
+ .get_ts_info = gem_get_ts_info,
+ .get_hwtst = gem_get_hwtst,
+ .set_hwtst = gem_set_hwtst,
+};
+#endif
+
static int macb_get_ts_info(struct net_device *netdev,
struct ethtool_ts_info *info)
{
@@ -2636,12 +2707,16 @@ static void macb_configure_caps(struct macb *bp,
dcfg = gem_readl(bp, DCFG2);
if ((dcfg & (GEM_BIT(RX_PKT_BUFF) | GEM_BIT(TX_PKT_BUFF))) == 0)
bp->caps |= MACB_CAPS_FIFO_MODE;
- if (IS_ENABLED(CONFIG_MACB_USE_HWSTAMP) && gem_has_ptp(bp)) {
+#ifdef CONFIG_MACB_USE_HWSTAMP
+ if (gem_has_ptp(bp)) {
if (!GEM_BFEXT(TSU, gem_readl(bp, DCFG5)))
pr_err("GEM doesn't support hardware ptp.\n");
- else
+ else {
bp->hw_dma_cap |= HW_DMA_CAP_PTP;
+ bp->ptp_info = &gem_ptp_info;
+ }
}
+#endif
}
dev_dbg(&bp->pdev->dev, "Cadence caps 0x%08x\n", bp->caps);
@@ -3247,7 +3322,9 @@ static const struct macb_config np4_config = {
};
static const struct macb_config zynqmp_config = {
- .caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE | MACB_CAPS_JUMBO,
+ .caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE |
+ MACB_CAPS_JUMBO |
+ MACB_CAPS_GEM_HAS_PTP,
.dma_burst_length = 16,
.clk_init = macb_clk_init,
.init = macb_init,
@@ -3281,7 +3358,9 @@ MODULE_DEVICE_TABLE(of, macb_dt_ids);
#endif /* CONFIG_OF */
static const struct macb_config default_gem_config = {
- .caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE | MACB_CAPS_JUMBO,
+ .caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE |
+ MACB_CAPS_JUMBO |
+ MACB_CAPS_GEM_HAS_PTP,
.dma_burst_length = 16,
.clk_init = macb_clk_init,
.init = macb_init,
diff --git a/drivers/net/ethernet/cadence/macb_ptp.c b/drivers/net/ethernet/cadence/macb_ptp.c
new file mode 100755
index 0000000..d536970
--- /dev/null
+++ b/drivers/net/ethernet/cadence/macb_ptp.c
@@ -0,0 +1,512 @@
+/**
+ * 1588 PTP support for Cadence GEM device.
+ *
+ * Copyright (C) 2017 Cadence Design Systems - http://www.cadence.com
+ *
+ * Authors: Rafal Ozieblo <rafalo@cadence.com>
+ * Bartosz Folta <bfolta@cadence.com>
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 of
+ * the License as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#include <linux/kernel.h>
+#include <linux/types.h>
+#include <linux/clk.h>
+#include <linux/device.h>
+#include <linux/etherdevice.h>
+#include <linux/platform_device.h>
+#include <linux/time64.h>
+#include <linux/ptp_classify.h>
+#include <linux/if_ether.h>
+#include <linux/if_vlan.h>
+#include <linux/net_tstamp.h>
+#include <linux/circ_buf.h>
+#include <linux/spinlock.h>
+
+#include "macb.h"
+
+#define GEM_PTP_TIMER_NAME "gem-ptp-timer"
+
+static struct macb_dma_desc_ptp *macb_ptp_desc(struct macb *bp,
+ struct macb_dma_desc *desc)
+{
+ if (bp->hw_dma_cap == HW_DMA_CAP_PTP)
+ return (struct macb_dma_desc_ptp *)
+ ((u8 *)desc + sizeof(struct macb_dma_desc));
+ if (bp->hw_dma_cap == HW_DMA_CAP_64B_PTP)
+ return (struct macb_dma_desc_ptp *)
+ ((u8 *)desc + sizeof(struct macb_dma_desc)
+ + sizeof(struct macb_dma_desc_64));
+ return NULL;
+}
+
+static int gem_tsu_get_time(struct ptp_clock_info *ptp, struct timespec64 *ts)
+{
+ long first, second;
+ u32 secl, sech;
+ unsigned long flags;
+ struct macb *bp = container_of(ptp, struct macb, ptp_clock_info);
+
+ spin_lock_irqsave(&bp->tsu_clk_lock, flags);
+ first = gem_readl(bp, TN);
+ secl = gem_readl(bp, TSL);
+ sech = gem_readl(bp, TSH);
+ second = gem_readl(bp, TN);
+
+ /* test for nsec rollover */
+ if (first > second) {
+ /* if so, use later read & re-read seconds
+ * (assume all done within 1s)
+ */
+ ts->tv_nsec = gem_readl(bp, TN);
+ secl = gem_readl(bp, TSL);
+ sech = gem_readl(bp, TSH);
+ } else {
+ ts->tv_nsec = first;
+ }
+
+ spin_unlock_irqrestore(&bp->tsu_clk_lock, flags);
+ ts->tv_sec = (((u64)sech << GEM_TSL_SIZE) | secl)
+ & TSU_SEC_MAX_VAL;
+ return 0;
+}
+
+static int gem_tsu_set_time(struct ptp_clock_info *ptp,
+ const struct timespec64 *ts)
+{
+ u32 ns, sech, secl;
+ unsigned long flags;
+ struct macb *bp = container_of(ptp, struct macb, ptp_clock_info);
+
+ secl = (u32)ts->tv_sec;
+ sech = (ts->tv_sec >> GEM_TSL_SIZE) & ((1 << GEM_TSH_SIZE) - 1);
+ ns = ts->tv_nsec;
+
+ spin_lock_irqsave(&bp->tsu_clk_lock, flags);
+
+ /* TSH doesn't latch the time and no atomicity! */
+ gem_writel(bp, TN, 0); /* clear to avoid overflow */
+ gem_writel(bp, TSH, sech);
+ /* write lower bits 2nd, for synchronized secs update */
+ gem_writel(bp, TSL, secl);
+ gem_writel(bp, TN, ns);
+
+ spin_unlock_irqrestore(&bp->tsu_clk_lock, flags);
+
+ return 0;
+}
+
+static int gem_tsu_incr_set(struct macb *bp, struct tsu_incr *incr_spec)
+{
+ unsigned long flags;
+
+ /* tsu_timer_incr register must be written after
+ * the tsu_timer_incr_sub_ns register and the write operation
+ * will cause the value written to the tsu_timer_incr_sub_ns register
+ * to take effect.
+ */
+ spin_lock_irqsave(&bp->tsu_clk_lock, flags);
+ gem_writel(bp, TISUBN, GEM_BF(SUBNSINCR, incr_spec->sub_ns));
+ gem_writel(bp, TI, GEM_BF(NSINCR, incr_spec->ns));
+ spin_unlock_irqrestore(&bp->tsu_clk_lock, flags);
+
+ return 0;
+}
+
+static int gem_ptp_adjfine(struct ptp_clock_info *ptp, long scaled_ppm)
+{
+ struct tsu_incr incr_spec;
+ struct macb *bp = container_of(ptp, struct macb, ptp_clock_info);
+ u64 adj;
+ u32 word;
+ bool neg_adj = false;
+
+ if (scaled_ppm < 0) {
+ neg_adj = true;
+ scaled_ppm = -scaled_ppm;
+ }
+
+ /* Adjustment is relative to base frequency */
+ incr_spec.sub_ns = bp->tsu_incr.sub_ns;
+ incr_spec.ns = bp->tsu_incr.ns;
+
+ /* scaling: unused(8bit) | ns(8bit) | fractions(16bit) */
+ word = ((u64)incr_spec.ns << GEM_SUBNSINCR_SIZE) + incr_spec.sub_ns;
+ adj = (u64)scaled_ppm * word;
+ /* Divide with rounding, equivalent to floating dividing:
+ * (temp / USEC_PER_SEC) + 0.5
+ */
+ adj += (USEC_PER_SEC >> 1);
+ adj >>= GEM_SUBNSINCR_SIZE; /* remove fractions */
+ adj = div_u64(adj, USEC_PER_SEC);
+ adj = neg_adj ? (word - adj) : (word + adj);
+
+ incr_spec.ns = (adj >> GEM_SUBNSINCR_SIZE)
+ & ((1 << GEM_NSINCR_SIZE) - 1);
+ incr_spec.sub_ns = adj & ((1 << GEM_SUBNSINCR_SIZE) - 1);
+ gem_tsu_incr_set(bp, &incr_spec);
+ return 0;
+}
+
+static int gem_ptp_adjtime(struct ptp_clock_info *ptp, s64 delta)
+{
+ struct timespec64 now, then = ns_to_timespec64(delta);
+ struct macb *bp = container_of(ptp, struct macb, ptp_clock_info);
+ u32 adj, sign = 0;
+
+ if (delta < 0) {
+ sign = 1;
+ delta = -delta;
+ }
+
+ if (delta > TSU_NSEC_MAX_VAL) {
+ gem_tsu_get_time(&bp->ptp_clock_info, &now);
+ if (sign)
+ now = timespec64_sub(now, then);
+ else
+ now = timespec64_add(now, then);
+
+ gem_tsu_set_time(&bp->ptp_clock_info,
+ (const struct timespec64 *)&now);
+ } else {
+ adj = (sign << GEM_ADDSUB_OFFSET) | delta;
+
+ gem_writel(bp, TA, adj);
+ }
+
+ return 0;
+}
+
+static int gem_ptp_enable(struct ptp_clock_info *ptp,
+ struct ptp_clock_request *rq, int on)
+{
+ return -EOPNOTSUPP;
+}
+
+static struct ptp_clock_info gem_ptp_caps_template = {
+ .owner = THIS_MODULE,
+ .name = GEM_PTP_TIMER_NAME,
+ .max_adj = 0,
+ .n_alarm = 0,
+ .n_ext_ts = 0,
+ .n_per_out = 0,
+ .n_pins = 0,
+ .pps = 1,
+ .adjfine = gem_ptp_adjfine,
+ .adjtime = gem_ptp_adjtime,
+ .gettime64 = gem_tsu_get_time,
+ .settime64 = gem_tsu_set_time,
+ .enable = gem_ptp_enable,
+};
+
+static void gem_ptp_init_timer(struct macb *bp)
+{
+ u32 rem = 0;
+
+ bp->tsu_incr.ns = div_u64_rem(NSEC_PER_SEC, bp->tsu_rate, &rem);
+ if (rem) {
+ u64 adj = rem;
+
+ adj <<= GEM_SUBNSINCR_SIZE;
+ bp->tsu_incr.sub_ns = div_u64(adj, bp->tsu_rate);
+ } else {
+ bp->tsu_incr.sub_ns = 0;
+ }
+}
+
+static void gem_ptp_init_tsu(struct macb *bp)
+{
+ struct timespec64 ts;
+
+ /* 1. get current system time */
+ ts = ns_to_timespec64(ktime_to_ns(ktime_get_real()));
+
+ /* 2. set ptp timer */
+ gem_tsu_set_time(&bp->ptp_clock_info, &ts);
+
+ /* 3. set PTP timer increment value to BASE_INCREMENT */
+ gem_tsu_incr_set(bp, &bp->tsu_incr);
+
+ gem_writel(bp, TA, 0);
+}
+
+static void gem_ptp_clear_timer(struct macb *bp)
+{
+ bp->tsu_incr.ns = 0;
+ bp->tsu_incr.sub_ns = 0;
+
+ gem_writel(bp, TISUBN, GEM_BF(SUBNSINCR, 0));
+ gem_writel(bp, TI, GEM_BF(NSINCR, 0));
+ gem_writel(bp, TA, 0);
+}
+
+static int gem_hw_timestamp(struct macb *bp, u32 dma_desc_ts_1,
+ u32 dma_desc_ts_2, struct timespec64 *ts)
+{
+ struct timespec64 tsu;
+
+ ts->tv_sec = (GEM_BFEXT(DMA_SECH, dma_desc_ts_2) << GEM_DMA_SECL_SIZE) |
+ GEM_BFEXT(DMA_SECL, dma_desc_ts_1);
+ ts->tv_nsec = GEM_BFEXT(DMA_NSEC, dma_desc_ts_1);
+
+ /* TSU overlapping workaround
+ * The timestamp only contains lower few bits of seconds,
+ * so add value from 1588 timer
+ */
+ gem_tsu_get_time(&bp->ptp_clock_info, &tsu);
+
+ /* If the top bit is set in the timestamp,
+ * but not in 1588 timer, it has rolled over,
+ * so subtract max size
+ */
+ if ((ts->tv_sec & (GEM_DMA_SEC_TOP >> 1)) &&
+ !(tsu.tv_sec & (GEM_DMA_SEC_TOP >> 1)))
+ ts->tv_sec -= GEM_DMA_SEC_TOP;
+
+ ts->tv_sec += ((~GEM_DMA_SEC_MASK) & tsu.tv_sec);
+
+ return 0;
+}
+
+void gem_ptp_rxstamp(struct macb *bp, struct sk_buff *skb,
+ struct macb_dma_desc *desc)
+{
+ struct timespec64 ts;
+ struct skb_shared_hwtstamps *shhwtstamps = skb_hwtstamps(skb);
+ struct macb_dma_desc_ptp *desc_ptp;
+
+ if (GEM_BFEXT(DMA_RXVALID, desc->addr)) {
+ desc_ptp = macb_ptp_desc(bp, desc);
+ gem_hw_timestamp(bp, desc_ptp->ts_1, desc_ptp->ts_2, &ts);
+ memset(shhwtstamps, 0, sizeof(struct skb_shared_hwtstamps));
+ shhwtstamps->hwtstamp = ktime_set(ts.tv_sec, ts.tv_nsec);
+ }
+}
+
+static void gem_tstamp_tx(struct macb *bp, struct sk_buff *skb,
+ struct macb_dma_desc_ptp *desc_ptp)
+{
+ struct skb_shared_hwtstamps shhwtstamps;
+ struct timespec64 ts;
+
+ gem_hw_timestamp(bp, desc_ptp->ts_1, desc_ptp->ts_2, &ts);
+ memset(&shhwtstamps, 0, sizeof(shhwtstamps));
+ shhwtstamps.hwtstamp = ktime_set(ts.tv_sec, ts.tv_nsec);
+ skb_tstamp_tx(skb, &shhwtstamps);
+}
+
+int gem_ptp_txstamp(struct macb_queue *queue, struct sk_buff *skb,
+ struct macb_dma_desc *desc)
+{
+ struct gem_tx_ts *tx_timestamp;
+ struct macb_dma_desc_ptp *desc_ptp;
+ unsigned long head = queue->tx_ts_head;
+ unsigned long tail = READ_ONCE(queue->tx_ts_tail);
+
+ if (!GEM_BFEXT(DMA_TXVALID, desc->ctrl))
+ return -EINVAL;
+
+ if (CIRC_SPACE(head, tail, PTP_TS_BUFFER_SIZE) == 0)
+ return -ENOMEM;
+
+ skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
+ desc_ptp = macb_ptp_desc(queue->bp, desc);
+ tx_timestamp = &queue->tx_timestamps[head];
+ tx_timestamp->skb = skb;
+ tx_timestamp->desc_ptp.ts_1 = desc_ptp->ts_1;
+ tx_timestamp->desc_ptp.ts_2 = desc_ptp->ts_2;
+ /* move head */
+ smp_store_release(&queue->tx_ts_head,
+ (head + 1) & (PTP_TS_BUFFER_SIZE - 1));
+
+ schedule_work(&queue->tx_ts_task);
+ return 0;
+}
+
+static void gem_tx_timestamp_flush(struct work_struct *work)
+{
+ struct macb_queue *queue =
+ container_of(work, struct macb_queue, tx_ts_task);
+ struct gem_tx_ts *tx_ts;
+ unsigned long head, tail;
+
+ /* take current head */
+ head = smp_load_acquire(&queue->tx_ts_head);
+ tail = queue->tx_ts_tail;
+
+ while (CIRC_CNT(head, tail, PTP_TS_BUFFER_SIZE)) {
+ tx_ts = &queue->tx_timestamps[tail];
+ gem_tstamp_tx(queue->bp, tx_ts->skb, &tx_ts->desc_ptp);
+ /* cleanup */
+ dev_kfree_skb_any(tx_ts->skb);
+ /* remove old tail */
+ smp_store_release(&queue->tx_ts_tail,
+ (tail + 1) & (PTP_TS_BUFFER_SIZE - 1));
+ tail = queue->tx_ts_tail;
+ }
+}
+
+void gem_ptp_init(struct net_device *dev)
+{
+ struct macb *bp = netdev_priv(dev);
+ unsigned int q;
+ struct macb_queue *queue;
+
+ bp->ptp_clock_info = gem_ptp_caps_template;
+
+ /* nominal frequency and maximum adjustment in ppb */
+ bp->tsu_rate = bp->ptp_info->get_tsu_rate(bp);
+ bp->ptp_clock_info.max_adj = bp->ptp_info->get_ptp_max_adj();
+ gem_ptp_init_timer(bp);
+ bp->ptp_clock = ptp_clock_register(&bp->ptp_clock_info, &dev->dev);
+ if (IS_ERR(&bp->ptp_clock)) {
+ bp->ptp_clock = NULL;
+ pr_err("ptp clock register failed\n");
+ return;
+ }
+
+ spin_lock_init(&bp->tsu_clk_lock);
+ for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
+ queue->tx_ts_head = 0;
+ queue->tx_ts_tail = 0;
+ INIT_WORK(&queue->tx_ts_task, gem_tx_timestamp_flush);
+ }
+
+ gem_ptp_init_tsu(bp);
+
+ dev_info(&bp->pdev->dev, "%s ptp clock registered.\n",
+ GEM_PTP_TIMER_NAME);
+}
+
+void gem_ptp_remove(struct net_device *ndev)
+{
+ struct macb *bp = netdev_priv(ndev);
+
+ if (bp->ptp_clock)
+ ptp_clock_unregister(bp->ptp_clock);
+
+ gem_ptp_clear_timer(bp);
+
+ dev_info(&bp->pdev->dev, "%s ptp clock unregistered.\n",
+ GEM_PTP_TIMER_NAME);
+}
+
+static int gem_ptp_set_ts_mode(struct macb *bp,
+ enum macb_bd_control tx_bd_control,
+ enum macb_bd_control rx_bd_control)
+{
+ gem_writel(bp, TXBDCTRL, GEM_BF(TXTSMODE, tx_bd_control));
+ gem_writel(bp, RXBDCTRL, GEM_BF(RXTSMODE, rx_bd_control));
+
+ return 0;
+}
+
+int gem_get_hwtst(struct net_device *dev, struct ifreq *rq)
+{
+ struct macb *bp = netdev_priv(dev);
+ struct hwtstamp_config *tstamp_config = &bp->tstamp_config;
+
+ if ((bp->hw_dma_cap & HW_DMA_CAP_PTP) == 0)
+ return -EFAULT;
+
+ if (copy_to_user(rq->ifr_data, tstamp_config, sizeof(*tstamp_config)))
+ return -EFAULT;
+ else
+ return 0;
+}
+
+static int gem_ptp_set_one_step_sync(struct macb *bp, u8 enable)
+{
+ u32 reg_val;
+
+ reg_val = macb_readl(bp, NCR);
+
+ if (enable)
+ macb_writel(bp, NCR, reg_val | MACB_BIT(OSSMODE));
+ else
+ macb_writel(bp, NCR, reg_val & ~MACB_BIT(OSSMODE));
+
+ return 0;
+}
+
+int gem_set_hwtst(struct net_device *dev, struct ifreq *ifr, int cmd)
+{
+ struct macb *bp = netdev_priv(dev);
+ struct hwtstamp_config *tstamp_config = &bp->tstamp_config;
+ enum macb_bd_control tx_bd_control = TSTAMP_DISABLED;
+ enum macb_bd_control rx_bd_control = TSTAMP_DISABLED;
+ u32 regval;
+
+ if ((bp->hw_dma_cap & HW_DMA_CAP_PTP) == 0)
+ return -EFAULT;
+
+ if (copy_from_user(tstamp_config, ifr->ifr_data,
+ sizeof(*tstamp_config)))
+ return -EFAULT;
+
+ /* reserved for future extensions */
+ if (tstamp_config->flags)
+ return -EINVAL;
+
+ switch (tstamp_config->tx_type) {
+ case HWTSTAMP_TX_OFF:
+ break;
+ case HWTSTAMP_TX_ONESTEP_SYNC:
+ if (gem_ptp_set_one_step_sync(bp, 1) != 0)
+ return -ERANGE;
+ case HWTSTAMP_TX_ON:
+ tx_bd_control = TSTAMP_ALL_FRAMES;
+ break;
+ default:
+ return -ERANGE;
+ }
+
+ switch (tstamp_config->rx_filter) {
+ case HWTSTAMP_FILTER_NONE:
+ break;
+ case HWTSTAMP_FILTER_PTP_V1_L4_SYNC:
+ break;
+ case HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ:
+ break;
+ case HWTSTAMP_FILTER_PTP_V2_EVENT:
+ case HWTSTAMP_FILTER_PTP_V2_L2_EVENT:
+ case HWTSTAMP_FILTER_PTP_V2_L4_EVENT:
+ case HWTSTAMP_FILTER_PTP_V2_SYNC:
+ case HWTSTAMP_FILTER_PTP_V2_L2_SYNC:
+ case HWTSTAMP_FILTER_PTP_V2_L4_SYNC:
+ case HWTSTAMP_FILTER_PTP_V2_DELAY_REQ:
+ case HWTSTAMP_FILTER_PTP_V2_L2_DELAY_REQ:
+ case HWTSTAMP_FILTER_PTP_V2_L4_DELAY_REQ:
+ rx_bd_control = TSTAMP_ALL_PTP_FRAMES;
+ tstamp_config->rx_filter = HWTSTAMP_FILTER_PTP_V2_EVENT;
+ regval = macb_readl(bp, NCR);
+ macb_writel(bp, NCR, (regval | MACB_BIT(SRTSM)));
+ break;
+ case HWTSTAMP_FILTER_PTP_V1_L4_EVENT:
+ case HWTSTAMP_FILTER_ALL:
+ rx_bd_control = TSTAMP_ALL_FRAMES;
+ tstamp_config->rx_filter = HWTSTAMP_FILTER_ALL;
+ break;
+ default:
+ tstamp_config->rx_filter = HWTSTAMP_FILTER_NONE;
+ return -ERANGE;
+ }
+
+ if (gem_ptp_set_ts_mode(bp, tx_bd_control, rx_bd_control) != 0)
+ return -ERANGE;
+
+ if (copy_to_user(ifr->ifr_data, tstamp_config, sizeof(*tstamp_config)))
+ return -EFAULT;
+ else
+ return 0;
+}
+
--
2.4.5
^ permalink raw reply related
* [PATCH v2 net] ipv4: Fix misplaced EXPORT_SYMBOL_GPL(ping_hash) in net/ipv4/ping.c
From: Vladis Dronov @ 2017-05-09 9:39 UTC (permalink / raw)
To: netdev; +Cc: Vladis Dronov
Signed-off-by: Vladis Dronov <vdronov@redhat.com>
---
This is quite a smaill patch, please, feel free not to accept in separately,
but use as a part of any patch of yours.
net/ipv4/ping.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/ipv4/ping.c b/net/ipv4/ping.c
index ccfbce1..19f0b7b 100644
--- a/net/ipv4/ping.c
+++ b/net/ipv4/ping.c
@@ -71,7 +71,6 @@ static inline u32 ping_hashfn(const struct net *net, u32 num, u32 mask)
pr_debug("hash(%u) = %u\n", num, res);
return res;
}
-EXPORT_SYMBOL_GPL(ping_hash);
static inline struct hlist_nulls_head *ping_hashslot(struct ping_table *table,
struct net *net, unsigned int num)
@@ -152,6 +151,7 @@ int ping_hash(struct sock *sk)
return 0;
}
+EXPORT_SYMBOL_GPL(ping_hash);
void ping_unhash(struct sock *sk)
{
--
2.9.3
^ permalink raw reply related
* Re: [PATCH 0/6] cxgb4: Fine-tuning for some function implementations
From: Ganesh GR @ 2017-05-09 9:40 UTC (permalink / raw)
To: SF Markus Elfring, netdev@vger.kernel.org, Casey Leedom
Cc: LKML, kernel-janitors@vger.kernel.org
In-Reply-To: <a3216042-2461-b397-dfa1-1de178d58513@users.sourceforge.net>
Acked-by: Ganesh Goudar <ganeshgr@chelsio.com>
From: SF Markus Elfring <elfring@users.sourceforge.net>
Sent: Friday, May 5, 2017 2:00 AM
To: netdev@vger.kernel.org; Casey Leedom; Ganesh GR
Cc: LKML; kernel-janitors@vger.kernel.org
Subject: [PATCH 0/6] cxgb4: Fine-tuning for some function implementations
From: Markus Elfring <elfring@users.sourceforge.net>
Date: Thu, 4 May 2017 22:23:45 +0200
A few update suggestions were taken into account
from static source code analysis.
Markus Elfring (6):
Use seq_putc() in mboxlog_show()
Combine substrings for 24 messages
Adjust five checks for null pointers
Replace seven seq_puts() calls by seq_putc()
Use seq_puts() in cim_qcfg_show()
Combine substrings for two messages
drivers/net/ethernet/chelsio/cxgb4/cxgb4_debugfs.c | 42 +++----
.../net/ethernet/chelsio/cxgb4vf/cxgb4vf_main.c | 125 ++++++++++++---------
2 files changed, 86 insertions(+), 81 deletions(-)
--
2.12.2
^ permalink raw reply
* Re:Re: [PATCH net] driver: vrf: Fix one possible use-after-free issue
From: Gao Feng @ 2017-05-09 9:41 UTC (permalink / raw)
To: Florian Westphal; +Cc: dsa, shm, davem, netdev, gfree.wind@foxmail.com
In-Reply-To: <20170509092102.GB16263@breakpoint.cc>
At 2017-05-09 17:21:02, "Florian Westphal" <fw@strlen.de> wrote:
>gfree.wind@vip.163.com <gfree.wind@vip.163.com> wrote:
>> When one netfilter rule or hook stoles the skb and return NF_STOLEN,
>> it means the skb is taken by the rule, and other modules should not
>> touch this skb ever. Maybe the skb is queued or freed directly by the
>> rule.
>>
>> Now uses the nf_hook instead of NF_HOOK to get the result of netfilter,
>> and check the return value of nf_hook. Only when its value equals 1, it
>> means the skb could go ahead. Or reset the skb as NULL.
>>
>> BTW, because vrf_rcv_finish is empty function, so needn't invoke it
>> even though nf_hook returns 1.
>
>Thats a bug then.
>
>The okfn (if called) takes ownership of skb and must free it eventually.
>Otherwise userspace queueing leaks skb on reinjection.
>
>(see nf_reinject() and its use of okfn()).
Thanks, I only thought about the stolen case like synproxy which would free the skb directly,
and forget the userspace could reinject the skb.
I would update the patch.
Best Regards
Feng
^ permalink raw reply
* Re: [PATCH RFC net-next 0/6] net: reducing memory footprint of network devices
From: Nicolas Dichtel @ 2017-05-09 9:50 UTC (permalink / raw)
To: Florian Fainelli, David Ahern, netdev; +Cc: roopa
In-Reply-To: <89333e30-0a5d-0f05-4d14-d75dc157d6ba@gmail.com>
Le 08/05/2017 à 19:35, Florian Fainelli a écrit :
> On 05/06/2017 09:07 AM, David Ahern wrote:
>> As I have mentioned many times[1], at ~43+kB per instance the use of
>> net_devices does not scale for deployments needing 10,000+ devices. At
>> netconf 1.2 there was a discussion about using a net_device_common for
>> the minimal set of common attributes with other structs built on top of
>> that one for "full" devices. It provided a means for the code to know
>> "non-standard" net_devices. Conceptually, that approach has its merits
>> but it is not practical given the sweeping changes required to the code
>> base. More importantly though struct net_device is not the problem; it
>> weighs in at less than 2kB so reorganizing the code base around a
>> refactored net_device is not going to solve the problem. The primary
>> issue is all of the initializations done *because* it is a struct
>> net_device -- kobject and sysfs and the protocols (e.g., ipv4, ipv6,
>> mpls, neighbors).
>>
>> So, how do you keep the desired attributes of a net device -- network
>> addresses, xmit function, qdisc, netfilter rules, tcpdump -- while
>> lowering the overhead of a net_device instance and without sweeping
>> changes across net/ and drivers/net/?
>>
>> This patch set introduces the concept of labeling net_devices as
>> "lightweight", first mentioned at netdev 1.1 [1]. Users have to opt
>> in to lightweight devices by passing a new attribute, IFLA_LWT_NETDEV,
>> in the new link request. This lightweight tag is meant for virtual
>> devices such as vlan, vrf, vti, and dummy where the user expects to
>> create a lot of them and does not want the duplication of resources.
>> Each device type can always opt out of a lightweight label if necessary
>> by failing device creates.
>>
>> Labeling a virtual device as "lightweight" reduces the footprint for
>> device creation from ~43kB to ~6kB. That reduction in memory is obtained
>> by:
>> 1. no entry in sysfs
>> - kobject in net_device.device is not initialized
>>
>> 2. no entry in procfs
>> - no sysctl option for these devices
>>
>> 3. deferred ipv4, ipv6, mpls initialization
>> - network layer must be enabled before an address can be assigned
>> or mpls labels can be processed
>> - enables what Florian called L2 only devices [2]
>>
>> Once the core premise of a lightweight device is accepted, follow on
>> patches can reduce the overhead of network initializations. e.g.,
>>
>> 1. remove devconf per device (ipv4 and ipv6)
>> - lightweight devices use the default settings rather than replicate
>> the same data for each device
>>
>> 2. reduce / remove / opt out of snmp mibs
>> - snmp6_alloc_dev and icmpv6msg_mib_device specifically is a heavy
>> hitter
>>
>> Patches can also be found here:
>> https://github.com/dsahern/linux lwt-dev-rfc
>>
>> And iproute2 here:
>> https://github.com/dsahern/iproute2 lwt-dev
>>
>> Example:
>> ip li add foo lwd type vrf table 123
>>
>> - creates VRF device 'foo' as a lightweight netdevice.
>
> This is really looking nice, thanks for posting this patch series! The
> only submission wide comment I have is that the flag is named
> IFF_LWT_NETDEV whereas the helper that checks for it is named
> netif_is_lwd() so we should reconcile the two. Since there is an
> existing lightweight tunnel infrastructure already, maybe using
> IFF_LWD_NETDEV (or just IFF_LWD) would be good enough here?
Yep, thank you for the series, it also looks good to me.
I also vote for the IFF_LWD_NETDEV or IFF_LWD to avoid confusion with
lightweight tunnel and to be consistent with it (lightweight was abbreviated lw,
not lwt ;-)).
Your initial patch tried to make those interfaces transparent, this is not the
case anymore here. It would probably be useful to be able to filter those
interfaces in the kernel during a dump.
Regards,
Nicolas
^ permalink raw reply
* [PATCH net v2] driver: vrf: Fix one possible use-after-free issue
From: gfree.wind @ 2017-05-09 10:27 UTC (permalink / raw)
To: dsa, shm, davem, fw, netdev; +Cc: Gao Feng
From: Gao Feng <gfree.wind@vip.163.com>
The current codes only deal with the case that the skb is dropped, it
may meet one use-after-free issue when NF_HOOK returns 0 that means
the skb is stolen by one netfilter rule or hook.
When one netfilter rule or hook stoles the skb and return NF_STOLEN,
it means the skb is taken by the rule, and other modules should not
touch this skb ever. Maybe the skb is queued or freed directly by the
rule.
Now uses the nf_hook instead of NF_HOOK to get the result of netfilter,
and check the return value of nf_hook. Only when its value equals 1, it
means the skb could go ahead. Or reset the skb as NULL.
BTW, because vrf_rcv_finish is empty function, so needn't invoke it
even though nf_hook returns 1. But we need to modify vrf_rcv_finish
to deal with the NF_STOLEN case.
There are two cases when skb is stolen.
1. The skb is stolen and freed directly.
There is nothing we need to do, and vrf_rcv_finish isn't invoked.
2. The skb is queued and reinjected again.
The vrf_rcv_finish would be invoked as okfn, so need to free the
skb in it.
Signed-off-by: Gao Feng <gfree.wind@vip.163.com>
---
v2: Free the skb in vrf_rcv_finish, per Florian
v1: initial version
drivers/net/vrf.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/net/vrf.c b/drivers/net/vrf.c
index ceda586..db88249 100644
--- a/drivers/net/vrf.c
+++ b/drivers/net/vrf.c
@@ -989,6 +989,7 @@ static u32 vrf_fib_table(const struct net_device *dev)
static int vrf_rcv_finish(struct net *net, struct sock *sk, struct sk_buff *skb)
{
+ kfree_skb(skb);
return 0;
}
@@ -998,7 +999,7 @@ static struct sk_buff *vrf_rcv_nfhook(u8 pf, unsigned int hook,
{
struct net *net = dev_net(dev);
- if (NF_HOOK(pf, hook, net, NULL, skb, dev, NULL, vrf_rcv_finish) < 0)
+ if (nf_hook(pf, hook, net, NULL, skb, dev, NULL, vrf_rcv_finish) != 1)
skb = NULL; /* kfree_skb(skb) handled by nf code */
return skb;
--
1.9.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox