* Re: [PATCH 1/1] booke/wdt: fix incorrect WDIOC_GETSUPPORT return path
From: tiejun.chen @ 2012-08-06 3:10 UTC (permalink / raw)
To: benh, galak; +Cc: linuxppc-dev, linux-watchdog
In-Reply-To: <1343636122-23273-1-git-send-email-tiejun.chen@windriver.com>
On 07/30/2012 04:15 PM, Tiejun Chen wrote:
> We miss that correct WDIOC_GETSUPPORT return path when perform
> copy_to_user() properly.
Any comments?
Thanks
Tiejun
>
> Signed-off-by: Tiejun Chen <tiejun.chen@windriver.com>
> ---
> drivers/watchdog/booke_wdt.c | 7 ++++---
> 1 files changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/watchdog/booke_wdt.c b/drivers/watchdog/booke_wdt.c
> index 3fe82d0..2be7f29 100644
> --- a/drivers/watchdog/booke_wdt.c
> +++ b/drivers/watchdog/booke_wdt.c
> @@ -162,12 +162,13 @@ static long booke_wdt_ioctl(struct file *file,
> unsigned int cmd, unsigned long arg)
> {
> u32 tmp = 0;
> - u32 __user *p = (u32 __user *)arg;
> + void __user *argp = (u32 __user *)arg;
> + u32 __user *p = argp;
>
> switch (cmd) {
> case WDIOC_GETSUPPORT:
> - if (copy_to_user((void *)arg, &ident, sizeof(ident)))
> - return -EFAULT;
> + return copy_to_user(argp, &ident,
> + sizeof(ident)) ? -EFAULT : 0;
> case WDIOC_GETSTATUS:
> return put_user(0, p);
> case WDIOC_GETBOOTSTATUS:
>
^ permalink raw reply
* Re: [PATCH V4 3/3] powerpc/fsl-pci: Unify pci/pcie initialization code
From: Tabi Timur-B04825 @ 2012-08-06 4:11 UTC (permalink / raw)
To: Wood Scott-B07421
Cc: Wood Scott-B07421, linuxppc-dev@lists.ozlabs.org, Li Yang-R58472,
Jia Hongtao-B38951
In-Reply-To: <501BF7BC.1030208@freescale.com>
On Fri, Aug 3, 2012 at 11:09 AM, Scott Wood <scottwood@freescale.com> wrote=
:
> p1022ds OTOH is weird enough that it deserves its own board file.
What's so weird about the P1022DS?
Also, why do we need a default PCI bus if one isn't specified in the
device tree?
--=20
Timur Tabi
Linux kernel developer at Freescale=
^ permalink raw reply
* how to implement platform specific per process parameter?
From: Alexey Kardashevskiy @ 2012-08-06 5:08 UTC (permalink / raw)
To: Linux Kernel Mailing List, linuxppc-dev
Hi!
There is a per core register on modern POWER processors (POWER6+) called
"DSCR": The DSCR (aka Data Stream Control Register) is supported on some
server PowerPC chips and allow some control over the prefetch of data
streams.
Its "per process" support has been added by
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commitdiff;h=efcac6589a277c10060e4be44b9455cf43838dc1
"powerpc: Per process DSCR + some fixes (try#4)"
The patch adds the DSCR value into "struct thread_struct" and
saves/restores it when the context is switched. Also, there is some policy
to initialize the register when a thread starts. So far so good.
Here is the problem.
Some customer(s) of this feature want an ability to change the DSCR value
of the already running process. I.e. we need some kernel interface which
would accept pid and the register value and change it.
There are 4 options:
1. [not a kernel interface] use ptrace to execute the register changing
command inside the specified pid. The next context switch saves the new
value in the thread_struct. Dirty hack.
2. Add a new syscall which would receive pid + register value and do the
job. A bit too much.
3. Add some hook in /proc filesystem but so far there were no platform
specific bits, right?
4. Implement a static node "/sys/devices/system/cpu/dscr_control". write()
would parse the input stream, call scanf("%d %x", &pid, &dscr) and do the job.
What is the correct approach? Thanks.
--
Alexey
^ permalink raw reply
* Re: how to implement platform specific per process parameter?
From: Anshuman Khandual @ 2012-08-06 7:57 UTC (permalink / raw)
To: Alexey Kardashevskiy; +Cc: linuxppc-dev, Linux Kernel Mailing List
In-Reply-To: <501F514C.2090909@ozlabs.ru>
> There are 4 options:
> 1. [not a kernel interface] use ptrace to execute the register changing
> command inside the specified pid. The next context switch saves the new
> value in the thread_struct. Dirty hack.
>
> 2. Add a new syscall which would receive pid + register value and do the
> job. A bit too much.
>
> 3. Add some hook in /proc filesystem but so far there were no platform
> specific bits, right?
>
> 4. Implement a static node "/sys/devices/system/cpu/dscr_control".
> write() would parse the input stream, call scanf("%d %x", &pid, &dscr)
> and do the job.
>
/sys/ interface would be appropriate I believe. But in this way we can take a new
(pid, dscr) and update thread_struct. But there should be a way to enlist all
(pid, dscr) values which are explicitly set by the user and different than that
of /sys/devices/system/cpu/dscr_default. So that we can know which process is holding
to what value of DSCR at any point of time.
> What is the correct approach? Thanks.
>
>
Regards
Anshuman
^ permalink raw reply
* Re: mpc8xxx PCIe hotplug needs fixing, some clues ..
From: Joakim Tjernlund @ 2012-08-06 9:39 UTC (permalink / raw)
Cc: scottwood, linuxppc-dev
In-Reply-To: <OF1A29C1FF.28BA9274-ONC1257A44.005B7A8B-C1257A44.005C12EB@transmode.se>
>
> Joakim Tjernlund/Transmode wrote on 2012/07/21 18:11:32:
> >
> > Kumar Gala <galak@kernel.crashing.org> wrote on 2012/07/20 20:53:10:
> > >
> > >
> > > On Jul 20, 2012, at 2:17 AM, Joakim Tjernlund wrote:
> > >
> > > >
> > > > Hi Guys
> > > >
> > > > I see that you have been hacking Freescale PCI before so I send this to you(and the list)
> > > >
> > > > We are using PCIe(as RC) on P2010(basically a mpc85xx) and have PCI device that
> > > > started from user space (needs advance clock conf) so when linux boots there is
> > > > no device at all.
> > > > Trying to "hotplug" the device after it is enabled fails, no amount of recan/remove using
> > > > either fake or real hotplug makes a difference.
> > > >
> > > > I found the cause eventually but I can't fix it properly as I known almost nothing about PCI.
> > > > Cause:
> > > > indirect_pci.c:indirect_read_config() tests for if (hose->indirect_type & PPC_INDIRECT_TYPE_NO_PCIE_LINK)
> > > > and returns PCIBIOS_DEVICE_NOT_FOUND
> > > >
> > > > PPC_INDIRECT_TYPE_NO_PCIE_LINK get set by fsl_pci.c (look for fsl_pcie_check_link) but is never cleared.
> > > > Clearing it as appropriate makes a small difference. If you
> > > > remove the RC and do a few of rescan's then the device appears.
> > > >
> > > > Hacking some more, like so:
> > > >
> > > > int fsl_pcie_check_link(struct pci_controller *hose)
> > > > {
> > > > u32 val;
> > > >
> > > > early_read_config_dword(hose, 0, 0, PCIE_LTSSM, &val);
> > > > hose->indirect_type |= PPC_INDIRECT_TYPE_NO_PCIE_LINK;
> > > > if (val < PCIE_LTSSM_L0)
> > > > return 1;
> > > > hose->indirect_type &= ~PPC_INDIRECT_TYPE_NO_PCIE_LINK;
> > > > return 0;
> > > > }
> > > > and then using it carefully(it is easy to make linux hang) in indirect_read_config():
> > > > indirect_read_config(struct pci_bus *bus, unsigned int devfn, int offset,
> > > > int len, u32 *val)
> > > > {
> > > > struct pci_controller *hose = pci_bus_to_host(bus);
> > > > volatile void __iomem *cfg_data;
> > > > u8 cfg_type = 0;
> > > > u32 bus_no, reg;
> > > >
> > > > if (hose->indirect_type & PPC_INDIRECT_TYPE_NO_PCIE_LINK) {
> > > > if (bus->number != hose->first_busno ||
> > > > devfn != 0) {
> > > > fsl_pcie_check_link(hose);
> > > > return PCIBIOS_DEVICE_NOT_FOUND;
> > > > }
> > > > }
> > > >
> > > > Now it works, just one rescan and the device appears!
> > > > This is a hack, I don't known what other trouble it can cause, I hope you can
> > > > sort this out.
> > >
> > > How are you forcing the re-scan? We can see if we can add a re-check of the link state in that flow somewhere.
> >
> > echo 1 > /sys/bus/pci/rescan
> >
> > Why is that check important? Seems like some very ppc specific workaround for something.
> >
> > >
> > > Can you do a dump_stack() or something to get a call chain?
>
> > here?
> > indirect_read_config(struct pci_bus *bus, unsigned int devfn, int offset,
> > int len, u32 *val)
> > {
> > struct pci_controller *hose = pci_bus_to_host(bus);
> > volatile void __iomem *cfg_data;
> > u8 cfg_type = 0;
> > u32 bus_no, reg;
> > static int first_dump;
> >
> > if (!first_dump) {
> > dump_stack();
> > first_dump = 1;
> > }
> > ...
> >
> > I am not at work and and my board needs a reset button press to recover :(
> > Furthermore, my vacation starts next week, not sure I can get it fixed soon enough
>
> So I managed to get someone to connect the BDI. Here is a dump according to the above:
> Memory CAM mapping: 256/256 Mb, residual: 0Mb
> Linux version 3.4.0+ (jocke@gentoo-jocke) (gcc version 4.5.3 (Gentoo 4.5.3-r2 p1.1, pie-0.4.7) ) #2088 PREEMPT Sat Jul 21 18:05:10 CEST 2012
> bootconsole [udbg0] enabled
> setup_arch: bootmem
> p1010_rdb_setup_arch()
> Call Trace:
> [c0399e50] [c0006f28] show_stack+0x54/0x158 (unreliable)
> [c0399e90] [c0019390] indirect_read_config+0x23c/0x250
> [c0399ec0] [c017e630] pci_bus_read_config_byte+0x4c/0x94
> [c0399ef0] [c000f4d4] early_read_config_byte+0x30/0x44
> [c0399f10] [c0364524] fsl_add_bridge+0x118/0xa5c
> [c0399f90] [c03655b4] p1010_rdb_setup_arch+0x8c/0xcc
> [c0399fb0] [c0361850] setup_arch+0x1a8/0x1e8
> [c0399fc0] [c035d558] start_kernel+0x80/0x2f0
> [c0399ff0] [c00003ac] skpinv+0x298/0x2d4
> Found FSL PCI host bridge at 0x00000000ff70a000. Firmware bus number: 0->1
> PCI host bridge /pcie@ff70a000 ranges:
> MEM 0x0000000080000000..0x000000009fffffff -> 0x0000000080000000
> IO 0x00000000ffc00000..0x00000000ffc0ffff -> 0x0000000000000000
> /pcie@ff70a000: PCICSRBAR @ 0xfff00000
>
> So early_read_config_byte uses indirect_read_config so obviously you cannot
> use early_read_config_xxx from within indirect_read_config as this gets you
> into a nice recursion loop :)
>
> Anyhow, I will be travelling for the better part of this week and will
> have very limited Internet access.
Any ideas on this?
I think my hack will work in the general case too witch a small improvement:
indirect_read_config(struct pci_bus *bus, unsigned int devfn, int offset,
int len, u32 *val)
{
struct pci_controller *hose = pci_bus_to_host(bus);
volatile void __iomem *cfg_data;
u8 cfg_type = 0;
u32 bus_no, reg;
if (hose->indirect_type & PPC_INDIRECT_TYPE_NO_PCIE_LINK) {
if (bus->number != hose->first_busno ||
devfn != 0) {
fsl_pcie_check_link(hose);
if (!(hose->indirect_type & PPC_INDIRECT_TYPE_NO_PCIE_LINK))
goto link_ok;
return PCIBIOS_DEVICE_NOT_FOUND;
}
}
link_ok:
or we could opencode fsl_pcie_check_link directly here to avoid exporting the function
Jocke
^ permalink raw reply
* [PATCH v6 0/8] Raid: enable talitos xor offload for improving performance
From: qiang.liu @ 2012-08-06 10:10 UTC (permalink / raw)
To: linux-crypto, vinod.koul, dan.j.williams, herbert, arnd, gregkh,
linuxppc-dev, linux-kernel, dan.j.williams
Hi all,
The following 8 patches enabling fsl-dma and talitos offload raid
operations for improving raid performance and balancing CPU load.
These patches include talitos, fsl-dma and carma module (caram uses
some features of fsl-dma).
Write performance will be improved by 25-30% tested by iozone.
Write performance is improved about 2% after using spin_lock_bh replace
spin_lock_irqsave.
CPU load will be reduced by 8%.
Changes in v6:
- swap the order of original patch 3/6 and 4/6;
- merge Ira's patch to reduce the size of original patch;
- merge Ira's patch of carma in 8/8;
- update documents and descriptions according to Ira's advice;
Changes in v5:
- add detail description in patch 3/6 about the process of completed
descriptor, the process is in align with fsl-dma Reference Manual,
illustrate the potential risk and how to reproduce it;
- drop the patch 7/7 in v4 according to Timur's comments;
Changes in v4:
- fix an error in talitos when dest addr is same with src addr, dest
should be freed only one time if src is same with dest addr;
- correct coding style in fsl-dma according to Ira's comments;
- fix a race condition in fsl-dma fsl_tx_status(), remove the interface
which is used to free descriptors in queue ld_completed, this interface
has been included in fsldma_cleanup_descriptor(), in v3, there is one
place missed spin_lock protect;
- split the original patch 3/4 up to 2 patches 3/7 and 4/7 according to
Li Yang's comments;
- fix a warning of unitialized cookie;
- add memory copy self test in fsl-dma;
- add more detail description about use spin_lock_bh() to instead of
spin_lock_irqsave() according to Timur's comments.
Changes in v3:
- change release process of fsl-dma descriptor for resolve the
potential race condition;
- add test result when use spin_lock_bh replace spin_lock_irqsave;
- modify the benchmark results according to the latest patch.
Changes in v2:
- rebase onto cryptodev tree;
- split the patch 3/4 up to 3 independent patches;
- remove the patch 4/4, the fix is not for cryptodev tree;
Qiang Liu (8):
Talitos: Support for async_tx XOR offload
fsl-dma: remove attribute DMA_INTERRUPT of dmaengine
fsl-dma: add fsl_dma_free_descriptor() to reduce code duplication
fsl-dma: move functions to avoid forward declarations
fsl-dma: change release process of dma descriptor for supporting async_tx
fsl-dma: use spin_lock_bh to instead of spin_lock_irqsave
fsl-dma: fix a warning of unitialized cookie
carma: remove unnecessary DMA_INTERRUPT capability
drivers/crypto/Kconfig | 9 +
drivers/crypto/talitos.c | 413 ++++++++++++++++++++++++++
drivers/crypto/talitos.h | 53 ++++
drivers/dma/fsldma.c | 487 +++++++++++++++++--------------
drivers/dma/fsldma.h | 17 +-
drivers/misc/carma/carma-fpga-program.c | 1 -
drivers/misc/carma/carma-fpga.c | 2 +-
7 files changed, 760 insertions(+), 222 deletions(-)
^ permalink raw reply
* [PATCH v6 1/8] Talitos: Support for async_tx XOR offload
From: qiang.liu @ 2012-08-06 10:11 UTC (permalink / raw)
To: linux-crypto, herbert, davem, linux-kernel, linuxppc-dev
Cc: Qiang Liu, arnd, vinod.koul, gregkh, dan.j.williams,
dan.j.williams
From: Qiang Liu <qiang.liu@freescale.com>
Expose Talitos's XOR functionality to be used for RAID parity
calculation via the Async_tx layer.
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: David S. Miller <davem@davemloft.net>
Signed-off-by: Dipen Dudhat <Dipen.Dudhat@freescale.com>
Signed-off-by: Maneesh Gupta <Maneesh.Gupta@freescale.com>
Signed-off-by: Kim Phillips <kim.phillips@freescale.com>
Signed-off-by: Vishnu Suresh <Vishnu@freescale.com>
Signed-off-by: Qiang Liu <qiang.liu@freescale.com>
---
drivers/crypto/Kconfig | 9 +
drivers/crypto/talitos.c | 413 ++++++++++++++++++++++++++++++++++++++++++++++
drivers/crypto/talitos.h | 53 ++++++
3 files changed, 475 insertions(+), 0 deletions(-)
diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig
index be6b2ba..f0a7c29 100644
--- a/drivers/crypto/Kconfig
+++ b/drivers/crypto/Kconfig
@@ -222,6 +222,15 @@ config CRYPTO_DEV_TALITOS
To compile this driver as a module, choose M here: the module
will be called talitos.
+config CRYPTO_DEV_TALITOS_RAIDXOR
+ bool "Talitos RAID5 XOR Calculation Offload"
+ default y
+ select DMA_ENGINE
+ depends on CRYPTO_DEV_TALITOS
+ help
+ Say 'Y' here to use the Freescale Security Engine (SEC) to
+ offload RAID XOR parity Calculation
+
config CRYPTO_DEV_IXP4XX
tristate "Driver for IXP4xx crypto hardware acceleration"
depends on ARCH_IXP4XX
diff --git a/drivers/crypto/talitos.c b/drivers/crypto/talitos.c
index efff788..b34264e 100644
--- a/drivers/crypto/talitos.c
+++ b/drivers/crypto/talitos.c
@@ -619,6 +619,399 @@ static void talitos_unregister_rng(struct device *dev)
hwrng_unregister(&priv->rng);
}
+#ifdef CONFIG_CRYPTO_DEV_TALITOS_RAIDXOR
+static void talitos_release_xor(struct device *dev, struct talitos_desc *hwdesc,
+ void *context, int error);
+
+static enum dma_status talitos_is_tx_complete(struct dma_chan *chan,
+ dma_cookie_t cookie,
+ struct dma_tx_state *state)
+{
+ struct talitos_xor_chan *xor_chan;
+ dma_cookie_t last_used;
+ dma_cookie_t last_complete;
+
+ xor_chan = container_of(chan, struct talitos_xor_chan, common);
+
+ last_used = chan->cookie;
+ last_complete = xor_chan->completed_cookie;
+
+ if (state->last)
+ state->last = last_complete;
+
+ if (state->used)
+ state->used = last_used;
+
+ return dma_async_is_complete(cookie, last_complete, last_used);
+}
+
+static void talitos_process_pending(struct talitos_xor_chan *xor_chan)
+{
+ struct talitos_xor_desc *desc, *_desc;
+ unsigned long flags;
+ int status;
+ struct talitos_private *priv;
+ int ch;
+
+ priv = dev_get_drvdata(xor_chan->dev);
+ ch = atomic_inc_return(&priv->last_chan) &
+ (priv->num_channels - 1);
+ spin_lock_irqsave(&xor_chan->desc_lock, flags);
+
+ list_for_each_entry_safe(desc, _desc, &xor_chan->pending_q, node) {
+ status = talitos_submit(xor_chan->dev, ch, &desc->hwdesc,
+ talitos_release_xor, desc);
+ if (status != -EINPROGRESS)
+ break;
+
+ list_del(&desc->node);
+ list_add_tail(&desc->node, &xor_chan->in_progress_q);
+ }
+
+ spin_unlock_irqrestore(&xor_chan->desc_lock, flags);
+}
+
+static void talitos_xor_run_tx_complete_actions(struct talitos_xor_desc *desc,
+ struct talitos_xor_chan *xor_chan)
+{
+ struct device *dev = xor_chan->dev;
+ dma_addr_t dest, addr;
+ unsigned int src_cnt = desc->unmap_src_cnt;
+ unsigned int len = desc->unmap_len;
+ enum dma_ctrl_flags flags = desc->async_tx.flags;
+ struct dma_async_tx_descriptor *tx = &desc->async_tx;
+
+ /* unmap dma addresses */
+ dest = desc->hwdesc.ptr[6].ptr;
+ if (likely(!(flags & DMA_COMPL_SKIP_DEST_UNMAP)))
+ dma_unmap_page(dev, dest, len, DMA_BIDIRECTIONAL);
+
+ desc->idx = 6 - src_cnt;
+ if (likely(!(flags & DMA_COMPL_SKIP_SRC_UNMAP))) {
+ while(desc->idx < 6) {
+ addr = desc->hwdesc.ptr[desc->idx++].ptr;
+ if (addr == dest)
+ continue;
+ dma_unmap_page(dev, addr, len, DMA_TO_DEVICE);
+ }
+ }
+
+ /* run dependent operations */
+ dma_run_dependencies(tx);
+}
+
+static void talitos_release_xor(struct device *dev, struct talitos_desc *hwdesc,
+ void *context, int error)
+{
+ struct talitos_xor_desc *desc = context;
+ struct talitos_xor_chan *xor_chan;
+ dma_async_tx_callback callback;
+ void *callback_param;
+
+ if (unlikely(error))
+ dev_err(dev, "xor operation: talitos error %d\n", error);
+
+ xor_chan = container_of(desc->async_tx.chan, struct talitos_xor_chan,
+ common);
+ spin_lock_bh(&xor_chan->desc_lock);
+ if (xor_chan->completed_cookie < desc->async_tx.cookie)
+ xor_chan->completed_cookie = desc->async_tx.cookie;
+
+ callback = desc->async_tx.callback;
+ callback_param = desc->async_tx.callback_param;
+
+ if (callback) {
+ spin_unlock_bh(&xor_chan->desc_lock);
+ callback(callback_param);
+ spin_lock_bh(&xor_chan->desc_lock);
+ }
+
+ talitos_xor_run_tx_complete_actions(desc, xor_chan);
+
+ list_del(&desc->node);
+ list_add_tail(&desc->node, &xor_chan->free_desc);
+ spin_unlock_bh(&xor_chan->desc_lock);
+ if (!list_empty(&xor_chan->pending_q))
+ talitos_process_pending(xor_chan);
+}
+
+/**
+ * talitos_issue_pending - move the descriptors in submit
+ * queue to pending queue and submit them for processing
+ * @chan: DMA channel
+ */
+static void talitos_issue_pending(struct dma_chan *chan)
+{
+ struct talitos_xor_chan *xor_chan;
+
+ xor_chan = container_of(chan, struct talitos_xor_chan, common);
+ spin_lock_bh(&xor_chan->desc_lock);
+ list_splice_tail_init(&xor_chan->submit_q,
+ &xor_chan->pending_q);
+ spin_unlock_bh(&xor_chan->desc_lock);
+ talitos_process_pending(xor_chan);
+}
+
+static dma_cookie_t talitos_async_tx_submit(struct dma_async_tx_descriptor *tx)
+{
+ struct talitos_xor_desc *desc;
+ struct talitos_xor_chan *xor_chan;
+ dma_cookie_t cookie;
+
+ desc = container_of(tx, struct talitos_xor_desc, async_tx);
+ xor_chan = container_of(tx->chan, struct talitos_xor_chan, common);
+
+ spin_lock_bh(&xor_chan->desc_lock);
+
+ cookie = xor_chan->common.cookie + 1;
+ if (cookie < 0)
+ cookie = 1;
+
+ desc->async_tx.cookie = cookie;
+ xor_chan->common.cookie = desc->async_tx.cookie;
+
+ list_splice_tail_init(&desc->tx_list,
+ &xor_chan->submit_q);
+
+ spin_unlock_bh(&xor_chan->desc_lock);
+
+ return cookie;
+}
+
+static struct talitos_xor_desc *talitos_xor_alloc_descriptor(
+ struct talitos_xor_chan *xor_chan, gfp_t flags)
+{
+ struct talitos_xor_desc *desc;
+
+ desc = kmalloc(sizeof(*desc), flags);
+ if (desc) {
+ xor_chan->total_desc++;
+ desc->async_tx.tx_submit = talitos_async_tx_submit;
+ }
+
+ return desc;
+}
+
+static void talitos_free_chan_resources(struct dma_chan *chan)
+{
+ struct talitos_xor_chan *xor_chan;
+ struct talitos_xor_desc *desc, *_desc;
+
+ xor_chan = container_of(chan, struct talitos_xor_chan, common);
+
+ spin_lock_bh(&xor_chan->desc_lock);
+
+ list_for_each_entry_safe(desc, _desc, &xor_chan->submit_q, node) {
+ list_del(&desc->node);
+ xor_chan->total_desc--;
+ kfree(desc);
+ }
+ list_for_each_entry_safe(desc, _desc, &xor_chan->pending_q, node) {
+ list_del(&desc->node);
+ xor_chan->total_desc--;
+ kfree(desc);
+ }
+ list_for_each_entry_safe(desc, _desc, &xor_chan->in_progress_q, node) {
+ list_del(&desc->node);
+ xor_chan->total_desc--;
+ kfree(desc);
+ }
+ list_for_each_entry_safe(desc, _desc, &xor_chan->free_desc, node) {
+ list_del(&desc->node);
+ xor_chan->total_desc--;
+ kfree(desc);
+ }
+
+ /* Some descriptor not freed? */
+ if (unlikely(xor_chan->total_desc))
+ dev_warn(chan->device->dev, "Failed to free xor channel resource\n");
+
+ spin_unlock_bh(&xor_chan->desc_lock);
+}
+
+static int talitos_alloc_chan_resources(struct dma_chan *chan)
+{
+ struct talitos_xor_chan *xor_chan;
+ struct talitos_xor_desc *desc;
+ LIST_HEAD(tmp_list);
+ int i;
+
+ xor_chan = container_of(chan, struct talitos_xor_chan, common);
+
+ if (!list_empty(&xor_chan->free_desc))
+ return xor_chan->total_desc;
+
+ for (i = 0; i < TALITOS_MAX_DESCRIPTOR_NR; i++) {
+ desc = talitos_xor_alloc_descriptor(xor_chan,
+ GFP_KERNEL | GFP_DMA);
+ if (!desc) {
+ dev_err(xor_chan->common.device->dev,
+ "Only %d initial descriptors\n", i);
+ break;
+ }
+ list_add_tail(&desc->node, &tmp_list);
+ }
+
+ if (!i)
+ return -ENOMEM;
+
+ /* At least one desc is allocated */
+ spin_lock_bh(&xor_chan->desc_lock);
+ list_splice_init(&tmp_list, &xor_chan->free_desc);
+ spin_unlock_bh(&xor_chan->desc_lock);
+
+ return xor_chan->total_desc;
+}
+
+static struct dma_async_tx_descriptor *talitos_prep_dma_xor(
+ struct dma_chan *chan, dma_addr_t dest, dma_addr_t *src,
+ unsigned int src_cnt, size_t len, unsigned long flags)
+{
+ struct talitos_xor_chan *xor_chan;
+ struct talitos_xor_desc *new;
+ struct talitos_desc *desc;
+ int i, j;
+
+ BUG_ON(len > TALITOS_MAX_DATA_LEN);
+
+ xor_chan = container_of(chan, struct talitos_xor_chan, common);
+
+ spin_lock_bh(&xor_chan->desc_lock);
+ if (!list_empty(&xor_chan->free_desc)) {
+ new = container_of(xor_chan->free_desc.next,
+ struct talitos_xor_desc, node);
+ list_del(&new->node);
+ } else {
+ new = talitos_xor_alloc_descriptor(xor_chan, GFP_KERNEL | GFP_DMA);
+ }
+ spin_unlock_bh(&xor_chan->desc_lock);
+
+ if (!new) {
+ dev_err(xor_chan->common.device->dev,
+ "No free memory for XOR DMA descriptor\n");
+ return NULL;
+ }
+ dma_async_tx_descriptor_init(&new->async_tx, &xor_chan->common);
+
+ INIT_LIST_HEAD(&new->node);
+ INIT_LIST_HEAD(&new->tx_list);
+
+ desc = &new->hwdesc;
+ /* Set destination: Last pointer pair */
+ to_talitos_ptr(&desc->ptr[6], dest);
+ desc->ptr[6].len = cpu_to_be16(len);
+ desc->ptr[6].j_extent = 0;
+ new->unmap_src_cnt = src_cnt;
+ new->unmap_len = len;
+
+ /* Set Sources: End loading from second-last pointer pair */
+ for (i = 5, j = 0; j < src_cnt && i >= 0; i--, j++) {
+ to_talitos_ptr(&desc->ptr[i], src[j]);
+ desc->ptr[i].len = cpu_to_be16(len);
+ desc->ptr[i].j_extent = 0;
+ }
+
+ /*
+ * documentation states first 0 ptr/len combo marks end of sources
+ * yet device produces scatter boundary error unless all subsequent
+ * sources are zeroed out
+ */
+ for (; i >= 0; i--) {
+ to_talitos_ptr(&desc->ptr[i], 0);
+ desc->ptr[i].len = 0;
+ desc->ptr[i].j_extent = 0;
+ }
+
+ desc->hdr = DESC_HDR_SEL0_AESU | DESC_HDR_MODE0_AESU_XOR |
+ DESC_HDR_TYPE_RAID_XOR;
+
+ new->async_tx.parent = NULL;
+ new->async_tx.next = NULL;
+ new->async_tx.cookie = 0;
+ async_tx_ack(&new->async_tx);
+
+ list_add_tail(&new->node, &new->tx_list);
+
+ new->async_tx.flags = flags;
+ new->async_tx.cookie = -EBUSY;
+
+ return &new->async_tx;
+}
+
+static void talitos_unregister_async_xor(struct device *dev)
+{
+ struct talitos_private *priv = dev_get_drvdata(dev);
+ struct talitos_xor_chan *xor_chan;
+ struct dma_chan *chan, *_chan;
+
+ if (priv->dma_dev_common.chancnt)
+ dma_async_device_unregister(&priv->dma_dev_common);
+
+ list_for_each_entry_safe(chan, _chan, &priv->dma_dev_common.channels,
+ device_node) {
+ xor_chan = container_of(chan, struct talitos_xor_chan,
+ common);
+ list_del(&chan->device_node);
+ priv->dma_dev_common.chancnt--;
+ kfree(xor_chan);
+ }
+}
+
+/**
+ * talitos_register_dma_async - Initialize the Freescale XOR ADMA device
+ * It is registered as a DMA device with the capability to perform
+ * XOR operation with the Async_tx layer.
+ * The various queues and channel resources are also allocated.
+ */
+static int talitos_register_async_tx(struct device *dev, int max_xor_srcs)
+{
+ struct talitos_private *priv = dev_get_drvdata(dev);
+ struct dma_device *dma_dev = &priv->dma_dev_common;
+ struct talitos_xor_chan *xor_chan;
+ int err;
+
+ xor_chan = kzalloc(sizeof(struct talitos_xor_chan), GFP_KERNEL);
+ if (!xor_chan) {
+ dev_err(dev, "unable to allocate xor channel\n");
+ return -ENOMEM;
+ }
+
+ dma_dev->dev = dev;
+ dma_dev->device_alloc_chan_resources = talitos_alloc_chan_resources;
+ dma_dev->device_free_chan_resources = talitos_free_chan_resources;
+ dma_dev->device_prep_dma_xor = talitos_prep_dma_xor;
+ dma_dev->max_xor = max_xor_srcs;
+ dma_dev->device_tx_status = talitos_is_tx_complete;
+ dma_dev->device_issue_pending = talitos_issue_pending;
+ INIT_LIST_HEAD(&dma_dev->channels);
+ dma_cap_set(DMA_XOR, dma_dev->cap_mask);
+
+ xor_chan->dev = dev;
+ xor_chan->common.device = dma_dev;
+ xor_chan->total_desc = 0;
+ INIT_LIST_HEAD(&xor_chan->submit_q);
+ INIT_LIST_HEAD(&xor_chan->pending_q);
+ INIT_LIST_HEAD(&xor_chan->in_progress_q);
+ INIT_LIST_HEAD(&xor_chan->free_desc);
+ spin_lock_init(&xor_chan->desc_lock);
+
+ list_add_tail(&xor_chan->common.device_node, &dma_dev->channels);
+ dma_dev->chancnt++;
+
+ err = dma_async_device_register(dma_dev);
+ if (err) {
+ dev_err(dev, "Unable to register XOR with Async_tx\n");
+ goto err_out;
+ }
+
+ return err;
+
+err_out:
+ talitos_unregister_async_xor(dev);
+ return err;
+}
+#endif
+
/*
* crypto alg
*/
@@ -2891,6 +3284,26 @@ static int talitos_probe(struct platform_device *ofdev)
dev_info(dev, "hwrng\n");
}
+#ifdef CONFIG_CRYPTO_DEV_TALITOS_RAIDXOR
+ /*
+ * register with async_tx xor, if capable
+ * SEC 2.x support up to 3 RAID sources,
+ * SEC 3.x support up to 6
+ */
+ if (hw_supports(dev, DESC_HDR_SEL0_AESU | DESC_HDR_TYPE_RAID_XOR)) {
+ int max_xor_srcs = 3;
+ if (of_device_is_compatible(np, "fsl,sec3.0"))
+ max_xor_srcs = 6;
+ err = talitos_register_async_tx(dev, max_xor_srcs);
+ if (err) {
+ dev_err(dev, "failed to register async_tx xor: %d\n",
+ err);
+ goto err_out;
+ }
+ dev_info(dev, "max_xor_srcs %d\n", max_xor_srcs);
+ }
+#endif
+
/* register crypto algorithms the device supports */
for (i = 0; i < ARRAY_SIZE(driver_algs); i++) {
if (hw_supports(dev, driver_algs[i].desc_hdr_template)) {
diff --git a/drivers/crypto/talitos.h b/drivers/crypto/talitos.h
index 61a1405..fc9d125 100644
--- a/drivers/crypto/talitos.h
+++ b/drivers/crypto/talitos.h
@@ -30,6 +30,7 @@
#define TALITOS_TIMEOUT 100000
#define TALITOS_MAX_DATA_LEN 65535
+#define TALITOS_MAX_DESCRIPTOR_NR 256
#define DESC_TYPE(desc_hdr) ((be32_to_cpu(desc_hdr) >> 3) & 0x1f)
#define PRIMARY_EU(desc_hdr) ((be32_to_cpu(desc_hdr) >> 28) & 0xf)
@@ -131,7 +132,57 @@ struct talitos_private {
/* hwrng device */
struct hwrng rng;
+
+#ifdef CONFIG_CRYPTO_DEV_TALITOS_RAIDXOR
+ /* XOR Device */
+ struct dma_device dma_dev_common;
+#endif
+};
+
+#ifdef CONFIG_CRYPTO_DEV_TALITOS_RAIDXOR
+/**
+ * talitos_xor_chan - context management for the async_tx channel
+ * @completed_cookie: the last completed cookie
+ * @desc_lock: lock for tx queue
+ * @total_desc: number of descriptors allocated
+ * @submit_q: queue of submitted descriptors
+ * @pending_q: queue of pending descriptors
+ * @in_progress_q: queue of descriptors in progress
+ * @free_desc: queue of unused descriptors
+ * @dev: talitos device implementing this channel
+ * @common: the corresponding xor channel in async_tx
+ */
+struct talitos_xor_chan {
+ dma_cookie_t completed_cookie;
+ spinlock_t desc_lock;
+ unsigned int total_desc;
+ struct list_head submit_q;
+ struct list_head pending_q;
+ struct list_head in_progress_q;
+ struct list_head free_desc;
+ struct device *dev;
+ struct dma_chan common;
+};
+
+/**
+ * talitos_xor_desc - software xor descriptor
+ * @async_tx: the referring async_tx descriptor
+ * @node:
+ * @hwdesc: h/w descriptor
+ * @unmap_src_cnt: number of xor sources
+ * @unmap_len: transaction byte count
+ * @idx: index of xor sources
+ */
+struct talitos_xor_desc {
+ struct dma_async_tx_descriptor async_tx;
+ struct list_head tx_list;
+ struct list_head node;
+ struct talitos_desc hwdesc;
+ unsigned int unmap_src_cnt;
+ unsigned int unmap_len;
+ unsigned int idx;
};
+#endif
extern int talitos_submit(struct device *dev, int ch, struct talitos_desc *desc,
void (*callback)(struct device *dev,
@@ -284,6 +335,7 @@ extern int talitos_submit(struct device *dev, int ch, struct talitos_desc *desc,
/* primary execution unit mode (MODE0) and derivatives */
#define DESC_HDR_MODE0_ENCRYPT cpu_to_be32(0x00100000)
#define DESC_HDR_MODE0_AESU_CBC cpu_to_be32(0x00200000)
+#define DESC_HDR_MODE0_AESU_XOR cpu_to_be32(0x0c600000)
#define DESC_HDR_MODE0_DEU_CBC cpu_to_be32(0x00400000)
#define DESC_HDR_MODE0_DEU_3DES cpu_to_be32(0x00200000)
#define DESC_HDR_MODE0_MDEU_CONT cpu_to_be32(0x08000000)
@@ -344,6 +396,7 @@ extern int talitos_submit(struct device *dev, int ch, struct talitos_desc *desc,
#define DESC_HDR_TYPE_IPSEC_ESP cpu_to_be32(1 << 3)
#define DESC_HDR_TYPE_COMMON_NONSNOOP_NO_AFEU cpu_to_be32(2 << 3)
#define DESC_HDR_TYPE_HMAC_SNOOP_NO_AFEU cpu_to_be32(4 << 3)
+#define DESC_HDR_TYPE_RAID_XOR cpu_to_be32(21 << 3)
/* link table extent field bits */
#define DESC_PTR_LNKTBL_JUMP 0x80
--
1.7.5.1
^ permalink raw reply related
* [PATCH v6 2/8] fsl-dma: remove attribute DMA_INTERRUPT of dmaengine
From: qiang.liu @ 2012-08-06 10:12 UTC (permalink / raw)
To: linux-crypto, linuxppc-dev, dan.j.williams, linux-kernel,
dan.j.williams, vinod.koul
Cc: arnd, gregkh, Qiang Liu, herbert, davem
From: Qiang Liu <qiang.liu@freescale.com>
Delete attribute DMA_INTERRUPT because fsl-dma doesn't support this function,
exception will be thrown if talitos is used to offload xor at the same time.
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dan Williams <dan.j.williams@gmail.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Li Yang <leoli@freescale.com>
Signed-off-by: Qiang Liu <qiang.liu@freescale.com>
Acked-by: Ira W. Snyder <iws@ovro.caltech.edu>
---
drivers/dma/fsldma.c | 31 -------------------------------
1 files changed, 0 insertions(+), 31 deletions(-)
diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c
index 8f84761..4f2f212 100644
--- a/drivers/dma/fsldma.c
+++ b/drivers/dma/fsldma.c
@@ -543,35 +543,6 @@ static void fsl_dma_free_chan_resources(struct dma_chan *dchan)
}
static struct dma_async_tx_descriptor *
-fsl_dma_prep_interrupt(struct dma_chan *dchan, unsigned long flags)
-{
- struct fsldma_chan *chan;
- struct fsl_desc_sw *new;
-
- if (!dchan)
- return NULL;
-
- chan = to_fsl_chan(dchan);
-
- new = fsl_dma_alloc_descriptor(chan);
- if (!new) {
- chan_err(chan, "%s\n", msg_ld_oom);
- return NULL;
- }
-
- new->async_tx.cookie = -EBUSY;
- new->async_tx.flags = flags;
-
- /* Insert the link descriptor to the LD ring */
- list_add_tail(&new->node, &new->tx_list);
-
- /* Set End-of-link to the last link descriptor of new list */
- set_ld_eol(chan, new);
-
- return &new->async_tx;
-}
-
-static struct dma_async_tx_descriptor *
fsl_dma_prep_memcpy(struct dma_chan *dchan,
dma_addr_t dma_dst, dma_addr_t dma_src,
size_t len, unsigned long flags)
@@ -1352,12 +1323,10 @@ static int __devinit fsldma_of_probe(struct platform_device *op)
fdev->irq = irq_of_parse_and_map(op->dev.of_node, 0);
dma_cap_set(DMA_MEMCPY, fdev->common.cap_mask);
- dma_cap_set(DMA_INTERRUPT, fdev->common.cap_mask);
dma_cap_set(DMA_SG, fdev->common.cap_mask);
dma_cap_set(DMA_SLAVE, fdev->common.cap_mask);
fdev->common.device_alloc_chan_resources = fsl_dma_alloc_chan_resources;
fdev->common.device_free_chan_resources = fsl_dma_free_chan_resources;
- fdev->common.device_prep_dma_interrupt = fsl_dma_prep_interrupt;
fdev->common.device_prep_dma_memcpy = fsl_dma_prep_memcpy;
fdev->common.device_prep_dma_sg = fsl_dma_prep_sg;
fdev->common.device_tx_status = fsl_tx_status;
--
1.7.5.1
^ permalink raw reply related
* [PATCH v6 3/8] fsl-dma: add fsl_dma_free_descriptor() to reduce code duplication
From: qiang.liu @ 2012-08-06 10:13 UTC (permalink / raw)
To: linux-crypto, linuxppc-dev, dan.j.williams, linux-kernel,
dan.j.williams, vinod.koul
Cc: arnd, Ira W. Snyder, gregkh, Qiang Liu, herbert, davem
From: Qiang Liu <qiang.liu@freescale.com>
There are several places where descriptors are freed using identical
code. Put this code into a function to reduce code duplication.
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dan Williams <dan.j.williams@gmail.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: Ira W. Snyder <iws@ovro.caltech.edu>
---
drivers/dma/fsldma.c | 38 ++++++++++++++++++++------------------
1 files changed, 20 insertions(+), 18 deletions(-)
diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c
index 4f2f212..d4720d3 100644
--- a/drivers/dma/fsldma.c
+++ b/drivers/dma/fsldma.c
@@ -427,6 +427,21 @@ static dma_cookie_t fsl_dma_tx_submit(struct dma_async_tx_descriptor *tx)
}
/**
+ * fsl_dma_free_descriptor - Free descriptor from channel's DMA pool.
+ * @chan : Freescale DMA channel
+ * @desc: descriptor to be freed
+ */
+static void fsl_dma_free_descriptor(struct fsldma_chan *chan,
+ struct fsl_desc_sw *desc)
+{
+ list_del(&desc->node);
+#ifdef FSL_DMA_LD_DEBUG
+ chan_dbg(chan, "LD %p free\n", desc);
+#endif
+ dma_pool_free(chan->desc_pool, desc, desc->async_tx.phys);
+}
+
+/**
* fsl_dma_alloc_descriptor - Allocate descriptor from channel's DMA pool.
* @chan : Freescale DMA channel
*
@@ -500,13 +515,8 @@ static void fsldma_free_desc_list(struct fsldma_chan *chan,
{
struct fsl_desc_sw *desc, *_desc;
- list_for_each_entry_safe(desc, _desc, list, node) {
- list_del(&desc->node);
-#ifdef FSL_DMA_LD_DEBUG
- chan_dbg(chan, "LD %p free\n", desc);
-#endif
- dma_pool_free(chan->desc_pool, desc, desc->async_tx.phys);
- }
+ list_for_each_entry_safe(desc, _desc, list, node)
+ fsl_dma_free_descriptor(chan, desc);
}
static void fsldma_free_desc_list_reverse(struct fsldma_chan *chan,
@@ -514,13 +524,8 @@ static void fsldma_free_desc_list_reverse(struct fsldma_chan *chan,
{
struct fsl_desc_sw *desc, *_desc;
- list_for_each_entry_safe_reverse(desc, _desc, list, node) {
- list_del(&desc->node);
-#ifdef FSL_DMA_LD_DEBUG
- chan_dbg(chan, "LD %p free\n", desc);
-#endif
- dma_pool_free(chan->desc_pool, desc, desc->async_tx.phys);
- }
+ list_for_each_entry_safe_reverse(desc, _desc, list, node)
+ fsl_dma_free_descriptor(chan, desc);
}
/**
@@ -855,10 +860,7 @@ static void fsldma_cleanup_descriptor(struct fsldma_chan *chan,
dma_unmap_page(dev, src, len, DMA_TO_DEVICE);
}
-#ifdef FSL_DMA_LD_DEBUG
- chan_dbg(chan, "LD %p free\n", desc);
-#endif
- dma_pool_free(chan->desc_pool, desc, txd->phys);
+ fsl_dma_free_descriptor(chan, desc);
}
/**
--
1.7.5.1
^ permalink raw reply related
* [PATCH v6 4/8] fsl-dma: move functions to avoid forward declarations
From: qiang.liu @ 2012-08-06 10:14 UTC (permalink / raw)
To: linux-crypto, linuxppc-dev, dan.j.williams, linux-kernel,
dan.j.williams, vinod.koul
Cc: arnd, Ira W. Snyder, gregkh, Qiang Liu, herbert, davem
From: Qiang Liu <qiang.liu@freescale.com>
These functions will be modified in the next patch in the series. By
moving the function in a patch separate from the changes, it will make
review easier.
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dan Williams <dan.j.williams@gmail.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Li Yang <leoli@freescale.com>
Signed-off-by: Ira W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: Qiang Liu <qiang.liu@freescale.com>
---
drivers/dma/fsldma.c | 230 +++++++++++++++++++++++++-------------------------
1 files changed, 115 insertions(+), 115 deletions(-)
diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c
index d4720d3..36490a3 100644
--- a/drivers/dma/fsldma.c
+++ b/drivers/dma/fsldma.c
@@ -472,6 +472,121 @@ static struct fsl_desc_sw *fsl_dma_alloc_descriptor(struct fsldma_chan *chan)
}
/**
+ * fsl_chan_xfer_ld_queue - transfer any pending transactions
+ * @chan : Freescale DMA channel
+ *
+ * HARDWARE STATE: idle
+ * LOCKING: must hold chan->desc_lock
+ */
+static void fsl_chan_xfer_ld_queue(struct fsldma_chan *chan)
+{
+ struct fsl_desc_sw *desc;
+
+ /*
+ * If the list of pending descriptors is empty, then we
+ * don't need to do any work at all
+ */
+ if (list_empty(&chan->ld_pending)) {
+ chan_dbg(chan, "no pending LDs\n");
+ return;
+ }
+
+ /*
+ * The DMA controller is not idle, which means that the interrupt
+ * handler will start any queued transactions when it runs after
+ * this transaction finishes
+ */
+ if (!chan->idle) {
+ chan_dbg(chan, "DMA controller still busy\n");
+ return;
+ }
+
+ /*
+ * If there are some link descriptors which have not been
+ * transferred, we need to start the controller
+ */
+
+ /*
+ * Move all elements from the queue of pending transactions
+ * onto the list of running transactions
+ */
+ chan_dbg(chan, "idle, starting controller\n");
+ desc = list_first_entry(&chan->ld_pending, struct fsl_desc_sw, node);
+ list_splice_tail_init(&chan->ld_pending, &chan->ld_running);
+
+ /*
+ * The 85xx DMA controller doesn't clear the channel start bit
+ * automatically at the end of a transfer. Therefore we must clear
+ * it in software before starting the transfer.
+ */
+ if ((chan->feature & FSL_DMA_IP_MASK) == FSL_DMA_IP_85XX) {
+ u32 mode;
+
+ mode = DMA_IN(chan, &chan->regs->mr, 32);
+ mode &= ~FSL_DMA_MR_CS;
+ DMA_OUT(chan, &chan->regs->mr, mode, 32);
+ }
+
+ /*
+ * Program the descriptor's address into the DMA controller,
+ * then start the DMA transaction
+ */
+ set_cdar(chan, desc->async_tx.phys);
+ get_cdar(chan);
+
+ dma_start(chan);
+ chan->idle = false;
+}
+
+/**
+ * fsldma_cleanup_descriptor - cleanup and free a single link descriptor
+ * @chan: Freescale DMA channel
+ * @desc: descriptor to cleanup and free
+ *
+ * This function is used on a descriptor which has been executed by the DMA
+ * controller. It will run any callbacks, submit any dependencies, and then
+ * free the descriptor.
+ */
+static void fsldma_cleanup_descriptor(struct fsldma_chan *chan,
+ struct fsl_desc_sw *desc)
+{
+ struct dma_async_tx_descriptor *txd = &desc->async_tx;
+ struct device *dev = chan->common.device->dev;
+ dma_addr_t src = get_desc_src(chan, desc);
+ dma_addr_t dst = get_desc_dst(chan, desc);
+ u32 len = get_desc_cnt(chan, desc);
+
+ /* Run the link descriptor callback function */
+ if (txd->callback) {
+#ifdef FSL_DMA_LD_DEBUG
+ chan_dbg(chan, "LD %p callback\n", desc);
+#endif
+ txd->callback(txd->callback_param);
+ }
+
+ /* Run any dependencies */
+ dma_run_dependencies(txd);
+
+ /* Unmap the dst buffer, if requested */
+ if (!(txd->flags & DMA_COMPL_SKIP_DEST_UNMAP)) {
+ if (txd->flags & DMA_COMPL_DEST_UNMAP_SINGLE)
+ dma_unmap_single(dev, dst, len, DMA_FROM_DEVICE);
+ else
+ dma_unmap_page(dev, dst, len, DMA_FROM_DEVICE);
+ }
+
+ /* Unmap the src buffer, if requested */
+ if (!(txd->flags & DMA_COMPL_SKIP_SRC_UNMAP)) {
+ if (txd->flags & DMA_COMPL_SRC_UNMAP_SINGLE)
+ dma_unmap_single(dev, src, len, DMA_TO_DEVICE);
+ else
+ dma_unmap_page(dev, src, len, DMA_TO_DEVICE);
+ }
+
+ fsl_dma_free_descriptor(chan, desc);
+}
+
+/**
* fsl_dma_alloc_chan_resources - Allocate resources for DMA channel.
* @chan : Freescale DMA channel
*
@@ -816,121 +931,6 @@ static int fsl_dma_device_control(struct dma_chan *dchan,
}
/**
- * fsldma_cleanup_descriptor - cleanup and free a single link descriptor
- * @chan: Freescale DMA channel
- * @desc: descriptor to cleanup and free
- *
- * This function is used on a descriptor which has been executed by the DMA
- * controller. It will run any callbacks, submit any dependencies, and then
- * free the descriptor.
- */
-static void fsldma_cleanup_descriptor(struct fsldma_chan *chan,
- struct fsl_desc_sw *desc)
-{
- struct dma_async_tx_descriptor *txd = &desc->async_tx;
- struct device *dev = chan->common.device->dev;
- dma_addr_t src = get_desc_src(chan, desc);
- dma_addr_t dst = get_desc_dst(chan, desc);
- u32 len = get_desc_cnt(chan, desc);
-
- /* Run the link descriptor callback function */
- if (txd->callback) {
-#ifdef FSL_DMA_LD_DEBUG
- chan_dbg(chan, "LD %p callback\n", desc);
-#endif
- txd->callback(txd->callback_param);
- }
-
- /* Run any dependencies */
- dma_run_dependencies(txd);
-
- /* Unmap the dst buffer, if requested */
- if (!(txd->flags & DMA_COMPL_SKIP_DEST_UNMAP)) {
- if (txd->flags & DMA_COMPL_DEST_UNMAP_SINGLE)
- dma_unmap_single(dev, dst, len, DMA_FROM_DEVICE);
- else
- dma_unmap_page(dev, dst, len, DMA_FROM_DEVICE);
- }
-
- /* Unmap the src buffer, if requested */
- if (!(txd->flags & DMA_COMPL_SKIP_SRC_UNMAP)) {
- if (txd->flags & DMA_COMPL_SRC_UNMAP_SINGLE)
- dma_unmap_single(dev, src, len, DMA_TO_DEVICE);
- else
- dma_unmap_page(dev, src, len, DMA_TO_DEVICE);
- }
-
- fsl_dma_free_descriptor(chan, desc);
-}
-
-/**
- * fsl_chan_xfer_ld_queue - transfer any pending transactions
- * @chan : Freescale DMA channel
- *
- * HARDWARE STATE: idle
- * LOCKING: must hold chan->desc_lock
- */
-static void fsl_chan_xfer_ld_queue(struct fsldma_chan *chan)
-{
- struct fsl_desc_sw *desc;
-
- /*
- * If the list of pending descriptors is empty, then we
- * don't need to do any work at all
- */
- if (list_empty(&chan->ld_pending)) {
- chan_dbg(chan, "no pending LDs\n");
- return;
- }
-
- /*
- * The DMA controller is not idle, which means that the interrupt
- * handler will start any queued transactions when it runs after
- * this transaction finishes
- */
- if (!chan->idle) {
- chan_dbg(chan, "DMA controller still busy\n");
- return;
- }
-
- /*
- * If there are some link descriptors which have not been
- * transferred, we need to start the controller
- */
-
- /*
- * Move all elements from the queue of pending transactions
- * onto the list of running transactions
- */
- chan_dbg(chan, "idle, starting controller\n");
- desc = list_first_entry(&chan->ld_pending, struct fsl_desc_sw, node);
- list_splice_tail_init(&chan->ld_pending, &chan->ld_running);
-
- /*
- * The 85xx DMA controller doesn't clear the channel start bit
- * automatically at the end of a transfer. Therefore we must clear
- * it in software before starting the transfer.
- */
- if ((chan->feature & FSL_DMA_IP_MASK) == FSL_DMA_IP_85XX) {
- u32 mode;
-
- mode = DMA_IN(chan, &chan->regs->mr, 32);
- mode &= ~FSL_DMA_MR_CS;
- DMA_OUT(chan, &chan->regs->mr, mode, 32);
- }
-
- /*
- * Program the descriptor's address into the DMA controller,
- * then start the DMA transaction
- */
- set_cdar(chan, desc->async_tx.phys);
- get_cdar(chan);
-
- dma_start(chan);
- chan->idle = false;
-}
-
-/**
* fsl_dma_memcpy_issue_pending - Issue the DMA start command
* @chan : Freescale DMA channel
*/
--
1.7.5.1
^ permalink raw reply related
* [PATCH v6 5/8] fsl-dma: change release process of dma descriptor for supporting async_tx
From: qiang.liu @ 2012-08-06 10:14 UTC (permalink / raw)
To: linux-crypto, linuxppc-dev, dan.j.williams, linux-kernel,
dan.j.williams, vinod.koul
Cc: arnd, Ira W. Snyder, gregkh, Qiang Liu, herbert, davem
From: Qiang Liu <qiang.liu@freescale.com>
Fix the potential risk when enable config NET_DMA and ASYNC_TX.
Async_tx is lack of support in current release process of dma descriptor,
all descriptors will be released whatever is acked or no-acked by async_tx,
so there is a potential race condition when dma engine is uesd by others
clients (e.g. when enable NET_DMA to offload TCP).
In our case, a race condition which is raised when use both of talitos
and dmaengine to offload xor is because napi scheduler will sync all
pending requests in dma channels, it affects the process of raid operations
due to ack_tx is not checked in fsl dma. The no-acked descriptor is freed
which is submitted just now, as a dependent tx, this freed descriptor trigger
BUG_ON(async_tx_test_ack(depend_tx)) in async_tx_submit().
TASK = ee1a94a0[1390] 'md0_raid5' THREAD: ecf40000 CPU: 0
GPR00: 00000001 ecf41ca0 ee44/921a94a0 0000003f 00000001 c00593e4 00000000 00000001
GPR08: 00000000 a7a7a7a7 00000001 045/920000002 42028042 100a38d4 ed576d98 00000000
GPR16: ed5a11b0 00000000 2b162000 00000200 046/920000000 2d555000 ed3015e8 c15a7aa0
GPR24: 00000000 c155fc40 00000000 ecb63220 ecf41d28 e47/92f640bb0 ef640c30 ecf41ca0
NIP [c02b048c] async_tx_submit+0x6c/0x2b4
LR [c02b068c] async_tx_submit+0x26c/0x2b4
Call Trace:
[ecf41ca0] [c02b068c] async_tx_submit+0x26c/0x2b448/92 (unreliable)
[ecf41cd0] [c02b0a4c] async_memcpy+0x240/0x25c
[ecf41d20] [c0421064] async_copy_data+0xa0/0x17c
[ecf41d70] [c0421cf4] __raid_run_ops+0x874/0xe10
[ecf41df0] [c0426ee4] handle_stripe+0x820/0x25e8
[ecf41e90] [c0429080] raid5d+0x3d4/0x5b4
[ecf41f40] [c04329b8] md_thread+0x138/0x16c
[ecf41f90] [c008277c] kthread+0x8c/0x90
[ecf41ff0] [c0011630] kernel_thread+0x4c/0x68
Another modification in this patch is the change of completed descriptors,
there is a potential risk which caused by exception interrupt, all descriptors
in ld_running list are seemed completed when an interrupt raised, it works fine
under normal condition, but if there is an exception occured, it cannot work
as our excepted. Hardware should not be depend on s/w list, the right way is
to read current descriptor address register to find the last completed
descriptor. If an interrupt is raised by an error, all descriptors in ld_running
should not be seemed finished, or these unfinished descriptors in ld_running
will be released wrongly.
A simple way to reproduce,
Enable dmatest first, then insert some bad descriptors which can trigger
Programming Error interrupts before the good descriptors. Last, the good
descriptors will be freed before they are processsed because of the exception
intrerrupt.
Note: the bad descriptors are only for simulating an exception interrupt.
This case can illustrate the potential risk in current fsl-dma very well.
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dan Williams <dan.j.williams@gmail.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Li Yang <leoli@freescale.com>
Signed-off-by: Qiang Liu <qiang.liu@freescale.com>
Signed-off-by: Ira W. Snyder <iws@ovro.caltech.edu>
---
drivers/dma/fsldma.c | 232 ++++++++++++++++++++++++++++++++++----------------
drivers/dma/fsldma.h | 17 +++-
2 files changed, 174 insertions(+), 75 deletions(-)
diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c
index 36490a3..938d8c1 100644
--- a/drivers/dma/fsldma.c
+++ b/drivers/dma/fsldma.c
@@ -472,6 +472,110 @@ static struct fsl_desc_sw *fsl_dma_alloc_descriptor(struct fsldma_chan *chan)
}
/**
+ * fsldma_clean_completed_descriptor - free all descriptors which
+ * has been completed and acked
+ * @chan: Freescale DMA channel
+ *
+ * This function is used on all completed and acked descriptors.
+ * All descriptors should only be freed in this function.
+ */
+static void
+fsldma_clean_completed_descriptor(struct fsldma_chan *chan)
+{
+ struct fsl_desc_sw *desc, *_desc;
+
+ /* Run the callback for each descriptor, in order */
+ list_for_each_entry_safe(desc, _desc, &chan->ld_completed, node)
+ if (async_tx_test_ack(&desc->async_tx))
+ fsl_dma_free_descriptor(chan, desc);
+}
+
+/**
+ * fsldma_run_tx_complete_actions - cleanup a single link descriptor
+ * @chan: Freescale DMA channel
+ * @desc: descriptor to cleanup and free
+ * @cookie: Freescale DMA transaction identifier
+ *
+ * This function is used on a descriptor which has been executed by the DMA
+ * controller. It will run any callbacks, submit any dependencies.
+ */
+static dma_cookie_t fsldma_run_tx_complete_actions(struct fsldma_chan *chan,
+ struct fsl_desc_sw *desc, dma_cookie_t cookie)
+{
+ struct dma_async_tx_descriptor *txd = &desc->async_tx;
+ struct device *dev = chan->common.device->dev;
+ dma_addr_t src = get_desc_src(chan, desc);
+ dma_addr_t dst = get_desc_dst(chan, desc);
+ u32 len = get_desc_cnt(chan, desc);
+
+ BUG_ON(txd->cookie < 0);
+
+ if (txd->cookie > 0) {
+ cookie = txd->cookie;
+
+ /* Run the link descriptor callback function */
+ if (txd->callback) {
+#ifdef FSL_DMA_LD_DEBUG
+ chan_dbg(chan, "LD %p callback\n", desc);
+#endif
+ txd->callback(txd->callback_param);
+ }
+
+ /* Unmap the dst buffer, if requested */
+ if (!(txd->flags & DMA_COMPL_SKIP_DEST_UNMAP)) {
+ if (txd->flags & DMA_COMPL_DEST_UNMAP_SINGLE)
+ dma_unmap_single(dev, dst, len, DMA_FROM_DEVICE);
+ else
+ dma_unmap_page(dev, dst, len, DMA_FROM_DEVICE);
+ }
+
+ /* Unmap the src buffer, if requested */
+ if (!(txd->flags & DMA_COMPL_SKIP_SRC_UNMAP)) {
+ if (txd->flags & DMA_COMPL_SRC_UNMAP_SINGLE)
+ dma_unmap_single(dev, src, len, DMA_TO_DEVICE);
+ else
+ dma_unmap_page(dev, src, len, DMA_TO_DEVICE);
+ }
+ }
+
+ /* Run any dependencies */
+ dma_run_dependencies(txd);
+
+ return cookie;
+}
+
+/**
+ * fsldma_clean_running_descriptor - move the completed descriptor from
+ * ld_running to ld_completed
+ * @chan: Freescale DMA channel
+ * @desc: the descriptor which is completed
+ *
+ * Free the descriptor directly if acked by async_tx api, or move it to
+ * queue ld_completed.
+ */
+static void
+fsldma_clean_running_descriptor(struct fsldma_chan *chan,
+ struct fsl_desc_sw *desc)
+{
+ /* Remove from the list of transactions */
+ list_del(&desc->node);
+ /*
+ * the client is allowed to attach dependent operations
+ * until 'ack' is set
+ */
+ if (!async_tx_test_ack(&desc->async_tx)) {
+ /*
+ * Move this descriptor to the list of descriptors which is
+ * completed, but still awaiting the 'ack' bit to be set.
+ */
+ list_add_tail(&desc->node, &chan->ld_completed);
+ return;
+ }
+
+ dma_pool_free(chan->desc_pool, desc, desc->async_tx.phys);
+}
+
+/**
* fsl_chan_xfer_ld_queue - transfer any pending transactions
* @chan : Freescale DMA channel
*
@@ -539,51 +643,58 @@ static void fsl_chan_xfer_ld_queue(struct fsldma_chan *chan)
}
/**
- * fsldma_cleanup_descriptor - cleanup and free a single link descriptor
+ * fsldma_cleanup_descriptors - cleanup link descriptors which are completed
+ * and move them to ld_completed to free until flag 'ack' is set
* @chan: Freescale DMA channel
- * @desc: descriptor to cleanup and free
*
- * This function is used on a descriptor which has been executed by the DMA
- * controller. It will run any callbacks, submit any dependencies, and then
- * free the descriptor.
+ * This function is used on descriptors which have been executed by the DMA
+ * controller. It will run any callbacks, submit any dependencies, then
+ * free these descriptors if flag 'ack' is set.
*/
-static void fsldma_cleanup_descriptor(struct fsldma_chan *chan,
- struct fsl_desc_sw *desc)
+static void fsldma_cleanup_descriptors(struct fsldma_chan *chan)
{
- struct dma_async_tx_descriptor *txd = &desc->async_tx;
- struct device *dev = chan->common.device->dev;
- dma_addr_t src = get_desc_src(chan, desc);
- dma_addr_t dst = get_desc_dst(chan, desc);
- u32 len = get_desc_cnt(chan, desc);
+ struct fsl_desc_sw *desc, *_desc;
+ dma_cookie_t cookie = 0;
+ dma_addr_t curr_phys = get_cdar(chan);
+ int seen_current = 0;
- /* Run the link descriptor callback function */
- if (txd->callback) {
-#ifdef FSL_DMA_LD_DEBUG
- chan_dbg(chan, "LD %p callback\n", desc);
-#endif
- txd->callback(txd->callback_param);
- }
+ fsldma_clean_completed_descriptor(chan);
- /* Run any dependencies */
- dma_run_dependencies(txd);
+ /* Run the callback for each descriptor, in order */
+ list_for_each_entry_safe(desc, _desc, &chan->ld_running, node) {
+ /*
+ * do not advance past the current descriptor loaded into the
+ * hardware channel, subsequent descriptors are either in
+ * process or have not been submitted
+ */
+ if (seen_current)
+ break;
- /* Unmap the dst buffer, if requested */
- if (!(txd->flags & DMA_COMPL_SKIP_DEST_UNMAP)) {
- if (txd->flags & DMA_COMPL_DEST_UNMAP_SINGLE)
- dma_unmap_single(dev, dst, len, DMA_FROM_DEVICE);
- else
- dma_unmap_page(dev, dst, len, DMA_FROM_DEVICE);
- }
+ /*
+ * stop the search if we reach the current descriptor and the
+ * channel is busy
+ */
+ if (desc->async_tx.phys == curr_phys) {
+ seen_current = 1;
+ if (!dma_is_idle(chan))
+ break;
+ }
- /* Unmap the src buffer, if requested */
- if (!(txd->flags & DMA_COMPL_SKIP_SRC_UNMAP)) {
- if (txd->flags & DMA_COMPL_SRC_UNMAP_SINGLE)
- dma_unmap_single(dev, src, len, DMA_TO_DEVICE);
- else
- dma_unmap_page(dev, src, len, DMA_TO_DEVICE);
+ cookie = fsldma_run_tx_complete_actions(chan, desc, cookie);
+
+ fsldma_clean_running_descriptor(chan, desc);
}
- fsl_dma_free_descriptor(chan, desc);
+ /*
+ * Start any pending transactions automatically
+ *
+ * In the ideal case, we keep the DMA controller busy while we go
+ * ahead and free the descriptors below.
+ */
+ fsl_chan_xfer_ld_queue(chan);
+
+ if (cookie > 0)
+ chan->common.completed_cookie = cookie;
}
/**
@@ -654,8 +765,10 @@ static void fsl_dma_free_chan_resources(struct dma_chan *dchan)
chan_dbg(chan, "free all channel resources\n");
spin_lock_irqsave(&chan->desc_lock, flags);
+ fsldma_cleanup_descriptors(chan);
fsldma_free_desc_list(chan, &chan->ld_pending);
fsldma_free_desc_list(chan, &chan->ld_running);
+ fsldma_free_desc_list(chan, &chan->ld_completed);
spin_unlock_irqrestore(&chan->desc_lock, flags);
dma_pool_destroy(chan->desc_pool);
@@ -893,6 +1006,7 @@ static int fsl_dma_device_control(struct dma_chan *dchan,
/* Remove and free all of the descriptors in the LD queue */
fsldma_free_desc_list(chan, &chan->ld_pending);
fsldma_free_desc_list(chan, &chan->ld_running);
+ fsldma_free_desc_list(chan, &chan->ld_completed);
chan->idle = true;
spin_unlock_irqrestore(&chan->desc_lock, flags);
@@ -956,11 +1070,15 @@ static enum dma_status fsl_tx_status(struct dma_chan *dchan,
enum dma_status ret;
unsigned long flags;
- spin_lock_irqsave(&chan->desc_lock, flags);
ret = dma_cookie_status(dchan, cookie, txstate);
+ if (ret == DMA_SUCCESS)
+ return ret;
+
+ spin_lock_irqsave(&chan->desc_lock, flags);
+ fsldma_cleanup_descriptors(chan);
spin_unlock_irqrestore(&chan->desc_lock, flags);
- return ret;
+ return dma_cookie_status(dchan, cookie, txstate);
}
/*----------------------------------------------------------------------------*/
@@ -1037,52 +1155,19 @@ static irqreturn_t fsldma_chan_irq(int irq, void *data)
static void dma_do_tasklet(unsigned long data)
{
struct fsldma_chan *chan = (struct fsldma_chan *)data;
- struct fsl_desc_sw *desc, *_desc;
- LIST_HEAD(ld_cleanup);
unsigned long flags;
chan_dbg(chan, "tasklet entry\n");
spin_lock_irqsave(&chan->desc_lock, flags);
- /* update the cookie if we have some descriptors to cleanup */
- if (!list_empty(&chan->ld_running)) {
- dma_cookie_t cookie;
-
- desc = to_fsl_desc(chan->ld_running.prev);
- cookie = desc->async_tx.cookie;
- dma_cookie_complete(&desc->async_tx);
-
- chan_dbg(chan, "completed_cookie=%d\n", cookie);
- }
-
- /*
- * move the descriptors to a temporary list so we can drop the lock
- * during the entire cleanup operation
- */
- list_splice_tail_init(&chan->ld_running, &ld_cleanup);
-
/* the hardware is now idle and ready for more */
chan->idle = true;
- /*
- * Start any pending transactions automatically
- *
- * In the ideal case, we keep the DMA controller busy while we go
- * ahead and free the descriptors below.
- */
- fsl_chan_xfer_ld_queue(chan);
- spin_unlock_irqrestore(&chan->desc_lock, flags);
-
- /* Run the callback for each descriptor, in order */
- list_for_each_entry_safe(desc, _desc, &ld_cleanup, node) {
+ /* Run all cleanup for descriptors which have been completed */
+ fsldma_cleanup_descriptors(chan);
- /* Remove from the list of transactions */
- list_del(&desc->node);
-
- /* Run all cleanup for this descriptor */
- fsldma_cleanup_descriptor(chan, desc);
- }
+ spin_unlock_irqrestore(&chan->desc_lock, flags);
chan_dbg(chan, "tasklet exit\n");
}
@@ -1264,6 +1349,7 @@ static int __devinit fsl_dma_chan_probe(struct fsldma_device *fdev,
spin_lock_init(&chan->desc_lock);
INIT_LIST_HEAD(&chan->ld_pending);
INIT_LIST_HEAD(&chan->ld_running);
+ INIT_LIST_HEAD(&chan->ld_completed);
chan->idle = true;
chan->common.device = &fdev->common;
diff --git a/drivers/dma/fsldma.h b/drivers/dma/fsldma.h
index f5c3879..a58275a 100644
--- a/drivers/dma/fsldma.h
+++ b/drivers/dma/fsldma.h
@@ -138,8 +138,21 @@ struct fsldma_chan {
char name[8]; /* Channel name */
struct fsldma_chan_regs __iomem *regs;
spinlock_t desc_lock; /* Descriptor operation lock */
- struct list_head ld_pending; /* Link descriptors queue */
- struct list_head ld_running; /* Link descriptors queue */
+ /*
+ * Descriptors which are queued to run, but have not yet been
+ * submitted to the hardware for execution
+ */
+ struct list_head ld_pending;
+ /*
+ * Descriptors which are currently being executed by the hardware
+ */
+ struct list_head ld_running;
+ /*
+ * Descriptors which have finished execution by the hardware. These
+ * descriptors have already had their cleanup actions run. They are
+ * waiting for the ACK bit to be set by the async_tx API.
+ */
+ struct list_head ld_completed; /* Link descriptors queue */
struct dma_chan common; /* DMA common channel */
struct dma_pool *desc_pool; /* Descriptors pool */
struct device *dev; /* Channel device */
--
1.7.5.1
^ permalink raw reply related
* [PATCH 6/8] fsl-dma: use spin_lock_bh to instead of spin_lock_irqsave
From: qiang.liu @ 2012-08-06 10:14 UTC (permalink / raw)
To: linux-crypto, linuxppc-dev, dan.j.williams, linux-kernel,
dan.j.williams, vinod.koul
Cc: arnd, gregkh, Timur Tabi, Qiang Liu, herbert, davem
From: Qiang Liu <qiang.liu@freescale.com>
The use of spin_lock_irqsave() is a stronger locking mechanism than is
required throughout the driver. The minimum locking required should be
used instead. Interrupts will be turned off and context will be saved,
there is needless to use irqsave.
Change all instances of spin_lock_irqsave() to spin_lock_bh().
All manipulation of protected fields is done using tasklet context or
weaker, which makes spin_lock_bh() the correct choice.
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dan Williams <dan.j.williams@gmail.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Li Yang <leoli@freescale.com>
Cc: Timur Tabi <timur@freescale.com>
Signed-off-by: Qiang Liu <qiang.liu@freescale.com>
Acked-by: Ira W. Snyder <iws@ovro.caltech.edu>
---
drivers/dma/fsldma.c | 30 ++++++++++++------------------
1 files changed, 12 insertions(+), 18 deletions(-)
diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c
index 938d8c1..3f809df 100644
--- a/drivers/dma/fsldma.c
+++ b/drivers/dma/fsldma.c
@@ -405,10 +405,9 @@ static dma_cookie_t fsl_dma_tx_submit(struct dma_async_tx_descriptor *tx)
struct fsldma_chan *chan = to_fsl_chan(tx->chan);
struct fsl_desc_sw *desc = tx_to_fsl_desc(tx);
struct fsl_desc_sw *child;
- unsigned long flags;
dma_cookie_t cookie;
- spin_lock_irqsave(&chan->desc_lock, flags);
+ spin_lock_bh(&chan->desc_lock);
/*
* assign cookies to all of the software descriptors
@@ -421,7 +420,7 @@ static dma_cookie_t fsl_dma_tx_submit(struct dma_async_tx_descriptor *tx)
/* put this transaction onto the tail of the pending queue */
append_ld_queue(chan, desc);
- spin_unlock_irqrestore(&chan->desc_lock, flags);
+ spin_unlock_bh(&chan->desc_lock);
return cookie;
}
@@ -761,15 +760,14 @@ static void fsldma_free_desc_list_reverse(struct fsldma_chan *chan,
static void fsl_dma_free_chan_resources(struct dma_chan *dchan)
{
struct fsldma_chan *chan = to_fsl_chan(dchan);
- unsigned long flags;
chan_dbg(chan, "free all channel resources\n");
- spin_lock_irqsave(&chan->desc_lock, flags);
+ spin_lock_bh(&chan->desc_lock);
fsldma_cleanup_descriptors(chan);
fsldma_free_desc_list(chan, &chan->ld_pending);
fsldma_free_desc_list(chan, &chan->ld_running);
fsldma_free_desc_list(chan, &chan->ld_completed);
- spin_unlock_irqrestore(&chan->desc_lock, flags);
+ spin_unlock_bh(&chan->desc_lock);
dma_pool_destroy(chan->desc_pool);
chan->desc_pool = NULL;
@@ -988,7 +986,6 @@ static int fsl_dma_device_control(struct dma_chan *dchan,
{
struct dma_slave_config *config;
struct fsldma_chan *chan;
- unsigned long flags;
int size;
if (!dchan)
@@ -998,7 +995,7 @@ static int fsl_dma_device_control(struct dma_chan *dchan,
switch (cmd) {
case DMA_TERMINATE_ALL:
- spin_lock_irqsave(&chan->desc_lock, flags);
+ spin_lock_bh(&chan->desc_lock);
/* Halt the DMA engine */
dma_halt(chan);
@@ -1009,7 +1006,7 @@ static int fsl_dma_device_control(struct dma_chan *dchan,
fsldma_free_desc_list(chan, &chan->ld_completed);
chan->idle = true;
- spin_unlock_irqrestore(&chan->desc_lock, flags);
+ spin_unlock_bh(&chan->desc_lock);
return 0;
case DMA_SLAVE_CONFIG:
@@ -1051,11 +1048,10 @@ static int fsl_dma_device_control(struct dma_chan *dchan,
static void fsl_dma_memcpy_issue_pending(struct dma_chan *dchan)
{
struct fsldma_chan *chan = to_fsl_chan(dchan);
- unsigned long flags;
- spin_lock_irqsave(&chan->desc_lock, flags);
+ spin_lock_bh(&chan->desc_lock);
fsl_chan_xfer_ld_queue(chan);
- spin_unlock_irqrestore(&chan->desc_lock, flags);
+ spin_unlock_bh(&chan->desc_lock);
}
/**
@@ -1068,15 +1064,14 @@ static enum dma_status fsl_tx_status(struct dma_chan *dchan,
{
struct fsldma_chan *chan = to_fsl_chan(dchan);
enum dma_status ret;
- unsigned long flags;
ret = dma_cookie_status(dchan, cookie, txstate);
if (ret == DMA_SUCCESS)
return ret;
- spin_lock_irqsave(&chan->desc_lock, flags);
+ spin_lock_bh(&chan->desc_lock);
fsldma_cleanup_descriptors(chan);
- spin_unlock_irqrestore(&chan->desc_lock, flags);
+ spin_unlock_bh(&chan->desc_lock);
return dma_cookie_status(dchan, cookie, txstate);
}
@@ -1155,11 +1150,10 @@ static irqreturn_t fsldma_chan_irq(int irq, void *data)
static void dma_do_tasklet(unsigned long data)
{
struct fsldma_chan *chan = (struct fsldma_chan *)data;
- unsigned long flags;
chan_dbg(chan, "tasklet entry\n");
- spin_lock_irqsave(&chan->desc_lock, flags);
+ spin_lock_bh(&chan->desc_lock);
/* the hardware is now idle and ready for more */
chan->idle = true;
@@ -1167,7 +1161,7 @@ static void dma_do_tasklet(unsigned long data)
/* Run all cleanup for descriptors which have been completed */
fsldma_cleanup_descriptors(chan);
- spin_unlock_irqrestore(&chan->desc_lock, flags);
+ spin_unlock_bh(&chan->desc_lock);
chan_dbg(chan, "tasklet exit\n");
}
--
1.7.5.1
^ permalink raw reply related
* [PATCH v6 7/8] fsl-dma: fix a warning of unitialized cookie
From: qiang.liu @ 2012-08-06 10:14 UTC (permalink / raw)
To: linux-crypto, linuxppc-dev, dan.j.williams, linux-kernel,
dan.j.williams, vinod.koul
Cc: arnd, gregkh, Qiang Liu, herbert, davem
From: Qiang Liu <qiang.liu@freescale.com>
Fix a warning of unitialized value when compile with -Wuninitialized.
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dan Williams <dan.j.williams@gmail.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Li Yang <leoli@freescale.com>
Signed-off-by: Qiang Liu <qiang.liu@freescale.com>
Reported-by: Kim Phillips <kim.phillips@freescale.com>
Acked-by: Ira W. Snyder <iws@ovro.caltech.edu>
---
drivers/dma/fsldma.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c
index 3f809df..6275bdf 100644
--- a/drivers/dma/fsldma.c
+++ b/drivers/dma/fsldma.c
@@ -405,7 +405,7 @@ static dma_cookie_t fsl_dma_tx_submit(struct dma_async_tx_descriptor *tx)
struct fsldma_chan *chan = to_fsl_chan(tx->chan);
struct fsl_desc_sw *desc = tx_to_fsl_desc(tx);
struct fsl_desc_sw *child;
- dma_cookie_t cookie;
+ dma_cookie_t cookie = 0;
spin_lock_bh(&chan->desc_lock);
--
1.7.5.1
^ permalink raw reply related
* [PATCH v6 8/8] carma: remove unnecessary DMA_INTERRUPT capability
From: qiang.liu @ 2012-08-06 10:15 UTC (permalink / raw)
To: linux-crypto, linuxppc-dev, dan.j.williams, arnd, gregkh,
linux-kernel, dan.j.williams, vinod.koul
Cc: Qiang Liu, herbert, davem, Ira W. Snyder
From: Qiang Liu <qiang.liu@freescale.com>
These drivers set the DMA_INTERRUPT capability bit when requesting a DMA
controller channel. This was historical, and is no longer needed.
Recent changes to the drivers/dma/fsldma.c driver have removed support
for this flag. This makes the carma drivers unable to find a DMA channel
with the required capabilities.
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Ira W. Snyder <iws@ovro.caltech.edu>
---
Based on git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git
against branch char-misc-next.
drivers/misc/carma/carma-fpga-program.c | 1 -
drivers/misc/carma/carma-fpga.c | 2 +-
2 files changed, 1 insertions(+), 2 deletions(-)
diff --git a/drivers/misc/carma/carma-fpga-program.c b/drivers/misc/carma/carma-fpga-program.c
index a2d25e4..eaddfe9 100644
--- a/drivers/misc/carma/carma-fpga-program.c
+++ b/drivers/misc/carma/carma-fpga-program.c
@@ -978,7 +978,6 @@ static int fpga_of_probe(struct platform_device *op)
dev_set_drvdata(priv->dev, priv);
dma_cap_zero(mask);
dma_cap_set(DMA_MEMCPY, mask);
- dma_cap_set(DMA_INTERRUPT, mask);
dma_cap_set(DMA_SLAVE, mask);
dma_cap_set(DMA_SG, mask);
diff --git a/drivers/misc/carma/carma-fpga.c b/drivers/misc/carma/carma-fpga.c
index 8c279da..0c43297 100644
--- a/drivers/misc/carma/carma-fpga.c
+++ b/drivers/misc/carma/carma-fpga.c
@@ -666,7 +666,7 @@ static int data_submit_dma(struct fpga_device *priv, struct data_buf *buf)
src = SYS_FPGA_BLOCK;
tx = chan->device->device_prep_dma_memcpy(chan, dst, src,
REG_BLOCK_SIZE,
- DMA_PREP_INTERRUPT);
+ 0);
if (!tx) {
dev_err(priv->dev, "unable to prep SYS-FPGA DMA\n");
return -ENOMEM;
--
1.7.5.1
^ permalink raw reply related
* RE: [PATCH 6/8] fsl-dma: use spin_lock_bh to instead of spin_lock_irqsave
From: Liu Qiang-B32616 @ 2012-08-06 11:01 UTC (permalink / raw)
To: Liu Qiang-B32616, linux-crypto@vger.kernel.org,
linuxppc-dev@lists.ozlabs.org, dan.j.williams@intel.com,
linux-kernel@vger.kernel.org, dan.j.williams@gmail.com,
vinod.koul@intel.com
Cc: Li Yang-R58472, arnd@arndb.de, gregkh@linuxfoundation.org,
Tabi Timur-B04825, Phillips Kim-R1AAHA,
herbert@gondor.hengli.com.au, davem@davemloft.net
In-Reply-To: <1344248085-9315-1-git-send-email-qiang.liu@freescale.com>
Hi all,
Please ignore this one. Thanks.
> -----Original Message-----
> From: Liu Qiang-B32616
> Sent: Monday, August 06, 2012 6:15 PM
> To: linux-crypto@vger.kernel.org; linuxppc-dev@lists.ozlabs.org;
> dan.j.williams@intel.com; linux-kernel@vger.kernel.org;
> dan.j.williams@gmail.com; vinod.koul@intel.com
> Cc: Phillips Kim-R1AAHA; herbert@gondor.hengli.com.au;
> davem@davemloft.net; arnd@arndb.de; gregkh@linuxfoundation.org; Liu
> Qiang-B32616; Li Yang-R58472; Tabi Timur-B04825
> Subject: [PATCH 6/8] fsl-dma: use spin_lock_bh to instead of
> spin_lock_irqsave
>=20
> From: Qiang Liu <qiang.liu@freescale.com>
>=20
> The use of spin_lock_irqsave() is a stronger locking mechanism than is
> required throughout the driver. The minimum locking required should be
> used instead. Interrupts will be turned off and context will be saved,
> there is needless to use irqsave.
>=20
> Change all instances of spin_lock_irqsave() to spin_lock_bh().
> All manipulation of protected fields is done using tasklet context or
> weaker, which makes spin_lock_bh() the correct choice.
>=20
> Cc: Dan Williams <dan.j.williams@intel.com>
> Cc: Dan Williams <dan.j.williams@gmail.com>
> Cc: Vinod Koul <vinod.koul@intel.com>
> Cc: Li Yang <leoli@freescale.com>
> Cc: Timur Tabi <timur@freescale.com>
> Signed-off-by: Qiang Liu <qiang.liu@freescale.com>
> Acked-by: Ira W. Snyder <iws@ovro.caltech.edu>
> ---
> drivers/dma/fsldma.c | 30 ++++++++++++------------------
> 1 files changed, 12 insertions(+), 18 deletions(-)
>=20
> diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c
> index 938d8c1..3f809df 100644
> --- a/drivers/dma/fsldma.c
> +++ b/drivers/dma/fsldma.c
> @@ -405,10 +405,9 @@ static dma_cookie_t fsl_dma_tx_submit(struct
> dma_async_tx_descriptor *tx)
> struct fsldma_chan *chan =3D to_fsl_chan(tx->chan);
> struct fsl_desc_sw *desc =3D tx_to_fsl_desc(tx);
> struct fsl_desc_sw *child;
> - unsigned long flags;
> dma_cookie_t cookie;
>=20
> - spin_lock_irqsave(&chan->desc_lock, flags);
> + spin_lock_bh(&chan->desc_lock);
>=20
> /*
> * assign cookies to all of the software descriptors
> @@ -421,7 +420,7 @@ static dma_cookie_t fsl_dma_tx_submit(struct
> dma_async_tx_descriptor *tx)
> /* put this transaction onto the tail of the pending queue */
> append_ld_queue(chan, desc);
>=20
> - spin_unlock_irqrestore(&chan->desc_lock, flags);
> + spin_unlock_bh(&chan->desc_lock);
>=20
> return cookie;
> }
> @@ -761,15 +760,14 @@ static void fsldma_free_desc_list_reverse(struct
> fsldma_chan *chan,
> static void fsl_dma_free_chan_resources(struct dma_chan *dchan)
> {
> struct fsldma_chan *chan =3D to_fsl_chan(dchan);
> - unsigned long flags;
>=20
> chan_dbg(chan, "free all channel resources\n");
> - spin_lock_irqsave(&chan->desc_lock, flags);
> + spin_lock_bh(&chan->desc_lock);
> fsldma_cleanup_descriptors(chan);
> fsldma_free_desc_list(chan, &chan->ld_pending);
> fsldma_free_desc_list(chan, &chan->ld_running);
> fsldma_free_desc_list(chan, &chan->ld_completed);
> - spin_unlock_irqrestore(&chan->desc_lock, flags);
> + spin_unlock_bh(&chan->desc_lock);
>=20
> dma_pool_destroy(chan->desc_pool);
> chan->desc_pool =3D NULL;
> @@ -988,7 +986,6 @@ static int fsl_dma_device_control(struct dma_chan
> *dchan,
> {
> struct dma_slave_config *config;
> struct fsldma_chan *chan;
> - unsigned long flags;
> int size;
>=20
> if (!dchan)
> @@ -998,7 +995,7 @@ static int fsl_dma_device_control(struct dma_chan
> *dchan,
>=20
> switch (cmd) {
> case DMA_TERMINATE_ALL:
> - spin_lock_irqsave(&chan->desc_lock, flags);
> + spin_lock_bh(&chan->desc_lock);
>=20
> /* Halt the DMA engine */
> dma_halt(chan);
> @@ -1009,7 +1006,7 @@ static int fsl_dma_device_control(struct dma_chan
> *dchan,
> fsldma_free_desc_list(chan, &chan->ld_completed);
> chan->idle =3D true;
>=20
> - spin_unlock_irqrestore(&chan->desc_lock, flags);
> + spin_unlock_bh(&chan->desc_lock);
> return 0;
>=20
> case DMA_SLAVE_CONFIG:
> @@ -1051,11 +1048,10 @@ static int fsl_dma_device_control(struct dma_chan
> *dchan,
> static void fsl_dma_memcpy_issue_pending(struct dma_chan *dchan)
> {
> struct fsldma_chan *chan =3D to_fsl_chan(dchan);
> - unsigned long flags;
>=20
> - spin_lock_irqsave(&chan->desc_lock, flags);
> + spin_lock_bh(&chan->desc_lock);
> fsl_chan_xfer_ld_queue(chan);
> - spin_unlock_irqrestore(&chan->desc_lock, flags);
> + spin_unlock_bh(&chan->desc_lock);
> }
>=20
> /**
> @@ -1068,15 +1064,14 @@ static enum dma_status fsl_tx_status(struct
> dma_chan *dchan,
> {
> struct fsldma_chan *chan =3D to_fsl_chan(dchan);
> enum dma_status ret;
> - unsigned long flags;
>=20
> ret =3D dma_cookie_status(dchan, cookie, txstate);
> if (ret =3D=3D DMA_SUCCESS)
> return ret;
>=20
> - spin_lock_irqsave(&chan->desc_lock, flags);
> + spin_lock_bh(&chan->desc_lock);
> fsldma_cleanup_descriptors(chan);
> - spin_unlock_irqrestore(&chan->desc_lock, flags);
> + spin_unlock_bh(&chan->desc_lock);
>=20
> return dma_cookie_status(dchan, cookie, txstate);
> }
> @@ -1155,11 +1150,10 @@ static irqreturn_t fsldma_chan_irq(int irq, void
> *data)
> static void dma_do_tasklet(unsigned long data)
> {
> struct fsldma_chan *chan =3D (struct fsldma_chan *)data;
> - unsigned long flags;
>=20
> chan_dbg(chan, "tasklet entry\n");
>=20
> - spin_lock_irqsave(&chan->desc_lock, flags);
> + spin_lock_bh(&chan->desc_lock);
>=20
> /* the hardware is now idle and ready for more */
> chan->idle =3D true;
> @@ -1167,7 +1161,7 @@ static void dma_do_tasklet(unsigned long data)
> /* Run all cleanup for descriptors which have been completed */
> fsldma_cleanup_descriptors(chan);
>=20
> - spin_unlock_irqrestore(&chan->desc_lock, flags);
> + spin_unlock_bh(&chan->desc_lock);
>=20
> chan_dbg(chan, "tasklet exit\n");
> }
> --
> 1.7.5.1
^ permalink raw reply
* [PATCH v6 6/8] fsl-dma: use spin_lock_bh to instead of spin_lock_irqsave
From: qiang.liu @ 2012-08-06 10:36 UTC (permalink / raw)
To: linux-crypto, linuxppc-dev, dan.j.williams, linux-kernel,
dan.j.williams, vinod.koul
Cc: arnd, gregkh, Timur Tabi, Qiang Liu, herbert, davem
From: Qiang Liu <qiang.liu@freescale.com>
The use of spin_lock_irqsave() is a stronger locking mechanism than is
required throughout the driver. The minimum locking required should be
used instead. Interrupts will be turned off and context will be saved,
there is needless to use irqsave.
Change all instances of spin_lock_irqsave() to spin_lock_bh().
All manipulation of protected fields is done using tasklet context or
weaker, which makes spin_lock_bh() the correct choice.
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dan Williams <dan.j.williams@gmail.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Li Yang <leoli@freescale.com>
Cc: Timur Tabi <timur@freescale.com>
Signed-off-by: Qiang Liu <qiang.liu@freescale.com>
Acked-by: Ira W. Snyder <iws@ovro.caltech.edu>
---
drivers/dma/fsldma.c | 30 ++++++++++++------------------
1 files changed, 12 insertions(+), 18 deletions(-)
diff --git a/drivers/dma/fsldma.c b/drivers/dma/fsldma.c
index 938d8c1..3f809df 100644
--- a/drivers/dma/fsldma.c
+++ b/drivers/dma/fsldma.c
@@ -405,10 +405,9 @@ static dma_cookie_t fsl_dma_tx_submit(struct dma_async_tx_descriptor *tx)
struct fsldma_chan *chan = to_fsl_chan(tx->chan);
struct fsl_desc_sw *desc = tx_to_fsl_desc(tx);
struct fsl_desc_sw *child;
- unsigned long flags;
dma_cookie_t cookie;
- spin_lock_irqsave(&chan->desc_lock, flags);
+ spin_lock_bh(&chan->desc_lock);
/*
* assign cookies to all of the software descriptors
@@ -421,7 +420,7 @@ static dma_cookie_t fsl_dma_tx_submit(struct dma_async_tx_descriptor *tx)
/* put this transaction onto the tail of the pending queue */
append_ld_queue(chan, desc);
- spin_unlock_irqrestore(&chan->desc_lock, flags);
+ spin_unlock_bh(&chan->desc_lock);
return cookie;
}
@@ -761,15 +760,14 @@ static void fsldma_free_desc_list_reverse(struct fsldma_chan *chan,
static void fsl_dma_free_chan_resources(struct dma_chan *dchan)
{
struct fsldma_chan *chan = to_fsl_chan(dchan);
- unsigned long flags;
chan_dbg(chan, "free all channel resources\n");
- spin_lock_irqsave(&chan->desc_lock, flags);
+ spin_lock_bh(&chan->desc_lock);
fsldma_cleanup_descriptors(chan);
fsldma_free_desc_list(chan, &chan->ld_pending);
fsldma_free_desc_list(chan, &chan->ld_running);
fsldma_free_desc_list(chan, &chan->ld_completed);
- spin_unlock_irqrestore(&chan->desc_lock, flags);
+ spin_unlock_bh(&chan->desc_lock);
dma_pool_destroy(chan->desc_pool);
chan->desc_pool = NULL;
@@ -988,7 +986,6 @@ static int fsl_dma_device_control(struct dma_chan *dchan,
{
struct dma_slave_config *config;
struct fsldma_chan *chan;
- unsigned long flags;
int size;
if (!dchan)
@@ -998,7 +995,7 @@ static int fsl_dma_device_control(struct dma_chan *dchan,
switch (cmd) {
case DMA_TERMINATE_ALL:
- spin_lock_irqsave(&chan->desc_lock, flags);
+ spin_lock_bh(&chan->desc_lock);
/* Halt the DMA engine */
dma_halt(chan);
@@ -1009,7 +1006,7 @@ static int fsl_dma_device_control(struct dma_chan *dchan,
fsldma_free_desc_list(chan, &chan->ld_completed);
chan->idle = true;
- spin_unlock_irqrestore(&chan->desc_lock, flags);
+ spin_unlock_bh(&chan->desc_lock);
return 0;
case DMA_SLAVE_CONFIG:
@@ -1051,11 +1048,10 @@ static int fsl_dma_device_control(struct dma_chan *dchan,
static void fsl_dma_memcpy_issue_pending(struct dma_chan *dchan)
{
struct fsldma_chan *chan = to_fsl_chan(dchan);
- unsigned long flags;
- spin_lock_irqsave(&chan->desc_lock, flags);
+ spin_lock_bh(&chan->desc_lock);
fsl_chan_xfer_ld_queue(chan);
- spin_unlock_irqrestore(&chan->desc_lock, flags);
+ spin_unlock_bh(&chan->desc_lock);
}
/**
@@ -1068,15 +1064,14 @@ static enum dma_status fsl_tx_status(struct dma_chan *dchan,
{
struct fsldma_chan *chan = to_fsl_chan(dchan);
enum dma_status ret;
- unsigned long flags;
ret = dma_cookie_status(dchan, cookie, txstate);
if (ret == DMA_SUCCESS)
return ret;
- spin_lock_irqsave(&chan->desc_lock, flags);
+ spin_lock_bh(&chan->desc_lock);
fsldma_cleanup_descriptors(chan);
- spin_unlock_irqrestore(&chan->desc_lock, flags);
+ spin_unlock_bh(&chan->desc_lock);
return dma_cookie_status(dchan, cookie, txstate);
}
@@ -1155,11 +1150,10 @@ static irqreturn_t fsldma_chan_irq(int irq, void *data)
static void dma_do_tasklet(unsigned long data)
{
struct fsldma_chan *chan = (struct fsldma_chan *)data;
- unsigned long flags;
chan_dbg(chan, "tasklet entry\n");
- spin_lock_irqsave(&chan->desc_lock, flags);
+ spin_lock_bh(&chan->desc_lock);
/* the hardware is now idle and ready for more */
chan->idle = true;
@@ -1167,7 +1161,7 @@ static void dma_do_tasklet(unsigned long data)
/* Run all cleanup for descriptors which have been completed */
fsldma_cleanup_descriptors(chan);
- spin_unlock_irqrestore(&chan->desc_lock, flags);
+ spin_unlock_bh(&chan->desc_lock);
chan_dbg(chan, "tasklet exit\n");
}
--
1.7.5.1
^ permalink raw reply related
* Re: [PATCH v6 6/8] fsl-dma: use spin_lock_bh to instead of spin_lock_irqsave
From: Arnd Bergmann @ 2012-08-06 11:57 UTC (permalink / raw)
To: qiang.liu
Cc: vinod.koul, gregkh, linux-kernel, Timur Tabi, dan.j.williams,
herbert, linux-crypto, dan.j.williams, linuxppc-dev, davem
In-Reply-To: <1344249361-10263-1-git-send-email-qiang.liu@freescale.com>
On Monday 06 August 2012, qiang.liu@freescale.com wrote:
>
> From: Qiang Liu <qiang.liu@freescale.com>
>
> The use of spin_lock_irqsave() is a stronger locking mechanism than is
> required throughout the driver. The minimum locking required should be
> used instead. Interrupts will be turned off and context will be saved,
> there is needless to use irqsave.
>
> Change all instances of spin_lock_irqsave() to spin_lock_bh().
> All manipulation of protected fields is done using tasklet context or
> weaker, which makes spin_lock_bh() the correct choice.
>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Cc: Dan Williams <dan.j.williams@gmail.com>
> Cc: Vinod Koul <vinod.koul@intel.com>
> Cc: Li Yang <leoli@freescale.com>
> Cc: Timur Tabi <timur@freescale.com>
> Signed-off-by: Qiang Liu <qiang.liu@freescale.com>
> Acked-by: Ira W. Snyder <iws@ovro.caltech.edu>
Acked-by: Arnd Bergmann <arnd@arndb.de>
You could actually change the use of spin_lock_bh inside of the tasklet
function (dma_do_tasklet) do just spin_lock(), because softirqs are
already disabled there, but your version is also ok.
^ permalink raw reply
* [PATCH 3/3 v4] powerpc/mpic: FSL MPIC error interrupt support.
From: Varun Sethi @ 2012-08-06 12:44 UTC (permalink / raw)
To: galak, linuxppc-dev; +Cc: Bogdan Hamciuc, Varun Sethi
All SOC device error interrupts are muxed and delivered to the core
as a single MPIC error interrupt. Currently all the device drivers
requiring access to device errors have to register for the MPIC error
interrupt as a shared interrupt.
With this patch we add interrupt demuxing capability in the mpic driver,
allowing device drivers to register for their individual error interrupts.
This is achieved by handling error interrupts in a cascaded fashion.
MPIC error interrupt is handled by the "error_int_handler", which
subsequently demuxes it using the EISR and delivers it to the respective
drivers.
The error interrupt capability is dependent on the MPIC EIMR register,
which was introduced in FSL MPIC version 4.1 (P4080 rev2). So, error
interrupt demuxing capability is dependent on the MPIC version and can
be used for versions >= 4.1.
Signed-off-by: Varun Sethi <Varun.Sethi@freescale.com>
Signed-off-by: Bogdan Hamciuc <bogdan.hamciuc@freescale.com>
[In the initial version of the patch we were using handle_simple_irq
as the handler for cascaded error interrupts, this resulted
in issues in case of threaded isrs (with RT kernel). This issue was
debugged by Bogdan and decision was taken to use the handle_level_irq
handler]
---
arch/powerpc/include/asm/mpic.h | 16 ++++
arch/powerpc/sysdev/Makefile | 2 +-
arch/powerpc/sysdev/fsl_mpic_err.c | 153 ++++++++++++++++++++++++++++++++++++
arch/powerpc/sysdev/mpic.c | 45 ++++++++++-
arch/powerpc/sysdev/mpic.h | 22 +++++
5 files changed, 236 insertions(+), 2 deletions(-)
create mode 100644 arch/powerpc/sysdev/fsl_mpic_err.c
diff --git a/arch/powerpc/include/asm/mpic.h b/arch/powerpc/include/asm/mpic.h
index e14d35d..6c8e53b 100644
--- a/arch/powerpc/include/asm/mpic.h
+++ b/arch/powerpc/include/asm/mpic.h
@@ -118,6 +118,9 @@
#define MPIC_MAX_CPUS 32
#define MPIC_MAX_ISU 32
+#define MPIC_MAX_ERR 32
+#define MPIC_FSL_ERR_INT 16
+
/*
* Tsi108 implementation of MPIC has many differences from the original one
*/
@@ -270,6 +273,7 @@ struct mpic
struct irq_chip hc_ipi;
#endif
struct irq_chip hc_tm;
+ struct irq_chip hc_err;
const char *name;
/* Flags */
unsigned int flags;
@@ -283,6 +287,8 @@ struct mpic
/* vector numbers used for internal sources (ipi/timers) */
unsigned int ipi_vecs[4];
unsigned int timer_vecs[8];
+ /* vector numbers used for FSL MPIC error interrupts */
+ unsigned int err_int_vecs[MPIC_MAX_ERR];
/* Spurious vector to program into unused sources */
unsigned int spurious_vec;
@@ -306,6 +312,11 @@ struct mpic
struct mpic_reg_bank cpuregs[MPIC_MAX_CPUS];
struct mpic_reg_bank isus[MPIC_MAX_ISU];
+ /* ioremap'ed base for error interrupt registers */
+ u32 __iomem *err_regs;
+ /* error interrupt config */
+ u32 err_int_config_done;
+
/* Protected sources */
unsigned long *protected;
@@ -370,6 +381,11 @@ struct mpic
#define MPIC_NO_RESET 0x00004000
/* Freescale MPIC (compatible includes "fsl,mpic") */
#define MPIC_FSL 0x00008000
+/* Freescale MPIC supports EIMR (error interrupt mask register).
+ * This flag is set for MPIC version >= 4.1 (version determined
+ * from the BRR1 register).
+*/
+#define MPIC_FSL_HAS_EIMR 0x00010000
/* MPIC HW modification ID */
#define MPIC_REGSET_MASK 0xf0000000
diff --git a/arch/powerpc/sysdev/Makefile b/arch/powerpc/sysdev/Makefile
index 1bd7ecb..a57600b 100644
--- a/arch/powerpc/sysdev/Makefile
+++ b/arch/powerpc/sysdev/Makefile
@@ -15,7 +15,7 @@ obj-$(CONFIG_PPC_DCR_NATIVE) += dcr-low.o
obj-$(CONFIG_PPC_PMI) += pmi.o
obj-$(CONFIG_U3_DART) += dart_iommu.o
obj-$(CONFIG_MMIO_NVRAM) += mmio_nvram.o
-obj-$(CONFIG_FSL_SOC) += fsl_soc.o
+obj-$(CONFIG_FSL_SOC) += fsl_soc.o fsl_mpic_err.o
obj-$(CONFIG_FSL_PCI) += fsl_pci.o $(fsl-msi-obj-y)
obj-$(CONFIG_FSL_PMC) += fsl_pmc.o
obj-$(CONFIG_FSL_LBC) += fsl_lbc.o
diff --git a/arch/powerpc/sysdev/fsl_mpic_err.c b/arch/powerpc/sysdev/fsl_mpic_err.c
new file mode 100644
index 0000000..179d054
--- /dev/null
+++ b/arch/powerpc/sysdev/fsl_mpic_err.c
@@ -0,0 +1,153 @@
+/*
+ * Copyright (C) 2012 Freescale Semiconductor, Inc.
+ *
+ * Author: Varun Sethi <varun.sethi@freescale.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; version 2 of the
+ * License.
+ *
+ */
+
+#include <linux/irq.h>
+#include <linux/smp.h>
+#include <linux/interrupt.h>
+
+#include <asm/io.h>
+#include <asm/irq.h>
+#include <asm/mpic.h>
+
+#include "mpic.h"
+
+#define MPIC_ERR_INT_BASE 0x3900
+#define MPIC_ERR_INT_EISR 0x0000
+#define MPIC_ERR_INT_EIMR 0x0010
+
+static inline u32 mpic_fsl_err_read(u32 __iomem *base, unsigned int err_reg)
+{
+ return in_be32(base + (err_reg >> 2));
+}
+
+static inline void mpic_fsl_err_write(u32 __iomem *base, u32 value)
+{
+ out_be32(base + (MPIC_ERR_INT_EIMR >> 2), value);
+}
+
+static void fsl_mpic_mask_err(struct irq_data *d)
+{
+ u32 eimr;
+ struct mpic *mpic = irq_data_get_irq_chip_data(d);
+ unsigned int src = virq_to_hw(d->irq) - mpic->err_int_vecs[0];
+
+ eimr = mpic_fsl_err_read(mpic->err_regs, MPIC_ERR_INT_EIMR);
+ eimr |= (1 << (31 - src));
+ mpic_fsl_err_write(mpic->err_regs, eimr);
+}
+
+static void fsl_mpic_unmask_err(struct irq_data *d)
+{
+ u32 eimr;
+ struct mpic *mpic = irq_data_get_irq_chip_data(d);
+ unsigned int src = virq_to_hw(d->irq) - mpic->err_int_vecs[0];
+
+ eimr = mpic_fsl_err_read(mpic->err_regs, MPIC_ERR_INT_EIMR);
+ eimr &= ~(1 << (31 - src));
+ mpic_fsl_err_write(mpic->err_regs, eimr);
+}
+
+static struct irq_chip fsl_mpic_err_chip = {
+ .irq_disable = fsl_mpic_mask_err,
+ .irq_mask = fsl_mpic_mask_err,
+ .irq_unmask = fsl_mpic_unmask_err,
+};
+
+int mpic_setup_error_int(struct mpic *mpic, int intvec)
+{
+ int i;
+
+ mpic->err_regs = ioremap(mpic->paddr + MPIC_ERR_INT_BASE, 0x1000);
+ if (!mpic->err_regs) {
+ pr_err("could not map mpic error registers\n");
+ return -ENOMEM;
+ }
+ mpic->hc_err = fsl_mpic_err_chip;
+ mpic->hc_err.name = mpic->name;
+ mpic->flags |= MPIC_FSL_HAS_EIMR;
+ /* allocate interrupt vectors for error interrupts */
+ for (i = MPIC_MAX_ERR - 1; i >= 0; i--)
+ mpic->err_int_vecs[i] = --intvec;
+
+ return 0;
+}
+
+int mpic_map_error_int(struct mpic *mpic, unsigned int virq, irq_hw_number_t hw)
+{
+ if ((mpic->flags & MPIC_FSL_HAS_EIMR) &&
+ (hw >= mpic->err_int_vecs[0] &&
+ hw <= mpic->err_int_vecs[MPIC_MAX_ERR - 1])) {
+ WARN_ON(mpic->flags & MPIC_SECONDARY);
+
+ pr_debug("mpic: mapping as Error Interrupt\n");
+ irq_set_chip_data(virq, mpic);
+ irq_set_chip_and_handler(virq, &mpic->hc_err,
+ handle_level_irq);
+ return 1;
+ }
+
+ return 0;
+}
+
+static irqreturn_t fsl_error_int_handler(int irq, void *data)
+{
+ struct mpic *mpic = (struct mpic *) data;
+ u32 eisr, eimr;
+ int errint;
+ unsigned int cascade_irq;
+
+ eisr = mpic_fsl_err_read(mpic->err_regs, MPIC_ERR_INT_EISR);
+ eimr = mpic_fsl_err_read(mpic->err_regs, MPIC_ERR_INT_EIMR);
+
+ if (!(eisr & ~eimr))
+ return IRQ_NONE;
+
+ while (eisr) {
+ errint = __builtin_clz(eisr);
+ cascade_irq = irq_linear_revmap(mpic->irqhost,
+ mpic->err_int_vecs[errint]);
+ WARN_ON(cascade_irq == NO_IRQ);
+ if (cascade_irq != NO_IRQ) {
+ generic_handle_irq(cascade_irq);
+ } else {
+ eimr |= 1 << (31 - errint);
+ mpic_fsl_err_write(mpic->err_regs, eimr);
+ }
+ eisr &= ~(1 << (31 - errint));
+ }
+
+ return IRQ_HANDLED;
+}
+
+int mpic_err_int_init(struct mpic *mpic, irq_hw_number_t irqnum)
+{
+ unsigned int virq;
+ int ret;
+
+ virq = irq_create_mapping(mpic->irqhost, irqnum);
+ if (virq == NO_IRQ) {
+ pr_err("Error interrupt setup failed\n");
+ return 0;
+ }
+
+ /* Mask all error interrupts */
+ mpic_fsl_err_write(mpic->err_regs, ~0);
+
+ ret = request_irq(virq, fsl_error_int_handler, IRQF_NO_THREAD,
+ "mpic-error-int", mpic);
+ if (ret) {
+ pr_err("Failed to register error interrupt handler\n");
+ return 0;
+ }
+
+ return 1;
+}
diff --git a/arch/powerpc/sysdev/mpic.c b/arch/powerpc/sysdev/mpic.c
index 7e32db7..3adb596 100644
--- a/arch/powerpc/sysdev/mpic.c
+++ b/arch/powerpc/sysdev/mpic.c
@@ -1026,6 +1026,9 @@ static int mpic_host_map(struct irq_domain *h, unsigned int virq,
return 0;
}
+ if (mpic_map_error_int(mpic, virq, hw))
+ return 0;
+
if (hw >= mpic->num_sources)
return -EINVAL;
@@ -1085,7 +1088,16 @@ static int mpic_host_xlate(struct irq_domain *h, struct device_node *ct,
*/
switch (intspec[2]) {
case 0:
- case 1: /* no EISR/EIMR support for now, treat as shared IRQ */
+ break;
+ case 1:
+ if (!mpic->err_int_config_done)
+ break;
+
+ if (intspec[3] >= ARRAY_SIZE(mpic->err_int_vecs))
+ return -EINVAL;
+
+ *out_hwirq = mpic->err_int_vecs[intspec[3]];
+
break;
case 2:
if (intspec[0] >= ARRAY_SIZE(mpic->ipi_vecs))
@@ -1302,6 +1314,9 @@ struct mpic * __init mpic_alloc(struct device_node *node,
mpic_map(mpic, mpic->paddr, &mpic->tmregs, MPIC_INFO(TIMER_BASE), 0x1000);
if (mpic->flags & MPIC_FSL) {
+ u32 brr1, version;
+ int ret;
+
/*
* Yes, Freescale really did put global registers in the
* magic per-cpu area -- and they don't even show up in the
@@ -1309,6 +1324,29 @@ struct mpic * __init mpic_alloc(struct device_node *node,
*/
mpic_map(mpic, mpic->paddr, &mpic->thiscpuregs,
MPIC_CPU_THISBASE, 0x1000);
+
+ brr1 = _mpic_read(mpic->reg_type, &mpic->thiscpuregs,
+ MPIC_FSL_BRR1);
+ version = brr1 & MPIC_FSL_BRR1_VER;
+
+ /* Error interrupt mask register (EIMR) is required for
+ * handling individual device error interrupts. EIMR
+ * was added in MPIC version 4.1.
+ *
+ * Over here we reserve vector number space for error
+ * interrupt vectors. This space is stolen from the
+ * global vector number space, as in case of ipis
+ * and timer interrupts.
+ *
+ * Available vector space = intvec_top - 12, where 12
+ * is the number of vectors which have been consumed by
+ * ipis and timer interrupts.
+ */
+ if (version >= 0x401) {
+ ret = mpic_setup_error_int(mpic, intvec_top - 12);
+ if (ret)
+ return NULL;
+ }
}
/* Reset */
@@ -1474,6 +1512,11 @@ void __init mpic_init(struct mpic *mpic)
num_timers = 8;
}
+ /* FSL mpic error interrupt intialization */
+ if (mpic->flags & MPIC_FSL_HAS_EIMR)
+ mpic->err_int_config_done =
+ mpic_err_int_init(mpic, MPIC_FSL_ERR_INT);
+
/* Initialize timers to our reserved vectors and mask them for now */
for (i = 0; i < num_timers; i++) {
unsigned int offset = mpic_tm_offset(mpic, i);
diff --git a/arch/powerpc/sysdev/mpic.h b/arch/powerpc/sysdev/mpic.h
index 13f3e89..e05d825 100644
--- a/arch/powerpc/sysdev/mpic.h
+++ b/arch/powerpc/sysdev/mpic.h
@@ -40,4 +40,26 @@ extern int mpic_set_affinity(struct irq_data *d,
const struct cpumask *cpumask, bool force);
extern void mpic_reset_core(int cpu);
+#ifdef CONFIG_FSL_SOC
+extern int mpic_map_error_int(struct mpic *mpic, unsigned int virq, irq_hw_number_t hw);
+extern int mpic_err_int_init(struct mpic *mpic, irq_hw_number_t irqnum);
+extern int mpic_setup_error_int(struct mpic *mpic, int intvec);
+#else
+static inline int mpic_map_error_int(struct mpic *mpic, unsigned int virq, irq_hw_number_t hw)
+{
+ return 0;
+}
+
+
+static inline int mpic_err_int_init(struct mpic *mpic, irq_hw_number_t irqnum)
+{
+ return -1;
+}
+
+static inline int mpic_setup_error_int(struct mpic *mpic, int intvec)
+{
+ return -1;
+}
+#endif
+
#endif /* _POWERPC_SYSDEV_MPIC_H */
--
1.7.4.1
^ permalink raw reply related
* [PATCH 0/6] powerpc/booke64: add core support for KVM
From: Mihai Caraman @ 2012-08-06 13:27 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Mihai Caraman, kvm, kvm-ppc
These patches add core support for KVM on 64-bit Book3E PowerPC, mainly
integrating the DO_KVM macro. KVM arch specific patches will follow shortly.
Mihai Caraman (6):
powerpc/booke64: fix machine check handler to use the right prolog
powerpc/booke64: use GSRR registers in Guest Doorbell interrupts
powerpc/booke64: add DO_KVM kernel hooks
powerpc/booke64: remove mfspr srr1 duplicate in exception prolog
powerpc/booke64: use SPRG0/3 scratch for bolted TLB miss & crit int
powerpc/booke64: restore VDSO information on critical exception
arch/powerpc/include/asm/exception-64e.h | 6 +-
arch/powerpc/include/asm/paca.h | 3 +
arch/powerpc/include/asm/reg.h | 6 +-
arch/powerpc/kernel/asm-offsets.c | 1 +
arch/powerpc/kernel/exceptions-64e.S | 194 +++++++++++++++++++-----------
arch/powerpc/kernel/vdso.c | 2 +
arch/powerpc/mm/tlb_low_64e.S | 18 ++-
7 files changed, 151 insertions(+), 79 deletions(-)
--
1.7.4.1
^ permalink raw reply
* [PATCH 4/6] powerpc/booke64: remove mfspr srr1 duplicate in exception prolog
From: Mihai Caraman @ 2012-08-06 13:27 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Mihai Caraman, kvm, kvm-ppc
In-Reply-To: <1344259628-31161-1-git-send-email-mihai.caraman@freescale.com>
Refactor exception prolog to get rid of mfspr srr1 duplicate. This was
introduced by KVM integration, with DO_KVM macro logic expecting srr1 value
earlier in r11.
Reserve r11 to hold srr1's value also required at the end of the prolog and
free up r10 to serve as spare in addition macros.
For syscalls case this change does not add any performance penalty. For irq
soft-disabled case the change adds a store/load of conditional register value
to/from a paca slot. Paca slots fit in one 64-byte cache line so these
additional operations have little impact on performance.
Signed-off-by: Mihai Caraman <mihai.caraman@freescale.com>
---
arch/powerpc/kernel/exceptions-64e.S | 67 ++++++++++++++++------------------
1 files changed, 32 insertions(+), 35 deletions(-)
diff --git a/arch/powerpc/kernel/exceptions-64e.S b/arch/powerpc/kernel/exceptions-64e.S
index 3b1ad1b..83c20e8 100644
--- a/arch/powerpc/kernel/exceptions-64e.S
+++ b/arch/powerpc/kernel/exceptions-64e.S
@@ -45,10 +45,9 @@
mfcr r10; /* save CR */ \
mfspr r11,SPRN_##type##_SRR1;/* what are we coming from */ \
DO_KVM intnum,SPRN_##type##_SRR1; /* KVM hook */ \
+ stw r10,PACA_EX##type+EX_CR(r13); /* save old CR in the PACA */ \
addition; /* additional code for that exc. */ \
std r1,PACA_EX##type+EX_R1(r13); /* save old r1 in the PACA */ \
- stw r10,PACA_EX##type+EX_CR(r13); /* save old CR in the PACA */ \
- mfspr r11,SPRN_##type##_SRR1;/* what are we coming from */ \
type##_SET_KSTACK; /* get special stack if necessary */\
andi. r10,r11,MSR_PR; /* save stack pointer */ \
beq 1f; /* branch around if supervisor */ \
@@ -109,8 +108,8 @@
#define PROLOG_ADDITION_NONE_MC(n)
#define PROLOG_ADDITION_MASKABLE_GEN(n) \
- lbz r11,PACASOFTIRQEN(r13); /* are irqs soft-disabled ? */ \
- cmpwi cr0,r11,0; /* yes -> go out of line */ \
+ lbz r10,PACASOFTIRQEN(r13); /* are irqs soft-disabled ? */ \
+ cmpwi cr0,r10,0; /* yes -> go out of line */ \
beq masked_interrupt_book3e_##n
#define PROLOG_ADDITION_2REGS_GEN(n) \
@@ -624,44 +623,42 @@ kernel_dbg_exc:
* accordingly and if the interrupt is level sensitive, we hard disable
*/
+.macro masked_interrupt_book3e paca_irq full_mask
+ lbz r10,PACAIRQHAPPENED(r13)
+ ori r10,r10,\paca_irq
+ stb r10,PACAIRQHAPPENED(r13)
+
+ .if \full_mask == 1
+ rldicl r10,r11,48,1 /* clear MSR_EE */
+ rotldi r11,r10,16
+ mtspr SPRN_SRR1,r11
+ .endif
+
+ lwz r11,PACA_EXGEN+EX_CR(r13)
+ mtcr r11
+ ld r10,PACA_EXGEN+EX_R10(r13)
+ ld r11,PACA_EXGEN+EX_R11(r13)
+ mfspr r13,SPRN_SPRG_GEN_SCRATCH
+ rfi
+ b .
+.endm
+
masked_interrupt_book3e_0x500:
- /* XXX When adding support for EPR, use PACA_IRQ_EE_EDGE */
- li r11,PACA_IRQ_EE
- b masked_interrupt_book3e_full_mask
+ // XXX When adding support for EPR, use PACA_IRQ_EE_EDGE
+ masked_interrupt_book3e PACA_IRQ_EE 1
masked_interrupt_book3e_0x900:
- ACK_DEC(r11);
- li r11,PACA_IRQ_DEC
- b masked_interrupt_book3e_no_mask
+ ACK_DEC(r10);
+ masked_interrupt_book3e PACA_IRQ_DEC 0
+
masked_interrupt_book3e_0x980:
- ACK_FIT(r11);
- li r11,PACA_IRQ_DEC
- b masked_interrupt_book3e_no_mask
+ ACK_FIT(r10);
+ masked_interrupt_book3e PACA_IRQ_DEC 0
+
masked_interrupt_book3e_0x280:
masked_interrupt_book3e_0x2c0:
- li r11,PACA_IRQ_DBELL
- b masked_interrupt_book3e_no_mask
+ masked_interrupt_book3e PACA_IRQ_DBELL 0
-masked_interrupt_book3e_no_mask:
- mtcr r10
- lbz r10,PACAIRQHAPPENED(r13)
- or r10,r10,r11
- stb r10,PACAIRQHAPPENED(r13)
- b 1f
-masked_interrupt_book3e_full_mask:
- mtcr r10
- lbz r10,PACAIRQHAPPENED(r13)
- or r10,r10,r11
- stb r10,PACAIRQHAPPENED(r13)
- mfspr r10,SPRN_SRR1
- rldicl r11,r10,48,1 /* clear MSR_EE */
- rotldi r10,r11,16
- mtspr SPRN_SRR1,r10
-1: ld r10,PACA_EXGEN+EX_R10(r13);
- ld r11,PACA_EXGEN+EX_R11(r13);
- mfspr r13,SPRN_SPRG_GEN_SCRATCH;
- rfi
- b .
/*
* Called from arch_local_irq_enable when an interrupt needs
* to be resent. r3 contains either 0x500,0x900,0x260 or 0x280
--
1.7.4.1
^ permalink raw reply related
* [PATCH 6/6] powerpc/booke64: restore VDSO information on critical exception
From: Mihai Caraman @ 2012-08-06 13:27 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Mihai Caraman, kvm, kvm-ppc
In-Reply-To: <1344259628-31161-1-git-send-email-mihai.caraman@freescale.com>
Critical exception handler on 64-bit booke uses user-visible SPRG3 as scratch.
Restore VDSO information in SPRG3 on exception prolog.
Signed-off-by: Mihai Caraman <mihai.caraman@freescale.com>
---
arch/powerpc/include/asm/paca.h | 3 +++
arch/powerpc/kernel/asm-offsets.c | 1 +
arch/powerpc/kernel/exceptions-64e.S | 4 +++-
arch/powerpc/kernel/vdso.c | 2 ++
4 files changed, 9 insertions(+), 1 deletions(-)
diff --git a/arch/powerpc/include/asm/paca.h b/arch/powerpc/include/asm/paca.h
index daf813f..6b98ad4 100644
--- a/arch/powerpc/include/asm/paca.h
+++ b/arch/powerpc/include/asm/paca.h
@@ -118,6 +118,9 @@ struct paca_struct {
void *mc_kstack;
void *crit_kstack;
void *dbg_kstack;
+
+ /* User-visible sprg information */
+ ulong sprg3;
#endif /* CONFIG_PPC_BOOK3E */
mm_context_t context;
diff --git a/arch/powerpc/kernel/asm-offsets.c b/arch/powerpc/kernel/asm-offsets.c
index 85b05c4..8ca338d 100644
--- a/arch/powerpc/kernel/asm-offsets.c
+++ b/arch/powerpc/kernel/asm-offsets.c
@@ -168,6 +168,7 @@ int main(void)
DEFINE(PACA_MC_STACK, offsetof(struct paca_struct, mc_kstack));
DEFINE(PACA_CRIT_STACK, offsetof(struct paca_struct, crit_kstack));
DEFINE(PACA_DBG_STACK, offsetof(struct paca_struct, dbg_kstack));
+ DEFINE(PACA_SPRG3, offsetof(struct paca_struct, sprg3));
#endif /* CONFIG_PPC_BOOK3E */
#ifdef CONFIG_PPC_STD_MMU_64
diff --git a/arch/powerpc/kernel/exceptions-64e.S b/arch/powerpc/kernel/exceptions-64e.S
index 7476b0a..87a82fb 100644
--- a/arch/powerpc/kernel/exceptions-64e.S
+++ b/arch/powerpc/kernel/exceptions-64e.S
@@ -110,7 +110,9 @@
#define PROLOG_STORE_RESTORE_SCRATCH_CRIT \
mfspr r10,SPRN_SPRG_CRIT_SCRATCH; /* get r13 */ \
- std r10,PACA_EXCRIT+EX_R13(r13)
+ std r10,PACA_EXCRIT+EX_R13(r13); \
+ ld r11,PACA_SPRG3(r13); \
+ mtspr SPRN_SPRG_CRIT_SCRATCH,r11;
/* Variants of the "addition" argument for the prolog
*/
diff --git a/arch/powerpc/kernel/vdso.c b/arch/powerpc/kernel/vdso.c
index b67db22..a0b0d08 100644
--- a/arch/powerpc/kernel/vdso.c
+++ b/arch/powerpc/kernel/vdso.c
@@ -725,6 +725,8 @@ int __cpuinit vdso_getcpu_init(void)
mtspr(SPRN_SPRG3, val);
#ifdef CONFIG_KVM_BOOK3S_HANDLER
get_paca()->kvm_hstate.sprg3 = val;
+#elif CONFIG_PPC_BOOK3E
+ get_paca()->sprg3 = val;
#endif
put_cpu();
--
1.7.4.1
^ permalink raw reply related
* [PATCH 1/6] powerpc/booke64: fix machine check handler to use the right prolog
From: Mihai Caraman @ 2012-08-06 13:27 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Mihai Caraman, kvm, kvm-ppc
In-Reply-To: <1344259628-31161-1-git-send-email-mihai.caraman@freescale.com>
Machine check exception handler was using a wrong prolog. Hypervisors like
KVM which are called early from the exception handler rely on the interrupt
source.
Signed-off-by: Mihai Caraman <mihai.caraman@freescale.com>
---
arch/powerpc/kernel/exceptions-64e.S | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/arch/powerpc/kernel/exceptions-64e.S b/arch/powerpc/kernel/exceptions-64e.S
index 98be7f0..0243b1b 100644
--- a/arch/powerpc/kernel/exceptions-64e.S
+++ b/arch/powerpc/kernel/exceptions-64e.S
@@ -297,7 +297,7 @@ interrupt_end_book3e:
/* Machine Check Interrupt */
START_EXCEPTION(machine_check);
- CRIT_EXCEPTION_PROLOG(0x200, PROLOG_ADDITION_NONE)
+ MC_EXCEPTION_PROLOG(0x200, PROLOG_ADDITION_NONE)
// EXCEPTION_COMMON(0x200, PACA_EXMC, INTS_DISABLE)
// bl special_reg_save_mc
// addi r3,r1,STACK_FRAME_OVERHEAD
--
1.7.4.1
^ permalink raw reply related
* [PATCH 5/6] powerpc/booke64: use SPRG0/3 scratch for bolted TLB miss & crit int
From: Mihai Caraman @ 2012-08-06 13:27 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Mihai Caraman, kvm, kvm-ppc
In-Reply-To: <1344259628-31161-1-git-send-email-mihai.caraman@freescale.com>
Embedded.Hypervisor category defines GSPRG0..3 physical registers for guests.
Avoid SPRG4-7 usage as scratch in host exception handlers, otherwise guest
SPRG4-7 registers will be clobbered.
For bolted TLB miss exception handlers, which is the version currently
supported by KVM, use SPRN_SPRG_GEN_SCRATCH aka SPRG0 instead of
SPRN_SPRG_TLB_SCRATCH aka SPRG6. Keep using TLB PACA slots to fit in one
64-byte cache line.
For critical exception handlers use SPRG3 instead of SPRG7. Provide a routine
to store and restore user-visible SPRGs. This will be subsequently used
to restore VDSO information in SPRG3. Add EX_R13 to paca slots to free up
SPRG3 and change the critical exception epilog to use it.
Signed-off-by: Mihai Caraman <mihai.caraman@freescale.com>
---
arch/powerpc/include/asm/exception-64e.h | 5 +++--
arch/powerpc/include/asm/reg.h | 5 +++--
arch/powerpc/kernel/exceptions-64e.S | 17 +++++++++++++++--
arch/powerpc/mm/tlb_low_64e.S | 4 ++--
4 files changed, 23 insertions(+), 8 deletions(-)
diff --git a/arch/powerpc/include/asm/exception-64e.h b/arch/powerpc/include/asm/exception-64e.h
index e73452f..51fa43e 100644
--- a/arch/powerpc/include/asm/exception-64e.h
+++ b/arch/powerpc/include/asm/exception-64e.h
@@ -46,8 +46,9 @@
#define EX_CR (1 * 8)
#define EX_R10 (2 * 8)
#define EX_R11 (3 * 8)
-#define EX_R14 (4 * 8)
-#define EX_R15 (5 * 8)
+#define EX_R13 (4 * 8)
+#define EX_R14 (5 * 8)
+#define EX_R15 (6 * 8)
/*
* The TLB miss exception uses different slots.
diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
index 349d9c3..63a6159 100644
--- a/arch/powerpc/include/asm/reg.h
+++ b/arch/powerpc/include/asm/reg.h
@@ -761,7 +761,8 @@
* 64-bit embedded
* - SPRG0 generic exception scratch
* - SPRG2 TLB exception stack
- * - SPRG3 CPU and NUMA node for VDSO getcpu (user visible)
+ * - SPRG3 critical exception scratch and
+ * CPU and NUMA node for VDSO getcpu (user visible)
* - SPRG4 unused (user visible)
* - SPRG6 TLB miss scratch (user visible, sorry !)
* - SPRG7 critical exception scratch
@@ -858,7 +859,7 @@
#ifdef CONFIG_PPC_BOOK3E_64
#define SPRN_SPRG_MC_SCRATCH SPRN_SPRG8
-#define SPRN_SPRG_CRIT_SCRATCH SPRN_SPRG7
+#define SPRN_SPRG_CRIT_SCRATCH SPRN_SPRG3
#define SPRN_SPRG_DBG_SCRATCH SPRN_SPRG9
#define SPRN_SPRG_TLB_EXFRAME SPRN_SPRG2
#define SPRN_SPRG_TLB_SCRATCH SPRN_SPRG6
diff --git a/arch/powerpc/kernel/exceptions-64e.S b/arch/powerpc/kernel/exceptions-64e.S
index 83c20e8..7476b0a 100644
--- a/arch/powerpc/kernel/exceptions-64e.S
+++ b/arch/powerpc/kernel/exceptions-64e.S
@@ -42,6 +42,7 @@
mfspr r13,SPRN_SPRG_PACA; /* get PACA */ \
std r10,PACA_EX##type+EX_R10(r13); \
std r11,PACA_EX##type+EX_R11(r13); \
+ PROLOG_STORE_RESTORE_SCRATCH_##type; \
mfcr r10; /* save CR */ \
mfspr r11,SPRN_##type##_SRR1;/* what are we coming from */ \
DO_KVM intnum,SPRN_##type##_SRR1; /* KVM hook */ \
@@ -99,6 +100,18 @@
#define GDBELL_EXCEPTION_PROLOG(n, intnum, addition) \
EXCEPTION_PROLOG(n, intnum, GDBELL, addition##_GDBELL(n))
+/*
+ * Store user-visible scratch in PACA exception slots and restore proper value
+ */
+#define PROLOG_STORE_RESTORE_SCRATCH_GEN
+#define PROLOG_STORE_RESTORE_SCRATCH_GDBELL
+#define PROLOG_STORE_RESTORE_SCRATCH_DBG
+#define PROLOG_STORE_RESTORE_SCRATCH_MC
+
+#define PROLOG_STORE_RESTORE_SCRATCH_CRIT \
+ mfspr r10,SPRN_SPRG_CRIT_SCRATCH; /* get r13 */ \
+ std r10,PACA_EXCRIT+EX_R13(r13)
+
/* Variants of the "addition" argument for the prolog
*/
#define PROLOG_ADDITION_NONE_GEN(n)
@@ -454,7 +467,7 @@ interrupt_end_book3e:
mtcr r10
ld r10,PACA_EXCRIT+EX_R10(r13) /* restore registers */
ld r11,PACA_EXCRIT+EX_R11(r13)
- mfspr r13,SPRN_SPRG_CRIT_SCRATCH
+ ld r13,PACA_EXCRIT+EX_R13(r13)
rfci
/* Normal debug exception */
@@ -467,7 +480,7 @@ interrupt_end_book3e:
/* Now we mash up things to make it look like we are coming on a
* normal exception
*/
- mfspr r15,SPRN_SPRG_CRIT_SCRATCH
+ ld r15,PACA_EXCRIT+EX_R13(r13)
mtspr SPRN_SPRG_GEN_SCRATCH,r15
mfspr r14,SPRN_DBSR
EXCEPTION_COMMON(0xd00, PACA_EXCRIT, INTS_DISABLE)
diff --git a/arch/powerpc/mm/tlb_low_64e.S b/arch/powerpc/mm/tlb_low_64e.S
index d884fa4..b4113bf 100644
--- a/arch/powerpc/mm/tlb_low_64e.S
+++ b/arch/powerpc/mm/tlb_low_64e.S
@@ -40,7 +40,7 @@
**********************************************************************/
.macro tlb_prolog_bolted intnum addr
- mtspr SPRN_SPRG_TLB_SCRATCH,r13
+ mtspr SPRN_SPRG_GEN_SCRATCH,r13
mfspr r13,SPRN_SPRG_PACA
std r10,PACA_EXTLB+EX_TLB_R10(r13)
mfcr r10
@@ -69,7 +69,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_EMB_HV)
ld r15,PACA_EXTLB+EX_TLB_R15(r13)
TLB_MISS_RESTORE_STATS_BOLTED
ld r16,PACA_EXTLB+EX_TLB_R16(r13)
- mfspr r13,SPRN_SPRG_TLB_SCRATCH
+ mfspr r13,SPRN_SPRG_GEN_SCRATCH
.endm
/* Data TLB miss */
--
1.7.4.1
^ permalink raw reply related
* [PATCH 2/6] powerpc/booke64: use GSRR registers in Guest Doorbell interrupts
From: Mihai Caraman @ 2012-08-06 13:27 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Mihai Caraman, kvm, kvm-ppc
In-Reply-To: <1344259628-31161-1-git-send-email-mihai.caraman@freescale.com>
Guest Doorbell interrupts use guest save and restore registers. Add a new
Guest Doorbell exception type to accommodate GSRR0/1 SPRs usage in exception
prolog and fix the exception handler.
Signed-off-by: Mihai Caraman <mihai.caraman@freescale.com>
---
arch/powerpc/include/asm/exception-64e.h | 1 +
arch/powerpc/include/asm/reg.h | 1 +
arch/powerpc/kernel/exceptions-64e.S | 21 +++++++++++++++++++--
3 files changed, 21 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/include/asm/exception-64e.h b/arch/powerpc/include/asm/exception-64e.h
index ac13add..e73452f 100644
--- a/arch/powerpc/include/asm/exception-64e.h
+++ b/arch/powerpc/include/asm/exception-64e.h
@@ -37,6 +37,7 @@
* critical data
*/
+#define PACA_EXGDBELL PACA_EXGEN
/* We are out of SPRGs so we save some things in the PACA. The normal
* exception frame is smaller than the CRIT or MC one though
diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
index 6386086..349d9c3 100644
--- a/arch/powerpc/include/asm/reg.h
+++ b/arch/powerpc/include/asm/reg.h
@@ -863,6 +863,7 @@
#define SPRN_SPRG_TLB_EXFRAME SPRN_SPRG2
#define SPRN_SPRG_TLB_SCRATCH SPRN_SPRG6
#define SPRN_SPRG_GEN_SCRATCH SPRN_SPRG0
+#define SPRN_SPRG_GDBELL_SCRATCH SPRN_SPRG_GEN_SCRATCH
#define SET_PACA(rX) mtspr SPRN_SPRG_PACA,rX
#define GET_PACA(rX) mfspr rX,SPRN_SPRG_PACA
diff --git a/arch/powerpc/kernel/exceptions-64e.S b/arch/powerpc/kernel/exceptions-64e.S
index 0243b1b..0f58a95 100644
--- a/arch/powerpc/kernel/exceptions-64e.S
+++ b/arch/powerpc/kernel/exceptions-64e.S
@@ -59,6 +59,10 @@
#define SPRN_GEN_SRR0 SPRN_SRR0
#define SPRN_GEN_SRR1 SPRN_SRR1
+#define GDBELL_SET_KSTACK GEN_SET_KSTACK
+#define SPRN_GDBELL_SRR0 SPRN_GSRR0
+#define SPRN_GDBELL_SRR1 SPRN_GSRR1
+
#define CRIT_SET_KSTACK \
ld r1,PACA_CRIT_STACK(r13); \
subi r1,r1,SPECIAL_EXC_FRAME_SIZE;
@@ -89,10 +93,13 @@
#define MC_EXCEPTION_PROLOG(n, addition) \
EXCEPTION_PROLOG(n, MC, addition##_MC(n))
+#define GDBELL_EXCEPTION_PROLOG(n, addition) \
+ EXCEPTION_PROLOG(n, GDBELL, addition##_GDBELL(n))
/* Variants of the "addition" argument for the prolog
*/
#define PROLOG_ADDITION_NONE_GEN(n)
+#define PROLOG_ADDITION_NONE_GDBELL(n)
#define PROLOG_ADDITION_NONE_CRIT(n)
#define PROLOG_ADDITION_NONE_DBG(n)
#define PROLOG_ADDITION_NONE_MC(n)
@@ -543,8 +550,18 @@ kernel_dbg_exc:
// b ret_from_crit_except
b .
-/* Guest Doorbell */
- MASKABLE_EXCEPTION(0x2c0, guest_doorbell, .unknown_exception, ACK_NONE)
+/*
+ * Guest doorbell interrupt
+ * This general exception use GSRRx save/restore registers
+ */
+ START_EXCEPTION(guest_doorbell);
+ GDBELL_EXCEPTION_PROLOG(0x2c0, PROLOG_ADDITION_NONE)
+ EXCEPTION_COMMON(0x2c0, PACA_EXGEN, INTS_KEEP)
+ addi r3,r1,STACK_FRAME_OVERHEAD
+ bl .save_nvgprs
+ INTS_RESTORE_HARD
+ bl .unknown_exception
+ b .ret_from_except
/* Guest Doorbell critical Interrupt */
START_EXCEPTION(guest_doorbell_crit);
--
1.7.4.1
^ permalink raw reply related
* [PATCH 3/6] powerpc/booke64: add DO_KVM kernel hooks
From: Mihai Caraman @ 2012-08-06 13:27 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Mihai Caraman, kvm, kvm-ppc
In-Reply-To: <1344259628-31161-1-git-send-email-mihai.caraman@freescale.com>
Hook DO_KVM macro into 64-bit booke for KVM integration. Extend interrupt
handlers' parameter list with interrupt vector numbers to accomodate the macro.
Only the bolted version of tlb miss handers is addressed now.
Signed-off-by: Mihai Caraman <mihai.caraman@freescale.com>
---
arch/powerpc/kernel/exceptions-64e.S | 93 +++++++++++++++++++++------------
arch/powerpc/mm/tlb_low_64e.S | 14 ++++-
2 files changed, 70 insertions(+), 37 deletions(-)
diff --git a/arch/powerpc/kernel/exceptions-64e.S b/arch/powerpc/kernel/exceptions-64e.S
index 0f58a95..3b1ad1b 100644
--- a/arch/powerpc/kernel/exceptions-64e.S
+++ b/arch/powerpc/kernel/exceptions-64e.S
@@ -25,6 +25,8 @@
#include <asm/ppc-opcode.h>
#include <asm/mmu.h>
#include <asm/hw_irq.h>
+#include <asm/kvm_asm.h>
+#include <asm/kvm_booke_hv_asm.h>
/* XXX This will ultimately add space for a special exception save
* structure used to save things like SRR0/SRR1, SPRGs, MAS, etc...
@@ -35,12 +37,14 @@
#define SPECIAL_EXC_FRAME_SIZE INT_FRAME_SIZE
/* Exception prolog code for all exceptions */
-#define EXCEPTION_PROLOG(n, type, addition) \
+#define EXCEPTION_PROLOG(n, intnum, type, addition) \
mtspr SPRN_SPRG_##type##_SCRATCH,r13; /* get spare registers */ \
mfspr r13,SPRN_SPRG_PACA; /* get PACA */ \
std r10,PACA_EX##type+EX_R10(r13); \
std r11,PACA_EX##type+EX_R11(r13); \
mfcr r10; /* save CR */ \
+ mfspr r11,SPRN_##type##_SRR1;/* what are we coming from */ \
+ DO_KVM intnum,SPRN_##type##_SRR1; /* KVM hook */ \
addition; /* additional code for that exc. */ \
std r1,PACA_EX##type+EX_R1(r13); /* save old r1 in the PACA */ \
stw r10,PACA_EX##type+EX_CR(r13); /* save old CR in the PACA */ \
@@ -81,20 +85,20 @@
#define SPRN_MC_SRR0 SPRN_MCSRR0
#define SPRN_MC_SRR1 SPRN_MCSRR1
-#define NORMAL_EXCEPTION_PROLOG(n, addition) \
- EXCEPTION_PROLOG(n, GEN, addition##_GEN(n))
+#define NORMAL_EXCEPTION_PROLOG(n, intnum, addition) \
+ EXCEPTION_PROLOG(n, intnum, GEN, addition##_GEN(n))
-#define CRIT_EXCEPTION_PROLOG(n, addition) \
- EXCEPTION_PROLOG(n, CRIT, addition##_CRIT(n))
+#define CRIT_EXCEPTION_PROLOG(n, intnum, addition) \
+ EXCEPTION_PROLOG(n, intnum, CRIT, addition##_CRIT(n))
-#define DBG_EXCEPTION_PROLOG(n, addition) \
- EXCEPTION_PROLOG(n, DBG, addition##_DBG(n))
+#define DBG_EXCEPTION_PROLOG(n, intnum, addition) \
+ EXCEPTION_PROLOG(n, intnum, DBG, addition##_DBG(n))
-#define MC_EXCEPTION_PROLOG(n, addition) \
- EXCEPTION_PROLOG(n, MC, addition##_MC(n))
+#define MC_EXCEPTION_PROLOG(n, intnum, addition) \
+ EXCEPTION_PROLOG(n, intnum, MC, addition##_MC(n))
-#define GDBELL_EXCEPTION_PROLOG(n, addition) \
- EXCEPTION_PROLOG(n, GDBELL, addition##_GDBELL(n))
+#define GDBELL_EXCEPTION_PROLOG(n, intnum, addition) \
+ EXCEPTION_PROLOG(n, intnum, GDBELL, addition##_GDBELL(n))
/* Variants of the "addition" argument for the prolog
*/
@@ -240,9 +244,9 @@ exc_##n##_bad_stack: \
1:
-#define MASKABLE_EXCEPTION(trapnum, label, hdlr, ack) \
+#define MASKABLE_EXCEPTION(trapnum, intnum, label, hdlr, ack) \
START_EXCEPTION(label); \
- NORMAL_EXCEPTION_PROLOG(trapnum, PROLOG_ADDITION_MASKABLE) \
+ NORMAL_EXCEPTION_PROLOG(trapnum, intnum, PROLOG_ADDITION_MASKABLE)\
EXCEPTION_COMMON(trapnum, PACA_EXGEN, INTS_DISABLE) \
ack(r8); \
CHECK_NAPPING(); \
@@ -293,7 +297,8 @@ interrupt_end_book3e:
/* Critical Input Interrupt */
START_EXCEPTION(critical_input);
- CRIT_EXCEPTION_PROLOG(0x100, PROLOG_ADDITION_NONE)
+ CRIT_EXCEPTION_PROLOG(0x100, BOOKE_INTERRUPT_CRITICAL,
+ PROLOG_ADDITION_NONE)
// EXCEPTION_COMMON(0x100, PACA_EXCRIT, INTS_DISABLE)
// bl special_reg_save_crit
// CHECK_NAPPING();
@@ -304,7 +309,8 @@ interrupt_end_book3e:
/* Machine Check Interrupt */
START_EXCEPTION(machine_check);
- MC_EXCEPTION_PROLOG(0x200, PROLOG_ADDITION_NONE)
+ MC_EXCEPTION_PROLOG(0x200, BOOKE_INTERRUPT_MACHINE_CHECK,
+ PROLOG_ADDITION_NONE)
// EXCEPTION_COMMON(0x200, PACA_EXMC, INTS_DISABLE)
// bl special_reg_save_mc
// addi r3,r1,STACK_FRAME_OVERHEAD
@@ -315,7 +321,8 @@ interrupt_end_book3e:
/* Data Storage Interrupt */
START_EXCEPTION(data_storage)
- NORMAL_EXCEPTION_PROLOG(0x300, PROLOG_ADDITION_2REGS)
+ NORMAL_EXCEPTION_PROLOG(0x300, BOOKE_INTERRUPT_DATA_STORAGE,
+ PROLOG_ADDITION_2REGS)
mfspr r14,SPRN_DEAR
mfspr r15,SPRN_ESR
EXCEPTION_COMMON(0x300, PACA_EXGEN, INTS_DISABLE)
@@ -323,18 +330,21 @@ interrupt_end_book3e:
/* Instruction Storage Interrupt */
START_EXCEPTION(instruction_storage);
- NORMAL_EXCEPTION_PROLOG(0x400, PROLOG_ADDITION_2REGS)
+ NORMAL_EXCEPTION_PROLOG(0x400, BOOKE_INTERRUPT_INST_STORAGE,
+ PROLOG_ADDITION_2REGS)
li r15,0
mr r14,r10
EXCEPTION_COMMON(0x400, PACA_EXGEN, INTS_DISABLE)
b storage_fault_common
/* External Input Interrupt */
- MASKABLE_EXCEPTION(0x500, external_input, .do_IRQ, ACK_NONE)
+ MASKABLE_EXCEPTION(0x500, BOOKE_INTERRUPT_EXTERNAL,
+ external_input, .do_IRQ, ACK_NONE)
/* Alignment */
START_EXCEPTION(alignment);
- NORMAL_EXCEPTION_PROLOG(0x600, PROLOG_ADDITION_2REGS)
+ NORMAL_EXCEPTION_PROLOG(0x600, BOOKE_INTERRUPT_ALIGNMENT,
+ PROLOG_ADDITION_2REGS)
mfspr r14,SPRN_DEAR
mfspr r15,SPRN_ESR
EXCEPTION_COMMON(0x600, PACA_EXGEN, INTS_KEEP)
@@ -342,7 +352,8 @@ interrupt_end_book3e:
/* Program Interrupt */
START_EXCEPTION(program);
- NORMAL_EXCEPTION_PROLOG(0x700, PROLOG_ADDITION_1REG)
+ NORMAL_EXCEPTION_PROLOG(0x700, BOOKE_INTERRUPT_PROGRAM,
+ PROLOG_ADDITION_1REG)
mfspr r14,SPRN_ESR
EXCEPTION_COMMON(0x700, PACA_EXGEN, INTS_DISABLE)
std r14,_DSISR(r1)
@@ -354,7 +365,8 @@ interrupt_end_book3e:
/* Floating Point Unavailable Interrupt */
START_EXCEPTION(fp_unavailable);
- NORMAL_EXCEPTION_PROLOG(0x800, PROLOG_ADDITION_NONE)
+ NORMAL_EXCEPTION_PROLOG(0x800, BOOKE_INTERRUPT_FP_UNAVAIL,
+ PROLOG_ADDITION_NONE)
/* we can probably do a shorter exception entry for that one... */
EXCEPTION_COMMON(0x800, PACA_EXGEN, INTS_KEEP)
ld r12,_MSR(r1)
@@ -369,14 +381,17 @@ interrupt_end_book3e:
b .ret_from_except
/* Decrementer Interrupt */
- MASKABLE_EXCEPTION(0x900, decrementer, .timer_interrupt, ACK_DEC)
+ MASKABLE_EXCEPTION(0x900, BOOKE_INTERRUPT_DECREMENTER,
+ decrementer, .timer_interrupt, ACK_DEC)
/* Fixed Interval Timer Interrupt */
- MASKABLE_EXCEPTION(0x980, fixed_interval, .unknown_exception, ACK_FIT)
+ MASKABLE_EXCEPTION(0x980, BOOKE_INTERRUPT_FIT,
+ fixed_interval, .unknown_exception, ACK_FIT)
/* Watchdog Timer Interrupt */
START_EXCEPTION(watchdog);
- CRIT_EXCEPTION_PROLOG(0x9f0, PROLOG_ADDITION_NONE)
+ CRIT_EXCEPTION_PROLOG(0x9f0, BOOKE_INTERRUPT_WATCHDOG,
+ PROLOG_ADDITION_NONE)
// EXCEPTION_COMMON(0x9f0, PACA_EXCRIT, INTS_DISABLE)
// bl special_reg_save_crit
// CHECK_NAPPING();
@@ -395,7 +410,8 @@ interrupt_end_book3e:
/* Auxiliary Processor Unavailable Interrupt */
START_EXCEPTION(ap_unavailable);
- NORMAL_EXCEPTION_PROLOG(0xf20, PROLOG_ADDITION_NONE)
+ NORMAL_EXCEPTION_PROLOG(0xf20, BOOKE_INTERRUPT_AP_UNAVAIL,
+ PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0xf20, PACA_EXGEN, INTS_DISABLE)
bl .save_nvgprs
addi r3,r1,STACK_FRAME_OVERHEAD
@@ -404,7 +420,8 @@ interrupt_end_book3e:
/* Debug exception as a critical interrupt*/
START_EXCEPTION(debug_crit);
- CRIT_EXCEPTION_PROLOG(0xd00, PROLOG_ADDITION_2REGS)
+ CRIT_EXCEPTION_PROLOG(0xd00, BOOKE_INTERRUPT_DEBUG,
+ PROLOG_ADDITION_2REGS)
/*
* If there is a single step or branch-taken exception in an
@@ -469,7 +486,8 @@ kernel_dbg_exc:
/* Debug exception as a debug interrupt*/
START_EXCEPTION(debug_debug);
- DBG_EXCEPTION_PROLOG(0xd08, PROLOG_ADDITION_2REGS)
+ DBG_EXCEPTION_PROLOG(0xd00, BOOKE_INTERRUPT_DEBUG,
+ PROLOG_ADDITION_2REGS)
/*
* If there is a single step or branch-taken exception in an
@@ -530,18 +548,21 @@ kernel_dbg_exc:
b .ret_from_except
START_EXCEPTION(perfmon);
- NORMAL_EXCEPTION_PROLOG(0x260, PROLOG_ADDITION_NONE)
+ NORMAL_EXCEPTION_PROLOG(0x260, BOOKE_INTERRUPT_PERFORMANCE_MONITOR,
+ PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0x260, PACA_EXGEN, INTS_DISABLE)
addi r3,r1,STACK_FRAME_OVERHEAD
bl .performance_monitor_exception
b .ret_from_except_lite
/* Doorbell interrupt */
- MASKABLE_EXCEPTION(0x280, doorbell, .doorbell_exception, ACK_NONE)
+ MASKABLE_EXCEPTION(0x280, BOOKE_INTERRUPT_DOORBELL,
+ doorbell, .doorbell_exception, ACK_NONE)
/* Doorbell critical Interrupt */
START_EXCEPTION(doorbell_crit);
- CRIT_EXCEPTION_PROLOG(0x2a0, PROLOG_ADDITION_NONE)
+ CRIT_EXCEPTION_PROLOG(0x2a0, BOOKE_INTERRUPT_DOORBELL_CRITICAL,
+ PROLOG_ADDITION_NONE)
// EXCEPTION_COMMON(0x2a0, PACA_EXCRIT, INTS_DISABLE)
// bl special_reg_save_crit
// CHECK_NAPPING();
@@ -555,7 +576,8 @@ kernel_dbg_exc:
* This general exception use GSRRx save/restore registers
*/
START_EXCEPTION(guest_doorbell);
- GDBELL_EXCEPTION_PROLOG(0x2c0, PROLOG_ADDITION_NONE)
+ GDBELL_EXCEPTION_PROLOG(0x2c0, BOOKE_INTERRUPT_GUEST_DBELL,
+ PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0x2c0, PACA_EXGEN, INTS_KEEP)
addi r3,r1,STACK_FRAME_OVERHEAD
bl .save_nvgprs
@@ -565,7 +587,8 @@ kernel_dbg_exc:
/* Guest Doorbell critical Interrupt */
START_EXCEPTION(guest_doorbell_crit);
- CRIT_EXCEPTION_PROLOG(0x2e0, PROLOG_ADDITION_NONE)
+ CRIT_EXCEPTION_PROLOG(0x2e0, BOOKE_INTERRUPT_GUEST_DBELL_CRIT,
+ PROLOG_ADDITION_NONE)
// EXCEPTION_COMMON(0x2e0, PACA_EXCRIT, INTS_DISABLE)
// bl special_reg_save_crit
// CHECK_NAPPING();
@@ -576,7 +599,8 @@ kernel_dbg_exc:
/* Hypervisor call */
START_EXCEPTION(hypercall);
- NORMAL_EXCEPTION_PROLOG(0x310, PROLOG_ADDITION_NONE)
+ NORMAL_EXCEPTION_PROLOG(0x310, BOOKE_INTERRUPT_HV_SYSCALL,
+ PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0x310, PACA_EXGEN, INTS_KEEP)
addi r3,r1,STACK_FRAME_OVERHEAD
bl .save_nvgprs
@@ -586,7 +610,8 @@ kernel_dbg_exc:
/* Embedded Hypervisor priviledged */
START_EXCEPTION(ehpriv);
- NORMAL_EXCEPTION_PROLOG(0x320, PROLOG_ADDITION_NONE)
+ NORMAL_EXCEPTION_PROLOG(0x320, BOOKE_INTERRUPT_HV_PRIV,
+ PROLOG_ADDITION_NONE)
EXCEPTION_COMMON(0x320, PACA_EXGEN, INTS_KEEP)
addi r3,r1,STACK_FRAME_OVERHEAD
bl .save_nvgprs
diff --git a/arch/powerpc/mm/tlb_low_64e.S b/arch/powerpc/mm/tlb_low_64e.S
index f09d48e..d884fa4 100644
--- a/arch/powerpc/mm/tlb_low_64e.S
+++ b/arch/powerpc/mm/tlb_low_64e.S
@@ -20,6 +20,8 @@
#include <asm/pgtable.h>
#include <asm/exception-64e.h>
#include <asm/ppc-opcode.h>
+#include <asm/kvm_asm.h>
+#include <asm/kvm_booke_hv_asm.h>
#ifdef CONFIG_PPC_64K_PAGES
#define VPTE_PMD_SHIFT (PTE_INDEX_SIZE+1)
@@ -37,12 +39,18 @@
* *
**********************************************************************/
-.macro tlb_prolog_bolted addr
+.macro tlb_prolog_bolted intnum addr
mtspr SPRN_SPRG_TLB_SCRATCH,r13
mfspr r13,SPRN_SPRG_PACA
std r10,PACA_EXTLB+EX_TLB_R10(r13)
mfcr r10
std r11,PACA_EXTLB+EX_TLB_R11(r13)
+#ifdef CONFIG_KVM_BOOKE_HV
+BEGIN_FTR_SECTION
+ mfspr r11, SPRN_SRR1
+END_FTR_SECTION_IFSET(CPU_FTR_EMB_HV)
+#endif
+ DO_KVM \intnum, SPRN_SRR1
std r16,PACA_EXTLB+EX_TLB_R16(r13)
mfspr r16,\addr /* get faulting address */
std r14,PACA_EXTLB+EX_TLB_R14(r13)
@@ -66,7 +74,7 @@
/* Data TLB miss */
START_EXCEPTION(data_tlb_miss_bolted)
- tlb_prolog_bolted SPRN_DEAR
+ tlb_prolog_bolted BOOKE_INTERRUPT_DTLB_MISS SPRN_DEAR
/* We need _PAGE_PRESENT and _PAGE_ACCESSED set */
@@ -214,7 +222,7 @@ itlb_miss_fault_bolted:
/* Instruction TLB miss */
START_EXCEPTION(instruction_tlb_miss_bolted)
- tlb_prolog_bolted SPRN_SRR0
+ tlb_prolog_bolted BOOKE_INTERRUPT_ITLB_MISS SPRN_SRR0
rldicl. r10,r16,64-PGTABLE_EADDR_SIZE,PGTABLE_EADDR_SIZE+4
srdi r15,r16,60 /* get region */
--
1.7.4.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox