* PS3: Fix build with 32-bit toolchains
From: Geoff Levand @ 2007-07-20 15:31 UTC (permalink / raw)
To: akpm; +Cc: linuxppc-dev, Paul Mackerras, Arnd Bergmann
In-Reply-To: <469FDBDC.5030700@am.sony.com>
The PS3 bootwrapper files use instructions only available on
64-bit CPUs. Add the code generation directive '.machine "ppc64"'
for toolchains configured for 32-bit CPUs.
Signed-off-by: Geoff Levand <geoffrey.levand@am.sony.com>
Acked-by: Grant Likely <grant.likely@secretlab.ca>
---
Hi Andrew,
It seems Paul is out on holiday, so I'm sending this to you.
This fixes a build error in Linus' current tree for a few
32 bit powerpc platforms. It had some review on the
linuxppc-dev ML, and Grant Likely reported it fixed the
problem.
-Geoff
arch/powerpc/boot/ps3-head.S | 2 ++
arch/powerpc/boot/ps3-hvcall.S | 2 ++
2 files changed, 4 insertions(+)
--- a/arch/powerpc/boot/ps3-head.S
+++ b/arch/powerpc/boot/ps3-head.S
@@ -20,6 +20,8 @@
#include "ppc_asm.h"
+ .machine "ppc64"
+
.text
/*
--- a/arch/powerpc/boot/ps3-hvcall.S
+++ b/arch/powerpc/boot/ps3-hvcall.S
@@ -20,6 +20,8 @@
#include "ppc_asm.h"
+ .machine "ppc64"
+
/*
* The PS3 hypervisor uses a 64 bit "C" language calling convention.
* The routines here marshal arguments between the 32 bit wrapper
^ permalink raw reply
* Re: Interrupts on xilinx ml403
From: Grant Likely @ 2007-07-20 15:12 UTC (permalink / raw)
To: Mirek23; +Cc: linuxppc-embedded
In-Reply-To: <11708607.post@talk.nabble.com>
On 7/20/07, Mirek23 <miroslaw.dach@psi.ch> wrote:
> Does it mean that DIP Switches do not use INTC interrupt controller?
> How to handle the DIP Switches interrupt?
> Does the Interrupt handler routine have to acknowledge the interrupt from
> Dip Switches?
It probably means that the GPIO irq out line is not hooked up to your
INTC in the system.mhs file.
g.
--
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.
grant.likely@secretlab.ca
(403) 399-0195
^ permalink raw reply
* Re: [PATCH] powerpc: mpc5200 low power mode, .globl fix
From: Grant Likely @ 2007-07-20 15:09 UTC (permalink / raw)
To: Domen Puncer; +Cc: Linux PPC Mailing List, Paul Mackerras
In-Reply-To: <20070720105550.GA1385@moe.telargo.com>
On 7/20/07, Domen Puncer <domen.puncer@telargo.com> wrote:
> flush_data_cache doesn't need to be global, and can cause problems.
> Thanks to Milton Miller for noticing this.
>
>
> Signed-off-by: Domen Puncer <domen.puncer@telargo.com>
Signed-off-by: Grant Likely <grant.likely@secretlab.ca>
Paulus, can you pick this up please?
> ---
> arch/powerpc/platforms/52xx/lite5200_sleep.S | 1 -
> 1 file changed, 1 deletion(-)
>
> Index: work-powerpc.git/arch/powerpc/platforms/52xx/lite5200_sleep.S
> ===================================================================
> --- work-powerpc.git.orig/arch/powerpc/platforms/52xx/lite5200_sleep.S
> +++ work-powerpc.git/arch/powerpc/platforms/52xx/lite5200_sleep.S
> @@ -400,7 +400,6 @@ restore_regs:
> * Flush data cache
> * Do this by just reading lots of stuff into the cache.
> */
> - .globl flush_data_cache
> flush_data_cache:
> lis r3,CONFIG_KERNEL_START@h
> ori r3,r3,CONFIG_KERNEL_START@l
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-dev
>
--
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.
grant.likely@secretlab.ca
(403) 399-0195
^ permalink raw reply
* [PATCH 5/5] ehca: Support small QP queues
From: Hoang-Nam Nguyen @ 2007-07-20 14:04 UTC (permalink / raw)
To: Roland Dreier, linux-kernel, linuxppc-dev, openib-general
Cc: fenkes, raisch, stefan.roscher
From: Stefan Roscher <stefan.roscher at de.ibm.com>
Date: Fri, 20 Jul 2007 13:59:14 +0200
Subject: [PATCH 5/5] IB/ehca: Small QP queues
eHCA2 supports QP queues that can be as small as 512 bytes. This greatly
reduces memory overhead for consumers that use lots of QPs with small queues
(e.g. RDMA-only QPs). Apart from dealing with firmware, this code needs to
manage bite-sized chunks of kernel pages, making sure that no kernel page is
shared between different protection domains.
Signed-off-by: Hoang-Nam Nguyen <hnguyen@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_classes.h | 41 ++++--
drivers/infiniband/hw/ehca/ehca_cq.c | 8 +-
drivers/infiniband/hw/ehca/ehca_eq.c | 8 +-
drivers/infiniband/hw/ehca/ehca_main.c | 14 ++-
drivers/infiniband/hw/ehca/ehca_pd.c | 25 +++-
drivers/infiniband/hw/ehca/ehca_qp.c | 163 +++++++++++++---------
drivers/infiniband/hw/ehca/ehca_uverbs.c | 2 +-
drivers/infiniband/hw/ehca/hcp_if.c | 30 +++--
drivers/infiniband/hw/ehca/ipz_pt_fn.c | 222 ++++++++++++++++++++++-------
drivers/infiniband/hw/ehca/ipz_pt_fn.h | 26 +++-
10 files changed, 379 insertions(+), 160 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h
index 63b8b9f..3725aa8 100644
--- a/drivers/infiniband/hw/ehca/ehca_classes.h
+++ b/drivers/infiniband/hw/ehca/ehca_classes.h
@@ -43,7 +43,6 @@
#ifndef __EHCA_CLASSES_H__
#define __EHCA_CLASSES_H__
-
struct ehca_module;
struct ehca_qp;
struct ehca_cq;
@@ -129,6 +128,10 @@ struct ehca_pd {
struct ib_pd ib_pd;
struct ipz_pd fw_pd;
u32 ownpid;
+ /* small queue mgmt */
+ struct mutex lock;
+ struct list_head free[2];
+ struct list_head full[2];
};
enum ehca_ext_qp_type {
@@ -307,6 +310,8 @@ int ehca_init_av_cache(void);
void ehca_cleanup_av_cache(void);
int ehca_init_mrmw_cache(void);
void ehca_cleanup_mrmw_cache(void);
+int ehca_init_small_qp_cache(void);
+void ehca_cleanup_small_qp_cache(void);
extern rwlock_t ehca_qp_idr_lock;
extern rwlock_t ehca_cq_idr_lock;
@@ -324,7 +329,7 @@ struct ipzu_queue_resp {
u32 queue_length; /* queue length allocated in bytes */
u32 pagesize;
u32 toggle_state;
- u32 dummy; /* padding for 8 byte alignment */
+ u32 offset; /* save offset within a page for small_qp */
};
struct ehca_create_cq_resp {
@@ -366,15 +371,29 @@ enum ehca_ll_comp_flags {
LLQP_COMP_MASK = 0x60,
};
+struct ehca_alloc_queue_parms {
+ /* input parameters */
+ int max_wr;
+ int max_sge;
+ int page_size;
+ int is_small;
+
+ /* output parameters */
+ u16 act_nr_wqes;
+ u8 act_nr_sges;
+ u32 queue_size; /* bytes for small queues, pages otherwise */
+};
+
struct ehca_alloc_qp_parms {
-/* input parameters */
+ struct ehca_alloc_queue_parms squeue;
+ struct ehca_alloc_queue_parms rqueue;
+
+ /* input parameters */
enum ehca_service_type servicetype;
+ int qp_storage;
int sigtype;
enum ehca_ext_qp_type ext_type;
enum ehca_ll_comp_flags ll_comp_flags;
-
- int max_send_wr, max_recv_wr;
- int max_send_sge, max_recv_sge;
int ud_av_l_key_ctl;
u32 token;
@@ -384,18 +403,10 @@ struct ehca_alloc_qp_parms {
u32 srq_qpn, srq_token, srq_limit;
-/* output parameters */
+ /* output parameters */
u32 real_qp_num;
struct ipz_qp_handle qp_handle;
struct h_galpas galpas;
-
- u16 act_nr_send_wqes;
- u16 act_nr_recv_wqes;
- u8 act_nr_recv_sges;
- u8 act_nr_send_sges;
-
- u32 nr_rq_pages;
- u32 nr_sq_pages;
};
int ehca_cq_assign_qp(struct ehca_cq *cq, struct ehca_qp *qp);
diff --git a/drivers/infiniband/hw/ehca/ehca_cq.c b/drivers/infiniband/hw/ehca/ehca_cq.c
index 9e87883..5746787 100644
--- a/drivers/infiniband/hw/ehca/ehca_cq.c
+++ b/drivers/infiniband/hw/ehca/ehca_cq.c
@@ -190,8 +190,8 @@ struct ib_cq *ehca_create_cq(struct ib_device *device, int cqe, int comp_vector,
goto create_cq_exit2;
}
- ipz_rc = ipz_queue_ctor(&my_cq->ipz_queue, param.act_pages,
- EHCA_PAGESIZE, sizeof(struct ehca_cqe), 0);
+ ipz_rc = ipz_queue_ctor(NULL, &my_cq->ipz_queue, param.act_pages,
+ EHCA_PAGESIZE, sizeof(struct ehca_cqe), 0, 0);
if (!ipz_rc) {
ehca_err(device, "ipz_queue_ctor() failed ipz_rc=%x device=%p",
ipz_rc, device);
@@ -285,7 +285,7 @@ struct ib_cq *ehca_create_cq(struct ib_device *device, int cqe, int comp_vector,
return cq;
create_cq_exit4:
- ipz_queue_dtor(&my_cq->ipz_queue);
+ ipz_queue_dtor(NULL, &my_cq->ipz_queue);
create_cq_exit3:
h_ret = hipz_h_destroy_cq(adapter_handle, my_cq, 1);
@@ -359,7 +359,7 @@ int ehca_destroy_cq(struct ib_cq *cq)
"ehca_cq=%p cq_num=%x", h_ret, my_cq, cq_num);
return ehca2ib_return_code(h_ret);
}
- ipz_queue_dtor(&my_cq->ipz_queue);
+ ipz_queue_dtor(NULL, &my_cq->ipz_queue);
kmem_cache_free(cq_cache, my_cq);
return 0;
diff --git a/drivers/infiniband/hw/ehca/ehca_eq.c b/drivers/infiniband/hw/ehca/ehca_eq.c
index 4825975..1d41faa 100644
--- a/drivers/infiniband/hw/ehca/ehca_eq.c
+++ b/drivers/infiniband/hw/ehca/ehca_eq.c
@@ -86,8 +86,8 @@ int ehca_create_eq(struct ehca_shca *shca,
return -EINVAL;
}
- ret = ipz_queue_ctor(&eq->ipz_queue, nr_pages,
- EHCA_PAGESIZE, sizeof(struct ehca_eqe), 0);
+ ret = ipz_queue_ctor(NULL, &eq->ipz_queue, nr_pages,
+ EHCA_PAGESIZE, sizeof(struct ehca_eqe), 0, 0);
if (!ret) {
ehca_err(ib_dev, "Can't allocate EQ pages eq=%p", eq);
goto create_eq_exit1;
@@ -145,7 +145,7 @@ int ehca_create_eq(struct ehca_shca *shca,
return 0;
create_eq_exit2:
- ipz_queue_dtor(&eq->ipz_queue);
+ ipz_queue_dtor(NULL, &eq->ipz_queue);
create_eq_exit1:
hipz_h_destroy_eq(shca->ipz_hca_handle, eq);
@@ -181,7 +181,7 @@ int ehca_destroy_eq(struct ehca_shca *shca, struct ehca_eq *eq)
ehca_err(&shca->ib_device, "Can't free EQ resources.");
return -EINVAL;
}
- ipz_queue_dtor(&eq->ipz_queue);
+ ipz_queue_dtor(NULL, &eq->ipz_queue);
return 0;
}
diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c
index 3bd7afb..e09a2ae 100644
--- a/drivers/infiniband/hw/ehca/ehca_main.c
+++ b/drivers/infiniband/hw/ehca/ehca_main.c
@@ -181,6 +181,12 @@ static int ehca_create_slab_caches(void)
goto create_slab_caches5;
}
+ ret = ehca_init_small_qp_cache();
+ if (ret) {
+ ehca_gen_err("Cannot create small queue SLAB cache.");
+ goto create_slab_caches6;
+ }
+
#ifdef CONFIG_PPC_64K_PAGES
ctblk_cache = kmem_cache_create("ehca_cache_ctblk",
EHCA_PAGESIZE, H_CB_ALIGNMENT,
@@ -188,12 +194,15 @@ static int ehca_create_slab_caches(void)
NULL, NULL);
if (!ctblk_cache) {
ehca_gen_err("Cannot create ctblk SLAB cache.");
- ehca_cleanup_mrmw_cache();
- goto create_slab_caches5;
+ ehca_cleanup_small_qp_cache();
+ goto create_slab_caches6;
}
#endif
return 0;
+create_slab_caches6:
+ ehca_cleanup_mrmw_cache();
+
create_slab_caches5:
ehca_cleanup_av_cache();
@@ -211,6 +220,7 @@ create_slab_caches2:
static void ehca_destroy_slab_caches(void)
{
+ ehca_cleanup_small_qp_cache();
ehca_cleanup_mrmw_cache();
ehca_cleanup_av_cache();
ehca_cleanup_qp_cache();
diff --git a/drivers/infiniband/hw/ehca/ehca_pd.c b/drivers/infiniband/hw/ehca/ehca_pd.c
index 79d0591..79d5bc8 100644
--- a/drivers/infiniband/hw/ehca/ehca_pd.c
+++ b/drivers/infiniband/hw/ehca/ehca_pd.c
@@ -49,6 +49,7 @@ struct ib_pd *ehca_alloc_pd(struct ib_device *device,
struct ib_ucontext *context, struct ib_udata *udata)
{
struct ehca_pd *pd;
+ int i;
pd = kmem_cache_zalloc(pd_cache, GFP_KERNEL);
if (!pd) {
@@ -58,6 +59,11 @@ struct ib_pd *ehca_alloc_pd(struct ib_device *device,
}
pd->ownpid = current->tgid;
+ for (i = 0; i < 2; i++) {
+ INIT_LIST_HEAD(&pd->free[i]);
+ INIT_LIST_HEAD(&pd->full[i]);
+ }
+ mutex_init(&pd->lock);
/*
* Kernel PD: when device = -1, 0
@@ -81,6 +87,9 @@ int ehca_dealloc_pd(struct ib_pd *pd)
{
u32 cur_pid = current->tgid;
struct ehca_pd *my_pd = container_of(pd, struct ehca_pd, ib_pd);
+ int i, leftovers = 0;
+ extern struct kmem_cache *small_qp_cache;
+ struct ipz_small_queue_page *page, *tmp;
if (my_pd->ib_pd.uobject && my_pd->ib_pd.uobject->context &&
my_pd->ownpid != cur_pid) {
@@ -89,8 +98,20 @@ int ehca_dealloc_pd(struct ib_pd *pd)
return -EINVAL;
}
- kmem_cache_free(pd_cache,
- container_of(pd, struct ehca_pd, ib_pd));
+ for (i = 0; i < 2; i++) {
+ list_splice(&my_pd->full[i], &my_pd->free[i]);
+ list_for_each_entry_safe(page, tmp, &my_pd->free[i], list) {
+ leftovers = 1;
+ free_page(page->page);
+ kmem_cache_free(small_qp_cache, page);
+ }
+ }
+
+ if (leftovers)
+ ehca_warn(pd->device,
+ "Some small queue pages were not freed");
+
+ kmem_cache_free(pd_cache, my_pd);
return 0;
}
diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c
index b916d9c..6c6f9d9 100644
--- a/drivers/infiniband/hw/ehca/ehca_qp.c
+++ b/drivers/infiniband/hw/ehca/ehca_qp.c
@@ -275,34 +275,39 @@ static inline void queue2resp(struct ipzu_queue_resp *resp,
resp->toggle_state = queue->toggle_state;
}
-static inline int ll_qp_msg_size(int nr_sge)
-{
- return 128 << nr_sge;
-}
-
/*
* init_qp_queue initializes/constructs r/squeue and registers queue pages.
*/
static inline int init_qp_queue(struct ehca_shca *shca,
+ struct ehca_pd *pd,
struct ehca_qp *my_qp,
struct ipz_queue *queue,
int q_type,
u64 expected_hret,
- int nr_q_pages,
- int wqe_size,
- int nr_sges)
+ struct ehca_alloc_queue_parms *parms,
+ int wqe_size)
{
- int ret, cnt, ipz_rc;
+ int ret, cnt, ipz_rc, nr_q_pages;
void *vpage;
u64 rpage, h_ret;
struct ib_device *ib_dev = &shca->ib_device;
struct ipz_adapter_handle ipz_hca_handle = shca->ipz_hca_handle;
- if (!nr_q_pages)
+ if (!parms->queue_size)
return 0;
- ipz_rc = ipz_queue_ctor(queue, nr_q_pages, EHCA_PAGESIZE,
- wqe_size, nr_sges);
+ if (parms->is_small) {
+ nr_q_pages = 1;
+ ipz_rc = ipz_queue_ctor(pd, queue, nr_q_pages,
+ 128 << parms->page_size,
+ wqe_size, parms->act_nr_sges, 1);
+ } else {
+ nr_q_pages = parms->queue_size;
+ ipz_rc = ipz_queue_ctor(pd, queue, nr_q_pages,
+ EHCA_PAGESIZE, wqe_size,
+ parms->act_nr_sges, 0);
+ }
+
if (!ipz_rc) {
ehca_err(ib_dev, "Cannot allocate page for queue. ipz_rc=%x",
ipz_rc);
@@ -323,7 +328,7 @@ static inline int init_qp_queue(struct ehca_shca *shca,
h_ret = hipz_h_register_rpage_qp(ipz_hca_handle,
my_qp->ipz_qp_handle,
NULL, 0, q_type,
- rpage, 1,
+ rpage, parms->is_small ? 0 : 1,
my_qp->galpas.kernel);
if (cnt == (nr_q_pages - 1)) { /* last page! */
if (h_ret != expected_hret) {
@@ -354,10 +359,45 @@ static inline int init_qp_queue(struct ehca_shca *shca,
return 0;
init_qp_queue1:
- ipz_queue_dtor(queue);
+ ipz_queue_dtor(pd, queue);
return ret;
}
+static inline int ehca_calc_wqe_size(int act_nr_sge, int is_llqp)
+{
+ if (is_llqp)
+ return 128 << act_nr_sge;
+ else
+ return offsetof(struct ehca_wqe,
+ u.nud.sg_list[act_nr_sge]);
+}
+
+static void ehca_determine_small_queue(struct ehca_alloc_queue_parms *queue,
+ int req_nr_sge, int is_llqp)
+{
+ u32 wqe_size, q_size;
+ int act_nr_sge = req_nr_sge;
+
+ if (!is_llqp)
+ /* round up #SGEs so WQE size is a power of 2 */
+ for (act_nr_sge = 4; act_nr_sge <= 252;
+ act_nr_sge = 4 + 2 * act_nr_sge)
+ if (act_nr_sge >= req_nr_sge)
+ break;
+
+ wqe_size = ehca_calc_wqe_size(act_nr_sge, is_llqp);
+ q_size = wqe_size * (queue->max_wr + 1);
+
+ if (q_size <= 512)
+ queue->page_size = 2;
+ else if (q_size <= 1024)
+ queue->page_size = 3;
+ else
+ queue->page_size = 0;
+
+ queue->is_small = (queue->page_size != 0);
+}
+
/*
* Create an ib_qp struct that is either a QP or an SRQ, depending on
* the value of the is_srq parameter. If init_attr and srq_init_attr share
@@ -553,10 +593,20 @@ static struct ehca_qp *internal_create_qp(
if (my_qp->recv_cq)
parms.recv_cq_handle = my_qp->recv_cq->ipz_cq_handle;
- parms.max_send_wr = init_attr->cap.max_send_wr;
- parms.max_recv_wr = init_attr->cap.max_recv_wr;
- parms.max_send_sge = max_send_sge;
- parms.max_recv_sge = max_recv_sge;
+ parms.squeue.max_wr = init_attr->cap.max_send_wr;
+ parms.rqueue.max_wr = init_attr->cap.max_recv_wr;
+ parms.squeue.max_sge = max_send_sge;
+ parms.rqueue.max_sge = max_recv_sge;
+
+ if (EHCA_BMASK_GET(HCA_CAP_MINI_QP, shca->hca_cap)
+ && !(context && udata)) { /* no small QP support in userspace ATM */
+ ehca_determine_small_queue(
+ &parms.squeue, max_send_sge, is_llqp);
+ ehca_determine_small_queue(
+ &parms.rqueue, max_recv_sge, is_llqp);
+ parms.qp_storage =
+ (parms.squeue.is_small || parms.rqueue.is_small);
+ }
h_ret = hipz_h_alloc_resource_qp(shca->ipz_hca_handle, &parms);
if (h_ret != H_SUCCESS) {
@@ -570,50 +620,33 @@ static struct ehca_qp *internal_create_qp(
my_qp->ipz_qp_handle = parms.qp_handle;
my_qp->galpas = parms.galpas;
+ swqe_size = ehca_calc_wqe_size(parms.squeue.act_nr_sges, is_llqp);
+ rwqe_size = ehca_calc_wqe_size(parms.rqueue.act_nr_sges, is_llqp);
+
switch (qp_type) {
case IB_QPT_RC:
- if (!is_llqp) {
- swqe_size = offsetof(struct ehca_wqe, u.nud.sg_list[
- (parms.act_nr_send_sges)]);
- rwqe_size = offsetof(struct ehca_wqe, u.nud.sg_list[
- (parms.act_nr_recv_sges)]);
- } else { /* for LLQP we need to use msg size, not wqe size */
- swqe_size = ll_qp_msg_size(max_send_sge);
- rwqe_size = ll_qp_msg_size(max_recv_sge);
- parms.act_nr_send_sges = 1;
- parms.act_nr_recv_sges = 1;
- }
- break;
- case IB_QPT_UC:
- swqe_size = offsetof(struct ehca_wqe,
- u.nud.sg_list[parms.act_nr_send_sges]);
- rwqe_size = offsetof(struct ehca_wqe,
- u.nud.sg_list[parms.act_nr_recv_sges]);
+ if (is_llqp) {
+ parms.squeue.act_nr_sges = 1;
+ parms.rqueue.act_nr_sges = 1;
+ }
break;
-
case IB_QPT_UD:
case IB_QPT_GSI:
case IB_QPT_SMI:
+ /* UD circumvention */
if (is_llqp) {
- swqe_size = ll_qp_msg_size(parms.act_nr_send_sges);
- rwqe_size = ll_qp_msg_size(parms.act_nr_recv_sges);
- parms.act_nr_send_sges = 1;
- parms.act_nr_recv_sges = 1;
+ parms.squeue.act_nr_sges = 1;
+ parms.rqueue.act_nr_sges = 1;
} else {
- /* UD circumvention */
- parms.act_nr_send_sges -= 2;
- parms.act_nr_recv_sges -= 2;
- swqe_size = offsetof(struct ehca_wqe, u.ud_av.sg_list[
- parms.act_nr_send_sges]);
- rwqe_size = offsetof(struct ehca_wqe, u.ud_av.sg_list[
- parms.act_nr_recv_sges]);
+ parms.squeue.act_nr_sges -= 2;
+ parms.rqueue.act_nr_sges -= 2;
}
if (IB_QPT_GSI == qp_type || IB_QPT_SMI == qp_type) {
- parms.act_nr_send_wqes = init_attr->cap.max_send_wr;
- parms.act_nr_recv_wqes = init_attr->cap.max_recv_wr;
- parms.act_nr_send_sges = init_attr->cap.max_send_sge;
- parms.act_nr_recv_sges = init_attr->cap.max_recv_sge;
+ parms.squeue.act_nr_wqes = init_attr->cap.max_send_wr;
+ parms.rqueue.act_nr_wqes = init_attr->cap.max_recv_wr;
+ parms.squeue.act_nr_sges = init_attr->cap.max_send_sge;
+ parms.rqueue.act_nr_sges = init_attr->cap.max_recv_sge;
ib_qp_num = (qp_type == IB_QPT_SMI) ? 0 : 1;
}
@@ -626,10 +659,9 @@ static struct ehca_qp *internal_create_qp(
/* initialize r/squeue and register queue pages */
if (HAS_SQ(my_qp)) {
ret = init_qp_queue(
- shca, my_qp, &my_qp->ipz_squeue, 0,
+ shca, my_pd, my_qp, &my_qp->ipz_squeue, 0,
HAS_RQ(my_qp) ? H_PAGE_REGISTERED : H_SUCCESS,
- parms.nr_sq_pages, swqe_size,
- parms.act_nr_send_sges);
+ &parms.squeue, swqe_size);
if (ret) {
ehca_err(pd->device, "Couldn't initialize squeue "
"and pages ret=%x", ret);
@@ -639,9 +671,8 @@ static struct ehca_qp *internal_create_qp(
if (HAS_RQ(my_qp)) {
ret = init_qp_queue(
- shca, my_qp, &my_qp->ipz_rqueue, 1,
- H_SUCCESS, parms.nr_rq_pages, rwqe_size,
- parms.act_nr_recv_sges);
+ shca, my_pd, my_qp, &my_qp->ipz_rqueue, 1,
+ H_SUCCESS, &parms.rqueue, rwqe_size);
if (ret) {
ehca_err(pd->device, "Couldn't initialize rqueue "
"and pages ret=%x", ret);
@@ -671,10 +702,10 @@ static struct ehca_qp *internal_create_qp(
}
init_attr->cap.max_inline_data = 0; /* not supported yet */
- init_attr->cap.max_recv_sge = parms.act_nr_recv_sges;
- init_attr->cap.max_recv_wr = parms.act_nr_recv_wqes;
- init_attr->cap.max_send_sge = parms.act_nr_send_sges;
- init_attr->cap.max_send_wr = parms.act_nr_send_wqes;
+ init_attr->cap.max_recv_sge = parms.rqueue.act_nr_sges;
+ init_attr->cap.max_recv_wr = parms.rqueue.act_nr_wqes;
+ init_attr->cap.max_send_sge = parms.squeue.act_nr_sges;
+ init_attr->cap.max_send_wr = parms.squeue.act_nr_wqes;
my_qp->init_attr = *init_attr;
/* NOTE: define_apq0() not supported yet */
@@ -708,6 +739,8 @@ static struct ehca_qp *internal_create_qp(
resp.ext_type = my_qp->ext_type;
resp.qkey = my_qp->qkey;
resp.real_qp_num = my_qp->real_qp_num;
+ resp.ipz_rqueue.offset = my_qp->ipz_rqueue.offset;
+ resp.ipz_squeue.offset = my_qp->ipz_squeue.offset;
if (HAS_SQ(my_qp))
queue2resp(&resp.ipz_squeue, &my_qp->ipz_squeue);
if (HAS_RQ(my_qp))
@@ -724,11 +757,11 @@ static struct ehca_qp *internal_create_qp(
create_qp_exit4:
if (HAS_RQ(my_qp))
- ipz_queue_dtor(&my_qp->ipz_rqueue);
+ ipz_queue_dtor(my_pd, &my_qp->ipz_rqueue);
create_qp_exit3:
if (HAS_SQ(my_qp))
- ipz_queue_dtor(&my_qp->ipz_squeue);
+ ipz_queue_dtor(my_pd, &my_qp->ipz_squeue);
create_qp_exit2:
hipz_h_destroy_qp(shca->ipz_hca_handle, my_qp);
@@ -1735,9 +1768,9 @@ static int internal_destroy_qp(struct ib_device *dev, struct ehca_qp *my_qp,
}
if (HAS_RQ(my_qp))
- ipz_queue_dtor(&my_qp->ipz_rqueue);
+ ipz_queue_dtor(my_pd, &my_qp->ipz_rqueue);
if (HAS_SQ(my_qp))
- ipz_queue_dtor(&my_qp->ipz_squeue);
+ ipz_queue_dtor(my_pd, &my_qp->ipz_squeue);
kmem_cache_free(qp_cache, my_qp);
return 0;
}
diff --git a/drivers/infiniband/hw/ehca/ehca_uverbs.c b/drivers/infiniband/hw/ehca/ehca_uverbs.c
index 05c4157..4bc687f 100644
--- a/drivers/infiniband/hw/ehca/ehca_uverbs.c
+++ b/drivers/infiniband/hw/ehca/ehca_uverbs.c
@@ -149,7 +149,7 @@ static int ehca_mmap_queue(struct vm_area_struct *vma, struct ipz_queue *queue,
ehca_gen_err("vm_insert_page() failed rc=%x", ret);
return ret;
}
- start += PAGE_SIZE;
+ start += PAGE_SIZE;
}
vma->vm_private_data = mm_count;
(*mm_count)++;
diff --git a/drivers/infiniband/hw/ehca/hcp_if.c b/drivers/infiniband/hw/ehca/hcp_if.c
index 358796c..fdbfebe 100644
--- a/drivers/infiniband/hw/ehca/hcp_if.c
+++ b/drivers/infiniband/hw/ehca/hcp_if.c
@@ -52,10 +52,13 @@
#define H_ALL_RES_QP_ENHANCED_OPS EHCA_BMASK_IBM(9, 11)
#define H_ALL_RES_QP_PTE_PIN EHCA_BMASK_IBM(12, 12)
#define H_ALL_RES_QP_SERVICE_TYPE EHCA_BMASK_IBM(13, 15)
+#define H_ALL_RES_QP_STORAGE EHCA_BMASK_IBM(16, 17)
#define H_ALL_RES_QP_LL_RQ_CQE_POSTING EHCA_BMASK_IBM(18, 18)
#define H_ALL_RES_QP_LL_SQ_CQE_POSTING EHCA_BMASK_IBM(19, 21)
#define H_ALL_RES_QP_SIGNALING_TYPE EHCA_BMASK_IBM(22, 23)
#define H_ALL_RES_QP_UD_AV_LKEY_CTRL EHCA_BMASK_IBM(31, 31)
+#define H_ALL_RES_QP_SMALL_SQ_PAGE_SIZE EHCA_BMASK_IBM(32, 35)
+#define H_ALL_RES_QP_SMALL_RQ_PAGE_SIZE EHCA_BMASK_IBM(36, 39)
#define H_ALL_RES_QP_RESOURCE_TYPE EHCA_BMASK_IBM(56, 63)
#define H_ALL_RES_QP_MAX_OUTST_SEND_WR EHCA_BMASK_IBM(0, 15)
@@ -299,6 +302,11 @@ u64 hipz_h_alloc_resource_qp(const struct ipz_adapter_handle adapter_handle,
| EHCA_BMASK_SET(H_ALL_RES_QP_PTE_PIN, 0)
| EHCA_BMASK_SET(H_ALL_RES_QP_SERVICE_TYPE, parms->servicetype)
| EHCA_BMASK_SET(H_ALL_RES_QP_SIGNALING_TYPE, parms->sigtype)
+ | EHCA_BMASK_SET(H_ALL_RES_QP_STORAGE, parms->qp_storage)
+ | EHCA_BMASK_SET(H_ALL_RES_QP_SMALL_SQ_PAGE_SIZE,
+ parms->squeue.page_size)
+ | EHCA_BMASK_SET(H_ALL_RES_QP_SMALL_RQ_PAGE_SIZE,
+ parms->rqueue.page_size)
| EHCA_BMASK_SET(H_ALL_RES_QP_LL_RQ_CQE_POSTING,
!!(parms->ll_comp_flags & LLQP_RECV_COMP))
| EHCA_BMASK_SET(H_ALL_RES_QP_LL_SQ_CQE_POSTING,
@@ -309,13 +317,13 @@ u64 hipz_h_alloc_resource_qp(const struct ipz_adapter_handle adapter_handle,
max_r10_reg =
EHCA_BMASK_SET(H_ALL_RES_QP_MAX_OUTST_SEND_WR,
- parms->max_send_wr + 1)
+ parms->squeue.max_wr + 1)
| EHCA_BMASK_SET(H_ALL_RES_QP_MAX_OUTST_RECV_WR,
- parms->max_recv_wr + 1)
+ parms->rqueue.max_wr + 1)
| EHCA_BMASK_SET(H_ALL_RES_QP_MAX_SEND_SGE,
- parms->max_send_sge)
+ parms->squeue.max_sge)
| EHCA_BMASK_SET(H_ALL_RES_QP_MAX_RECV_SGE,
- parms->max_recv_sge);
+ parms->rqueue.max_sge);
r11 = EHCA_BMASK_SET(H_ALL_RES_QP_SRQ_QP_TOKEN, parms->srq_token);
@@ -335,17 +343,17 @@ u64 hipz_h_alloc_resource_qp(const struct ipz_adapter_handle adapter_handle,
parms->qp_handle.handle = outs[0];
parms->real_qp_num = (u32)outs[1];
- parms->act_nr_send_wqes =
+ parms->squeue.act_nr_wqes =
(u16)EHCA_BMASK_GET(H_ALL_RES_QP_ACT_OUTST_SEND_WR, outs[2]);
- parms->act_nr_recv_wqes =
+ parms->rqueue.act_nr_wqes =
(u16)EHCA_BMASK_GET(H_ALL_RES_QP_ACT_OUTST_RECV_WR, outs[2]);
- parms->act_nr_send_sges =
+ parms->squeue.act_nr_sges =
(u8)EHCA_BMASK_GET(H_ALL_RES_QP_ACT_SEND_SGE, outs[3]);
- parms->act_nr_recv_sges =
+ parms->rqueue.act_nr_sges =
(u8)EHCA_BMASK_GET(H_ALL_RES_QP_ACT_RECV_SGE, outs[3]);
- parms->nr_sq_pages =
+ parms->squeue.queue_size =
(u32)EHCA_BMASK_GET(H_ALL_RES_QP_SQUEUE_SIZE_PAGES, outs[4]);
- parms->nr_rq_pages =
+ parms->rqueue.queue_size =
(u32)EHCA_BMASK_GET(H_ALL_RES_QP_RQUEUE_SIZE_PAGES, outs[4]);
if (ret == H_SUCCESS)
@@ -497,7 +505,7 @@ u64 hipz_h_register_rpage_qp(const struct ipz_adapter_handle adapter_handle,
const u64 count,
const struct h_galpa galpa)
{
- if (count != 1) {
+ if (count > 1) {
ehca_gen_err("Page counter=%lx", count);
return H_PARAMETER;
}
diff --git a/drivers/infiniband/hw/ehca/ipz_pt_fn.c b/drivers/infiniband/hw/ehca/ipz_pt_fn.c
index 9606f13..6506501 100644
--- a/drivers/infiniband/hw/ehca/ipz_pt_fn.c
+++ b/drivers/infiniband/hw/ehca/ipz_pt_fn.c
@@ -40,6 +40,11 @@
#include "ehca_tools.h"
#include "ipz_pt_fn.h"
+#include "ehca_classes.h"
+
+#define PAGES_PER_KPAGE (PAGE_SIZE >> EHCA_PAGESHIFT)
+
+struct kmem_cache *small_qp_cache;
void *ipz_qpageit_get_inc(struct ipz_queue *queue)
{
@@ -49,7 +54,7 @@ void *ipz_qpageit_get_inc(struct ipz_queue *queue)
queue->current_q_offset -= queue->pagesize;
ret = NULL;
}
- if (((u64)ret) % EHCA_PAGESIZE) {
+ if (((u64)ret) % queue->pagesize) {
ehca_gen_err("ERROR!! not at PAGE-Boundary");
return NULL;
}
@@ -83,80 +88,195 @@ int ipz_queue_abs_to_offset(struct ipz_queue *queue, u64 addr, u64 *q_offset)
return -EINVAL;
}
-int ipz_queue_ctor(struct ipz_queue *queue,
- const u32 nr_of_pages,
- const u32 pagesize, const u32 qe_size, const u32 nr_of_sg)
+#if PAGE_SHIFT < EHCA_PAGESHIFT
+#error Kernel pages must be at least as large than eHCA pages (4K) !
+#endif
+
+/*
+ * allocate pages for queue:
+ * outer loop allocates whole kernel pages (page aligned) and
+ * inner loop divides a kernel page into smaller hca queue pages
+ */
+static int alloc_queue_pages(struct ipz_queue *queue, const u32 nr_of_pages)
{
- int pages_per_kpage = PAGE_SIZE >> EHCA_PAGESHIFT;
- int f;
+ int k, f = 0;
+ u8 *kpage;
- if (pagesize > PAGE_SIZE) {
- ehca_gen_err("FATAL ERROR: pagesize=%x is greater "
- "than kernel page size", pagesize);
- return 0;
- }
- if (!pages_per_kpage) {
- ehca_gen_err("FATAL ERROR: invalid kernel page size. "
- "pages_per_kpage=%x", pages_per_kpage);
- return 0;
- }
- queue->queue_length = nr_of_pages * pagesize;
- queue->queue_pages = vmalloc(nr_of_pages * sizeof(void *));
- if (!queue->queue_pages) {
- ehca_gen_err("ERROR!! didn't get the memory");
- return 0;
- }
- memset(queue->queue_pages, 0, nr_of_pages * sizeof(void *));
- /*
- * allocate pages for queue:
- * outer loop allocates whole kernel pages (page aligned) and
- * inner loop divides a kernel page into smaller hca queue pages
- */
- f = 0;
while (f < nr_of_pages) {
- u8 *kpage = (u8 *)get_zeroed_page(GFP_KERNEL);
- int k;
+ kpage = (u8 *)get_zeroed_page(GFP_KERNEL);
if (!kpage)
- goto ipz_queue_ctor_exit0; /*NOMEM*/
- for (k = 0; k < pages_per_kpage && f < nr_of_pages; k++) {
- (queue->queue_pages)[f] = (struct ipz_page *)kpage;
+ goto out;
+
+ for (k = 0; k < PAGES_PER_KPAGE && f < nr_of_pages; k++) {
+ queue->queue_pages[f] = (struct ipz_page *)kpage;
kpage += EHCA_PAGESIZE;
f++;
}
}
+ return 1;
- queue->current_q_offset = 0;
+out:
+ for (f = 0; f < nr_of_pages && queue->queue_pages[f];
+ f += PAGES_PER_KPAGE)
+ free_page((unsigned long)(queue->queue_pages)[f]);
+ return 0;
+}
+
+static int alloc_small_queue_page(struct ipz_queue *queue, struct ehca_pd *pd)
+{
+ int order = ilog2(queue->pagesize) - 9;
+ struct ipz_small_queue_page *page;
+ unsigned long bit;
+
+ mutex_lock(&pd->lock);
+
+ if (!list_empty(&pd->free[order]))
+ page = list_entry(pd->free[order].next,
+ struct ipz_small_queue_page, list);
+ else {
+ page = kmem_cache_zalloc(small_qp_cache, GFP_KERNEL);
+ if (!page)
+ goto out;
+
+ page->page = get_zeroed_page(GFP_KERNEL);
+ if (!page->page) {
+ kmem_cache_free(small_qp_cache, page);
+ goto out;
+ }
+
+ list_add(&page->list, &pd->free[order]);
+ }
+
+ bit = find_first_zero_bit(page->bitmap, IPZ_SPAGE_PER_KPAGE >> order);
+ __set_bit(bit, page->bitmap);
+ page->fill++;
+
+ if (page->fill == IPZ_SPAGE_PER_KPAGE >> order)
+ list_move(&page->list, &pd->full[order]);
+
+ mutex_unlock(&pd->lock);
+
+ queue->queue_pages[0] = (void *)(page->page | (bit << (order + 9)));
+ queue->small_page = page;
+ return 1;
+
+out:
+ ehca_err(pd->ib_pd.device, "failed to allocate small queue page");
+ return 0;
+}
+
+static void free_small_queue_page(struct ipz_queue *queue, struct ehca_pd *pd)
+{
+ int order = ilog2(queue->pagesize) - 9;
+ struct ipz_small_queue_page *page = queue->small_page;
+ unsigned long bit;
+ int free_page = 0;
+
+ bit = ((unsigned long)queue->queue_pages[0] & PAGE_MASK)
+ >> (order + 9);
+
+ mutex_lock(&pd->lock);
+
+ __clear_bit(bit, page->bitmap);
+ page->fill--;
+
+ if (page->fill == 0) {
+ list_del(&page->list);
+ free_page = 1;
+ }
+
+ if (page->fill == (IPZ_SPAGE_PER_KPAGE >> order) - 1)
+ /* the page was full until we freed the chunk */
+ list_move_tail(&page->list, &pd->free[order]);
+
+ mutex_unlock(&pd->lock);
+
+ if (free_page) {
+ free_page(page->page);
+ kmem_cache_free(small_qp_cache, page);
+ }
+}
+
+int ipz_queue_ctor(struct ehca_pd *pd, struct ipz_queue *queue,
+ const u32 nr_of_pages, const u32 pagesize,
+ const u32 qe_size, const u32 nr_of_sg,
+ int is_small)
+{
+ if (pagesize > PAGE_SIZE) {
+ ehca_gen_err("FATAL ERROR: pagesize=%x "
+ "is greater than kernel page size", pagesize);
+ return 0;
+ }
+
+ /* init queue fields */
+ queue->queue_length = nr_of_pages * pagesize;
+ queue->pagesize = pagesize;
queue->qe_size = qe_size;
queue->act_nr_of_sg = nr_of_sg;
- queue->pagesize = pagesize;
+ queue->current_q_offset = 0;
queue->toggle_state = 1;
- return 1;
+ queue->small_page = NULL;
- ipz_queue_ctor_exit0:
- ehca_gen_err("Couldn't get alloc pages queue=%p f=%x nr_of_pages=%x",
- queue, f, nr_of_pages);
- for (f = 0; f < nr_of_pages; f += pages_per_kpage) {
- if (!(queue->queue_pages)[f])
- break;
- free_page((unsigned long)(queue->queue_pages)[f]);
+ /* allocate queue page pointers */
+ queue->queue_pages = vmalloc(nr_of_pages * sizeof(void *));
+ if (!queue->queue_pages) {
+ ehca_gen_err("Couldn't allocate queue page list");
+ return 0;
}
+ memset(queue->queue_pages, 0, nr_of_pages * sizeof(void *));
+
+ /* allocate actual queue pages */
+ if (is_small) {
+ if (!alloc_small_queue_page(queue, pd))
+ goto ipz_queue_ctor_exit0;
+ } else
+ if (!alloc_queue_pages(queue, nr_of_pages))
+ goto ipz_queue_ctor_exit0;
+
+ return 1;
+
+ipz_queue_ctor_exit0:
+ ehca_gen_err("Couldn't alloc pages queue=%p "
+ "nr_of_pages=%x", queue, nr_of_pages);
+ vfree(queue->queue_pages);
+
return 0;
}
-int ipz_queue_dtor(struct ipz_queue *queue)
+int ipz_queue_dtor(struct ehca_pd *pd, struct ipz_queue *queue)
{
- int pages_per_kpage = PAGE_SIZE >> EHCA_PAGESHIFT;
- int g;
- int nr_pages;
+ int i, nr_pages;
if (!queue || !queue->queue_pages) {
ehca_gen_dbg("queue or queue_pages is NULL");
return 0;
}
- nr_pages = queue->queue_length / queue->pagesize;
- for (g = 0; g < nr_pages; g += pages_per_kpage)
- free_page((unsigned long)(queue->queue_pages)[g]);
+
+ if (queue->small_page)
+ free_small_queue_page(queue, pd);
+ else {
+ nr_pages = queue->queue_length / queue->pagesize;
+ for (i = 0; i < nr_pages; i += PAGES_PER_KPAGE)
+ free_page((unsigned long)queue->queue_pages[i]);
+ }
+
vfree(queue->queue_pages);
return 1;
}
+
+int ehca_init_small_qp_cache(void)
+{
+ small_qp_cache = kmem_cache_create("ehca_cache_small_qp",
+ sizeof(struct ipz_small_queue_page),
+ 0, SLAB_HWCACHE_ALIGN, NULL, NULL);
+ if (!small_qp_cache)
+ return -ENOMEM;
+
+ return 0;
+}
+
+void ehca_cleanup_small_qp_cache(void)
+{
+ kmem_cache_destroy(small_qp_cache);
+}
diff --git a/drivers/infiniband/hw/ehca/ipz_pt_fn.h b/drivers/infiniband/hw/ehca/ipz_pt_fn.h
index 39a4f64..c6937a0 100644
--- a/drivers/infiniband/hw/ehca/ipz_pt_fn.h
+++ b/drivers/infiniband/hw/ehca/ipz_pt_fn.h
@@ -51,11 +51,25 @@
#include "ehca_tools.h"
#include "ehca_qes.h"
+struct ehca_pd;
+struct ipz_small_queue_page;
+
/* struct generic ehca page */
struct ipz_page {
u8 entries[EHCA_PAGESIZE];
};
+#define IPZ_SPAGE_PER_KPAGE (PAGE_SIZE / 512)
+
+struct ipz_small_queue_page {
+ unsigned long page;
+ unsigned long bitmap[IPZ_SPAGE_PER_KPAGE / BITS_PER_LONG];
+ int fill;
+ void *mapped_addr;
+ u32 mmap_count;
+ struct list_head list;
+};
+
/* struct generic queue in linux kernel virtual memory (kv) */
struct ipz_queue {
u64 current_q_offset; /* current queue entry */
@@ -66,7 +80,8 @@ struct ipz_queue {
u32 queue_length; /* queue length allocated in bytes */
u32 pagesize;
u32 toggle_state; /* toggle flag - per page */
- u32 dummy3; /* 64 bit alignment */
+ u32 offset; /* save offset within page for small_qp */
+ struct ipz_small_queue_page *small_page;
};
/*
@@ -188,9 +203,10 @@ struct ipz_qpt {
* see ipz_qpt_ctor()
* returns true if ok, false if out of memory
*/
-int ipz_queue_ctor(struct ipz_queue *queue, const u32 nr_of_pages,
- const u32 pagesize, const u32 qe_size,
- const u32 nr_of_sg);
+int ipz_queue_ctor(struct ehca_pd *pd, struct ipz_queue *queue,
+ const u32 nr_of_pages, const u32 pagesize,
+ const u32 qe_size, const u32 nr_of_sg,
+ int is_small);
/*
* destructor for a ipz_queue_t
@@ -198,7 +214,7 @@ int ipz_queue_ctor(struct ipz_queue *queue, const u32 nr_of_pages,
* see ipz_queue_ctor()
* returns true if ok, false if queue was NULL-ptr of free failed
*/
-int ipz_queue_dtor(struct ipz_queue *queue);
+int ipz_queue_dtor(struct ehca_pd *pd, struct ipz_queue *queue);
/*
* constructor for a ipz_qpt_t,
--
1.5.2
^ permalink raw reply related
* Interrupts on xilinx ml403
From: Mirek23 @ 2007-07-20 13:52 UTC (permalink / raw)
To: linuxppc-embedded
Dear All,
I use linux kernel 2.6 on ppc405 of my Avnet (xilinx like ml403)
evaluation board.
I have setup the virtex-4 FPGA to deal with Themac and Serial interfaces. As
input/output devices I have chosen 8 LEDs and DIP Switches.
With such a configuration I am able to control from Linux user applications
via GPIO driver the LEDs and DIP Switches.
Right now I just wanted to make use of the interrupts. I have configured the
Dip switches to use interrupt. The interrupt accoures when the DIP Switches
state has changed.
In the BSP generated by EDK 9.1 I see that macro :
#define XPAR_DIP_SWITCHES_8BIT_INTERRUPT_PRESENT 1
has changed from zero to one.
The macro XPAR_INTC_MAX_NUM_INTR_INPUTS is set to 1 as it was before. This
is due to the fact that
TEMAC uses one interrupt line.
Does it mean that DIP Switches do not use INTC interrupt controller?
How to handle the DIP Switches interrupt?
Does the Interrupt handler routine have to acknowledge the interrupt from
Dip Switches?
Many thanks in advance for any hint on that.
Best Regards
Mirek
--
View this message in context: http://www.nabble.com/Interrupts-on-xilinx-ml403-tf4117226.html#a11708607
Sent from the linuxppc-embedded mailing list archive at Nabble.com.
^ permalink raw reply
* [PATCH 4/5] ehca: Make internal_create/destroy_qp() static
From: Hoang-Nam Nguyen @ 2007-07-20 14:03 UTC (permalink / raw)
To: Roland Dreier, linux-kernel, linuxppc-dev, openib-general
Cc: fenkes, raisch, stefan.roscher
From: Joachim Fenkes <fenkes@de.ibm.com>
Date: Thu, 19 Jul 2007 21:40:00 +0200
Subject: [PATCH 4/5] IB/ehca: Make internal_{create,destroy}_qp() static
They're only used in ehca_qp.c
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_qp.c | 17 +++++++++--------
1 files changed, 9 insertions(+), 8 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c
index 48e9cea..b916d9c 100644
--- a/drivers/infiniband/hw/ehca/ehca_qp.c
+++ b/drivers/infiniband/hw/ehca/ehca_qp.c
@@ -363,10 +363,11 @@ init_qp_queue1:
* the value of the is_srq parameter. If init_attr and srq_init_attr share
* fields, the field out of init_attr is used.
*/
-struct ehca_qp *internal_create_qp(struct ib_pd *pd,
- struct ib_qp_init_attr *init_attr,
- struct ib_srq_init_attr *srq_init_attr,
- struct ib_udata *udata, int is_srq)
+static struct ehca_qp *internal_create_qp(
+ struct ib_pd *pd,
+ struct ib_qp_init_attr *init_attr,
+ struct ib_srq_init_attr *srq_init_attr,
+ struct ib_udata *udata, int is_srq)
{
struct ehca_qp *my_qp;
struct ehca_pd *my_pd = container_of(pd, struct ehca_pd, ib_pd);
@@ -752,8 +753,8 @@ struct ib_qp *ehca_create_qp(struct ib_pd *pd,
return IS_ERR(ret) ? (struct ib_qp *)ret : &ret->ib_qp;
}
-int internal_destroy_qp(struct ib_device *dev, struct ehca_qp *my_qp,
- struct ib_uobject *uobject);
+static int internal_destroy_qp(struct ib_device *dev, struct ehca_qp *my_qp,
+ struct ib_uobject *uobject);
struct ib_srq *ehca_create_srq(struct ib_pd *pd,
struct ib_srq_init_attr *srq_init_attr,
@@ -1669,8 +1670,8 @@ query_srq_exit1:
return ret;
}
-int internal_destroy_qp(struct ib_device *dev, struct ehca_qp *my_qp,
- struct ib_uobject *uobject)
+static int internal_destroy_qp(struct ib_device *dev, struct ehca_qp *my_qp,
+ struct ib_uobject *uobject)
{
struct ehca_shca *shca = container_of(dev, struct ehca_shca, ib_device);
struct ehca_pd *my_pd = container_of(my_qp->ib_qp.pd, struct ehca_pd,
--
1.5.2
^ permalink raw reply related
* [PATCH 3/5] ehca: Make ehca2ib_return_code() non-inline
From: Hoang-Nam Nguyen @ 2007-07-20 14:02 UTC (permalink / raw)
To: Roland Dreier, linux-kernel, linuxppc-dev, openib-general
Cc: fenkes, raisch, stefan.roscher
From: Joachim Fenkes <fenkes@de.ibm.com>
Date: Thu, 19 Jul 2007 21:13:57 +0200
Subject: [PATCH 3/5] IB/ehca: Make ehca2ib_return_code() non-inline
It's nowhere in the main path and making it non-inline saves ~1.5K of code.
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_main.c | 17 +++++++++++++++++
drivers/infiniband/hw/ehca/ehca_tools.h | 19 +------------------
2 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c
index 34661c3..3bd7afb 100644
--- a/drivers/infiniband/hw/ehca/ehca_main.c
+++ b/drivers/infiniband/hw/ehca/ehca_main.c
@@ -130,6 +130,23 @@ void ehca_free_fw_ctrlblock(void *ptr)
}
#endif
+int ehca2ib_return_code(u64 ehca_rc)
+{
+ switch (ehca_rc) {
+ case H_SUCCESS:
+ return 0;
+ case H_RESOURCE: /* Resource in use */
+ case H_BUSY:
+ return -EBUSY;
+ case H_NOT_ENOUGH_RESOURCES: /* insufficient resources */
+ case H_CONSTRAINED: /* resource constraint */
+ case H_NO_MEM:
+ return -ENOMEM;
+ default:
+ return -EINVAL;
+ }
+}
+
static int ehca_create_slab_caches(void)
{
int ret;
diff --git a/drivers/infiniband/hw/ehca/ehca_tools.h b/drivers/infiniband/hw/ehca/ehca_tools.h
index 678b813..57c77a7 100644
--- a/drivers/infiniband/hw/ehca/ehca_tools.h
+++ b/drivers/infiniband/hw/ehca/ehca_tools.h
@@ -154,24 +154,7 @@ extern int ehca_debug_level;
#define EHCA_BMASK_GET(mask, value) \
(EHCA_BMASK_MASK(mask) & (((u64)(value)) >> EHCA_BMASK_SHIFTPOS(mask)))
-
/* Converts ehca to ib return code */
-static inline int ehca2ib_return_code(u64 ehca_rc)
-{
- switch (ehca_rc) {
- case H_SUCCESS:
- return 0;
- case H_RESOURCE: /* Resource in use */
- case H_BUSY:
- return -EBUSY;
- case H_NOT_ENOUGH_RESOURCES: /* insufficient resources */
- case H_CONSTRAINED: /* resource constraint */
- case H_NO_MEM:
- return -ENOMEM;
- default:
- return -EINVAL;
- }
-}
-
+int ehca2ib_return_code(u64 ehca_rc);
#endif /* EHCA_TOOLS_H */
--
1.5.2
^ permalink raw reply related
* [PATCH 2/5] ehca: Generate event when SRQ limit reached
From: Hoang-Nam Nguyen @ 2007-07-20 14:02 UTC (permalink / raw)
To: Roland Dreier, linux-kernel, linuxppc-dev, openib-general
Cc: fenkes, raisch, stefan.roscher
From: Joachim Fenkes <fenkes@de.ibm.com>
Date: Thu, 19 Jul 2007 20:51:43 +0200
Subject: [PATCH 2/5] IB/ehca: Generate event when SRQ limit reached
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_irq.c | 42 ++++++++++++++++++++++-----------
1 files changed, 28 insertions(+), 14 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_irq.c b/drivers/infiniband/hw/ehca/ehca_irq.c
index 4fb01fc..71c0799 100644
--- a/drivers/infiniband/hw/ehca/ehca_irq.c
+++ b/drivers/infiniband/hw/ehca/ehca_irq.c
@@ -175,9 +175,8 @@ error_data1:
}
-static void qp_event_callback(struct ehca_shca *shca,
- u64 eqe,
- enum ib_event_type event_type)
+static void qp_event_callback(struct ehca_shca *shca, u64 eqe,
+ enum ib_event_type event_type, int fatal)
{
struct ib_event event;
struct ehca_qp *qp;
@@ -191,16 +190,26 @@ static void qp_event_callback(struct ehca_shca *shca,
if (!qp)
return;
- ehca_error_data(shca, qp, qp->ipz_qp_handle.handle);
+ if (fatal)
+ ehca_error_data(shca, qp, qp->ipz_qp_handle.handle);
- if (!qp->ib_qp.event_handler)
- return;
+ event.device = &shca->ib_device;
- event.device = &shca->ib_device;
- event.event = event_type;
- event.element.qp = &qp->ib_qp;
+ if (qp->ext_type == EQPT_SRQ) {
+ if (!qp->ib_srq.event_handler)
+ return;
- qp->ib_qp.event_handler(&event, qp->ib_qp.qp_context);
+ event.event = fatal ? IB_EVENT_SRQ_ERR : event_type;
+ event.element.srq = &qp->ib_srq;
+ qp->ib_srq.event_handler(&event, qp->ib_srq.srq_context);
+ } else {
+ if (!qp->ib_qp.event_handler)
+ return;
+
+ event.event = event_type;
+ event.element.qp = &qp->ib_qp;
+ qp->ib_qp.event_handler(&event, qp->ib_qp.qp_context);
+ }
return;
}
@@ -234,17 +243,17 @@ static void parse_identifier(struct ehca_shca *shca, u64 eqe)
switch (identifier) {
case 0x02: /* path migrated */
- qp_event_callback(shca, eqe, IB_EVENT_PATH_MIG);
+ qp_event_callback(shca, eqe, IB_EVENT_PATH_MIG, 0);
break;
case 0x03: /* communication established */
- qp_event_callback(shca, eqe, IB_EVENT_COMM_EST);
+ qp_event_callback(shca, eqe, IB_EVENT_COMM_EST, 0);
break;
case 0x04: /* send queue drained */
- qp_event_callback(shca, eqe, IB_EVENT_SQ_DRAINED);
+ qp_event_callback(shca, eqe, IB_EVENT_SQ_DRAINED, 0);
break;
case 0x05: /* QP error */
case 0x06: /* QP error */
- qp_event_callback(shca, eqe, IB_EVENT_QP_FATAL);
+ qp_event_callback(shca, eqe, IB_EVENT_QP_FATAL, 1);
break;
case 0x07: /* CQ error */
case 0x08: /* CQ error */
@@ -278,6 +287,11 @@ static void parse_identifier(struct ehca_shca *shca, u64 eqe)
ehca_err(&shca->ib_device, "Interface trace stopped.");
break;
case 0x14: /* first error capture info available */
+ ehca_info(&shca->ib_device, "First error capture available");
+ break;
+ case 0x15: /* SRQ limit reached */
+ qp_event_callback(shca, eqe, IB_EVENT_SRQ_LIMIT_REACHED, 0);
+ break;
default:
ehca_err(&shca->ib_device, "Unknown identifier: %x on %s.",
identifier, shca->ib_device.name);
--
1.5.2
^ permalink raw reply related
* [PATCH 1/5] ehca: Supports large page MRs
From: Hoang-Nam Nguyen @ 2007-07-20 14:01 UTC (permalink / raw)
To: Roland Dreier, linux-kernel, linuxppc-dev, openib-general
Cc: fenkes, raisch, stefan.roscher
From: Hoang-Nam Nguyen <hnguyen@de.ibm.com>
Date: Thu, 19 Jul 2007 20:48:04 +0200
Subject: [PATCH 1/5] IB/ehca: Support large page MRs
Add support for MR pages larger than 4K on eHCA2. This reduces firmware
memory consumption. If enabled via the mr_largepage module parameter, the MR
page size will be determined based on the MR length and the hardware
capabilities - if the MR is >= 16M, 16M pages are used, for example.
Signed-off-by: Joachim Fenkes <fenkes@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_classes.h | 9 +
drivers/infiniband/hw/ehca/ehca_main.c | 18 ++-
drivers/infiniband/hw/ehca/ehca_mrmw.c | 371 ++++++++++++++++++++++++-----
drivers/infiniband/hw/ehca/ehca_mrmw.h | 2 +-
drivers/infiniband/hw/ehca/hcp_if.c | 20 ++-
5 files changed, 357 insertions(+), 63 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h
index 043e4fb..63b8b9f 100644
--- a/drivers/infiniband/hw/ehca/ehca_classes.h
+++ b/drivers/infiniband/hw/ehca/ehca_classes.h
@@ -100,6 +100,11 @@ struct ehca_sport {
struct ehca_sma_attr saved_attr;
};
+#define HCA_CAP_MR_PGSIZE_4K 1
+#define HCA_CAP_MR_PGSIZE_64K 2
+#define HCA_CAP_MR_PGSIZE_1M 4
+#define HCA_CAP_MR_PGSIZE_16M 8
+
struct ehca_shca {
struct ib_device ib_device;
struct ibmebus_dev *ibmebus_dev;
@@ -115,6 +120,8 @@ struct ehca_shca {
struct h_galpas galpas;
struct mutex modify_mutex;
u64 hca_cap;
+ /* MR pgsize: bit 0-3 means 4K, 64K, 1M, 16M respectively */
+ u32 hca_cap_mr_pgsize;
int max_mtu;
};
@@ -206,6 +213,7 @@ struct ehca_mr {
enum ehca_mr_flag flags;
u32 num_kpages; /* number of kernel pages */
u32 num_hwpages; /* number of hw pages to form MR */
+ u64 hwpage_size; /* hw page size used for this MR */
int acl; /* ACL (stored here for usage in reregister) */
u64 *start; /* virtual start address (stored here for */
/* usage in reregister) */
@@ -240,6 +248,7 @@ struct ehca_mr_pginfo {
enum ehca_mr_pgi_type type;
u64 num_kpages;
u64 kpage_cnt;
+ u64 hwpage_size; /* hw page size used for this MR */
u64 num_hwpages; /* number of hw pages */
u64 hwpage_cnt; /* counter for hw pages */
u64 next_hwpage; /* next hw page in buffer/chunk/listelem */
diff --git a/drivers/infiniband/hw/ehca/ehca_main.c b/drivers/infiniband/hw/ehca/ehca_main.c
index 36377c6..34661c3 100644
--- a/drivers/infiniband/hw/ehca/ehca_main.c
+++ b/drivers/infiniband/hw/ehca/ehca_main.c
@@ -63,6 +63,7 @@ int ehca_port_act_time = 30;
int ehca_poll_all_eqs = 1;
int ehca_static_rate = -1;
int ehca_scaling_code = 0;
+int ehca_mr_largepage = 0;
module_param_named(open_aqp1, ehca_open_aqp1, int, 0);
module_param_named(debug_level, ehca_debug_level, int, 0);
@@ -72,7 +73,8 @@ module_param_named(use_hp_mr, ehca_use_hp_mr, int, 0);
module_param_named(port_act_time, ehca_port_act_time, int, 0);
module_param_named(poll_all_eqs, ehca_poll_all_eqs, int, 0);
module_param_named(static_rate, ehca_static_rate, int, 0);
-module_param_named(scaling_code, ehca_scaling_code, int, 0);
+module_param_named(scaling_code, ehca_scaling_code, int, 0);
+module_param_named(mr_largepage, ehca_mr_largepage, int, 0);
MODULE_PARM_DESC(open_aqp1,
"AQP1 on startup (0: no (default), 1: yes)");
@@ -95,6 +97,9 @@ MODULE_PARM_DESC(static_rate,
"set permanent static rate (default: disabled)");
MODULE_PARM_DESC(scaling_code,
"set scaling code (0: disabled/default, 1: enabled)");
+MODULE_PARM_DESC(mr_largepage,
+ "use large page for MR (0: use PAGE_SIZE (default), "
+ "1: use large page depending on MR size");
DEFINE_RWLOCK(ehca_qp_idr_lock);
DEFINE_RWLOCK(ehca_cq_idr_lock);
@@ -295,6 +300,8 @@ int ehca_sense_attributes(struct ehca_shca *shca)
if (EHCA_BMASK_GET(hca_cap_descr[i].mask, shca->hca_cap))
ehca_gen_dbg(" %s", hca_cap_descr[i].descr);
+ shca->hca_cap_mr_pgsize = rblock->memory_page_size_supported;
+
port = (struct hipz_query_port *)rblock;
h_ret = hipz_h_query_port(shca->ipz_hca_handle, 1, port);
if (h_ret != H_SUCCESS) {
@@ -590,6 +597,14 @@ static ssize_t ehca_show_adapter_handle(struct device *dev,
}
static DEVICE_ATTR(adapter_handle, S_IRUGO, ehca_show_adapter_handle, NULL);
+static ssize_t ehca_show_mr_largepage(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ return sprintf(buf, "%d\n", ehca_mr_largepage);
+}
+static DEVICE_ATTR(mr_largepage, S_IRUGO, ehca_show_mr_largepage, NULL);
+
static struct attribute *ehca_dev_attrs[] = {
&dev_attr_adapter_handle.attr,
&dev_attr_num_ports.attr,
@@ -606,6 +621,7 @@ static struct attribute *ehca_dev_attrs[] = {
&dev_attr_cur_mw.attr,
&dev_attr_max_pd.attr,
&dev_attr_max_ah.attr,
+ &dev_attr_mr_largepage.attr,
NULL
};
diff --git a/drivers/infiniband/hw/ehca/ehca_mrmw.c b/drivers/infiniband/hw/ehca/ehca_mrmw.c
index 6262c54..ba28783 100644
--- a/drivers/infiniband/hw/ehca/ehca_mrmw.c
+++ b/drivers/infiniband/hw/ehca/ehca_mrmw.c
@@ -5,6 +5,7 @@
*
* Authors: Dietmar Decker <ddecker@de.ibm.com>
* Christoph Raisch <raisch@de.ibm.com>
+ * Hoang-Nam Nguyen <hnguyen@de.ibm.com>
*
* Copyright (c) 2005 IBM Corporation
*
@@ -56,6 +57,37 @@
static struct kmem_cache *mr_cache;
static struct kmem_cache *mw_cache;
+enum ehca_mr_pgsize {
+ EHCA_MR_PGSIZE4K = 0x1000L,
+ EHCA_MR_PGSIZE64K = 0x10000L,
+ EHCA_MR_PGSIZE1M = 0x100000L,
+ EHCA_MR_PGSIZE16M = 0x1000000L
+};
+
+extern int ehca_mr_largepage;
+
+static u32 ehca_encode_hwpage_size(u32 pgsize)
+{
+ u32 idx = 0;
+ pgsize >>= 12;
+ /*
+ * map mr page size into hw code:
+ * 0, 1, 2, 3 for 4K, 64K, 1M, 64M
+ */
+ while (!(pgsize & 1)) {
+ idx++;
+ pgsize >>= 4;
+ }
+ return idx;
+}
+
+static u64 ehca_get_max_hwpage_size(struct ehca_shca *shca)
+{
+ if (shca->hca_cap_mr_pgsize & HCA_CAP_MR_PGSIZE_16M)
+ return EHCA_MR_PGSIZE16M;
+ return EHCA_MR_PGSIZE4K;
+}
+
static struct ehca_mr *ehca_mr_new(void)
{
struct ehca_mr *me;
@@ -207,19 +239,23 @@ struct ib_mr *ehca_reg_phys_mr(struct ib_pd *pd,
struct ehca_mr_pginfo pginfo;
u32 num_kpages;
u32 num_hwpages;
+ u64 hw_pgsize;
num_kpages = NUM_CHUNKS(((u64)iova_start % PAGE_SIZE) + size,
PAGE_SIZE);
- num_hwpages = NUM_CHUNKS(((u64)iova_start % EHCA_PAGESIZE) +
- size, EHCA_PAGESIZE);
+ /* for kernel space we try most possible pgsize */
+ hw_pgsize = ehca_get_max_hwpage_size(shca);
+ num_hwpages = NUM_CHUNKS(((u64)iova_start % hw_pgsize) + size,
+ hw_pgsize);
memset(&pginfo, 0, sizeof(pginfo));
pginfo.type = EHCA_MR_PGI_PHYS;
pginfo.num_kpages = num_kpages;
+ pginfo.hwpage_size = hw_pgsize;
pginfo.num_hwpages = num_hwpages;
pginfo.u.phy.num_phys_buf = num_phys_buf;
pginfo.u.phy.phys_buf_array = phys_buf_array;
- pginfo.next_hwpage = (((u64)iova_start & ~PAGE_MASK) /
- EHCA_PAGESIZE);
+ pginfo.next_hwpage =
+ ((u64)iova_start & ~(hw_pgsize - 1)) / hw_pgsize;
ret = ehca_reg_mr(shca, e_mr, iova_start, size, mr_access_flags,
e_pd, &pginfo, &e_mr->ib.ib_mr.lkey,
@@ -259,6 +295,7 @@ struct ib_mr *ehca_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
int ret;
u32 num_kpages;
u32 num_hwpages;
+ u64 hwpage_size;
if (!pd) {
ehca_gen_err("bad pd=%p", pd);
@@ -309,16 +346,32 @@ struct ib_mr *ehca_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
/* determine number of MR pages */
num_kpages = NUM_CHUNKS((virt % PAGE_SIZE) + length, PAGE_SIZE);
- num_hwpages = NUM_CHUNKS((virt % EHCA_PAGESIZE) + length,
- EHCA_PAGESIZE);
+ /* select proper hw_pgsize */
+ if (ehca_mr_largepage &&
+ (shca->hca_cap_mr_pgsize & HCA_CAP_MR_PGSIZE_16M)) {
+ if (length <= EHCA_MR_PGSIZE4K
+ && PAGE_SIZE == EHCA_MR_PGSIZE4K)
+ hwpage_size = EHCA_MR_PGSIZE4K;
+ else if (length <= EHCA_MR_PGSIZE64K)
+ hwpage_size = EHCA_MR_PGSIZE64K;
+ else if (length <= EHCA_MR_PGSIZE1M)
+ hwpage_size = EHCA_MR_PGSIZE1M;
+ else
+ hwpage_size = EHCA_MR_PGSIZE16M;
+ } else
+ hwpage_size = EHCA_MR_PGSIZE4K;
+ ehca_dbg(pd->device, "hwpage_size=%lx", hwpage_size);
+reg_user_mr_fallback:
+ num_hwpages = NUM_CHUNKS((virt % hwpage_size) + length, hwpage_size);
/* register MR on HCA */
memset(&pginfo, 0, sizeof(pginfo));
pginfo.type = EHCA_MR_PGI_USER;
+ pginfo.hwpage_size = hwpage_size;
pginfo.num_kpages = num_kpages;
pginfo.num_hwpages = num_hwpages;
pginfo.u.usr.region = e_mr->umem;
- pginfo.next_hwpage = e_mr->umem->offset / EHCA_PAGESIZE;
+ pginfo.next_hwpage = e_mr->umem->offset / hwpage_size;
pginfo.u.usr.next_chunk = list_prepare_entry(pginfo.u.usr.next_chunk,
(&e_mr->umem->chunk_list),
list);
@@ -326,6 +379,18 @@ struct ib_mr *ehca_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
ret = ehca_reg_mr(shca, e_mr, (u64 *)virt, length, mr_access_flags,
e_pd, &pginfo, &e_mr->ib.ib_mr.lkey,
&e_mr->ib.ib_mr.rkey);
+ if (ret == -EINVAL && pginfo.hwpage_size > PAGE_SIZE) {
+ ehca_warn(pd->device, "failed to register mr "
+ "with hwpage_size=%lx", hwpage_size);
+ ehca_info(pd->device, "try to register mr with "
+ "kpage_size=%lx", PAGE_SIZE);
+ /*
+ * this means kpages are not contiguous for a hw page
+ * try kernel page size as fallback solution
+ */
+ hwpage_size = PAGE_SIZE;
+ goto reg_user_mr_fallback;
+ }
if (ret) {
ib_mr = ERR_PTR(ret);
goto reg_user_mr_exit2;
@@ -452,6 +517,8 @@ int ehca_rereg_phys_mr(struct ib_mr *mr,
new_pd = container_of(mr->pd, struct ehca_pd, ib_pd);
if (mr_rereg_mask & IB_MR_REREG_TRANS) {
+ u64 hw_pgsize = ehca_get_max_hwpage_size(shca);
+
new_start = iova_start; /* change address */
/* check physical buffer list and calculate size */
ret = ehca_mr_chk_buf_and_calc_size(phys_buf_array,
@@ -468,16 +535,17 @@ int ehca_rereg_phys_mr(struct ib_mr *mr,
}
num_kpages = NUM_CHUNKS(((u64)new_start % PAGE_SIZE) +
new_size, PAGE_SIZE);
- num_hwpages = NUM_CHUNKS(((u64)new_start % EHCA_PAGESIZE) +
- new_size, EHCA_PAGESIZE);
+ num_hwpages = NUM_CHUNKS(((u64)new_start % hw_pgsize) +
+ new_size, hw_pgsize);
memset(&pginfo, 0, sizeof(pginfo));
pginfo.type = EHCA_MR_PGI_PHYS;
pginfo.num_kpages = num_kpages;
+ pginfo.hwpage_size = hw_pgsize;
pginfo.num_hwpages = num_hwpages;
pginfo.u.phy.num_phys_buf = num_phys_buf;
pginfo.u.phy.phys_buf_array = phys_buf_array;
- pginfo.next_hwpage = (((u64)iova_start & ~PAGE_MASK) /
- EHCA_PAGESIZE);
+ pginfo.next_hwpage =
+ ((u64)iova_start & ~(hw_pgsize - 1)) / hw_pgsize;
}
if (mr_rereg_mask & IB_MR_REREG_ACCESS)
new_acl = mr_access_flags;
@@ -709,6 +777,7 @@ struct ib_fmr *ehca_alloc_fmr(struct ib_pd *pd,
int ret;
u32 tmp_lkey, tmp_rkey;
struct ehca_mr_pginfo pginfo;
+ u64 hw_pgsize;
/* check other parameters */
if (((mr_access_flags & IB_ACCESS_REMOTE_WRITE) &&
@@ -738,8 +807,8 @@ struct ib_fmr *ehca_alloc_fmr(struct ib_pd *pd,
ib_fmr = ERR_PTR(-EINVAL);
goto alloc_fmr_exit0;
}
- if (((1 << fmr_attr->page_shift) != EHCA_PAGESIZE) &&
- ((1 << fmr_attr->page_shift) != PAGE_SIZE)) {
+ hw_pgsize = ehca_get_max_hwpage_size(shca);
+ if ((1 << fmr_attr->page_shift) != hw_pgsize) {
ehca_err(pd->device, "unsupported fmr_attr->page_shift=%x",
fmr_attr->page_shift);
ib_fmr = ERR_PTR(-EINVAL);
@@ -755,6 +824,10 @@ struct ib_fmr *ehca_alloc_fmr(struct ib_pd *pd,
/* register MR on HCA */
memset(&pginfo, 0, sizeof(pginfo));
+ /*
+ * pginfo.num_hwpages==0, ie register_rpages() will not be called
+ * but deferred to map_phys_fmr()
+ */
ret = ehca_reg_mr(shca, e_fmr, NULL,
fmr_attr->max_pages * (1 << fmr_attr->page_shift),
mr_access_flags, e_pd, &pginfo,
@@ -765,6 +838,7 @@ struct ib_fmr *ehca_alloc_fmr(struct ib_pd *pd,
}
/* successful */
+ e_fmr->hwpage_size = hw_pgsize;
e_fmr->fmr_page_size = 1 << fmr_attr->page_shift;
e_fmr->fmr_max_pages = fmr_attr->max_pages;
e_fmr->fmr_max_maps = fmr_attr->max_maps;
@@ -822,10 +896,12 @@ int ehca_map_phys_fmr(struct ib_fmr *fmr,
memset(&pginfo, 0, sizeof(pginfo));
pginfo.type = EHCA_MR_PGI_FMR;
pginfo.num_kpages = list_len;
- pginfo.num_hwpages = list_len * (e_fmr->fmr_page_size / EHCA_PAGESIZE);
+ pginfo.hwpage_size = e_fmr->hwpage_size;
+ pginfo.num_hwpages =
+ list_len * e_fmr->fmr_page_size / pginfo.hwpage_size;
pginfo.u.fmr.page_list = page_list;
- pginfo.next_hwpage = ((iova & (e_fmr->fmr_page_size-1)) /
- EHCA_PAGESIZE);
+ pginfo.next_hwpage =
+ (iova & (e_fmr->fmr_page_size-1)) / pginfo.hwpage_size;
pginfo.u.fmr.fmr_pgsize = e_fmr->fmr_page_size;
ret = ehca_rereg_mr(shca, e_fmr, (u64 *)iova,
@@ -964,7 +1040,7 @@ int ehca_reg_mr(struct ehca_shca *shca,
struct ehca_mr_hipzout_parms hipzout;
ehca_mrmw_map_acl(acl, &hipz_acl);
- ehca_mrmw_set_pgsize_hipz_acl(&hipz_acl);
+ ehca_mrmw_set_pgsize_hipz_acl(pginfo->hwpage_size, &hipz_acl);
if (ehca_use_hp_mr == 1)
hipz_acl |= 0x00000001;
@@ -987,6 +1063,7 @@ int ehca_reg_mr(struct ehca_shca *shca,
/* successful registration */
e_mr->num_kpages = pginfo->num_kpages;
e_mr->num_hwpages = pginfo->num_hwpages;
+ e_mr->hwpage_size = pginfo->hwpage_size;
e_mr->start = iova_start;
e_mr->size = size;
e_mr->acl = acl;
@@ -1029,6 +1106,9 @@ int ehca_reg_mr_rpages(struct ehca_shca *shca,
u32 i;
u64 *kpage;
+ if (!pginfo->num_hwpages) /* in case of fmr */
+ return 0;
+
kpage = ehca_alloc_fw_ctrlblock(GFP_KERNEL);
if (!kpage) {
ehca_err(&shca->ib_device, "kpage alloc failed");
@@ -1036,7 +1116,7 @@ int ehca_reg_mr_rpages(struct ehca_shca *shca,
goto ehca_reg_mr_rpages_exit0;
}
- /* max 512 pages per shot */
+ /* max MAX_RPAGES ehca mr pages per register call */
for (i = 0; i < NUM_CHUNKS(pginfo->num_hwpages, MAX_RPAGES); i++) {
if (i == NUM_CHUNKS(pginfo->num_hwpages, MAX_RPAGES) - 1) {
@@ -1049,8 +1129,8 @@ int ehca_reg_mr_rpages(struct ehca_shca *shca,
ret = ehca_set_pagebuf(pginfo, rnum, kpage);
if (ret) {
ehca_err(&shca->ib_device, "ehca_set_pagebuf "
- "bad rc, ret=%x rnum=%x kpage=%p",
- ret, rnum, kpage);
+ "bad rc, ret=%x rnum=%x kpage=%p",
+ ret, rnum, kpage);
goto ehca_reg_mr_rpages_exit1;
}
@@ -1065,9 +1145,10 @@ int ehca_reg_mr_rpages(struct ehca_shca *shca,
} else
rpage = *kpage;
- h_ret = hipz_h_register_rpage_mr(shca->ipz_hca_handle, e_mr,
- 0, /* pagesize 4k */
- 0, rpage, rnum);
+ h_ret = hipz_h_register_rpage_mr(
+ shca->ipz_hca_handle, e_mr,
+ ehca_encode_hwpage_size(pginfo->hwpage_size),
+ 0, rpage, rnum);
if (i == NUM_CHUNKS(pginfo->num_hwpages, MAX_RPAGES) - 1) {
/*
@@ -1131,7 +1212,7 @@ inline int ehca_rereg_mr_rereg1(struct ehca_shca *shca,
struct ehca_mr_hipzout_parms hipzout;
ehca_mrmw_map_acl(acl, &hipz_acl);
- ehca_mrmw_set_pgsize_hipz_acl(&hipz_acl);
+ ehca_mrmw_set_pgsize_hipz_acl(pginfo->hwpage_size, &hipz_acl);
kpage = ehca_alloc_fw_ctrlblock(GFP_KERNEL);
if (!kpage) {
@@ -1182,6 +1263,7 @@ inline int ehca_rereg_mr_rereg1(struct ehca_shca *shca,
*/
e_mr->num_kpages = pginfo->num_kpages;
e_mr->num_hwpages = pginfo->num_hwpages;
+ e_mr->hwpage_size = pginfo->hwpage_size;
e_mr->start = iova_start;
e_mr->size = size;
e_mr->acl = acl;
@@ -1268,13 +1350,14 @@ int ehca_rereg_mr(struct ehca_shca *shca,
/* set some MR values */
e_mr->flags = save_mr.flags;
+ e_mr->hwpage_size = save_mr.hwpage_size;
e_mr->fmr_page_size = save_mr.fmr_page_size;
e_mr->fmr_max_pages = save_mr.fmr_max_pages;
e_mr->fmr_max_maps = save_mr.fmr_max_maps;
e_mr->fmr_map_cnt = save_mr.fmr_map_cnt;
ret = ehca_reg_mr(shca, e_mr, iova_start, size, acl,
- e_pd, pginfo, lkey, rkey);
+ e_pd, pginfo, lkey, rkey);
if (ret) {
u32 offset = (u64)(&e_mr->flags) - (u64)e_mr;
memcpy(&e_mr->flags, &(save_mr.flags),
@@ -1355,6 +1438,7 @@ int ehca_unmap_one_fmr(struct ehca_shca *shca,
/* set some MR values */
e_fmr->flags = save_fmr.flags;
+ e_fmr->hwpage_size = save_fmr.hwpage_size;
e_fmr->fmr_page_size = save_fmr.fmr_page_size;
e_fmr->fmr_max_pages = save_fmr.fmr_max_pages;
e_fmr->fmr_max_maps = save_fmr.fmr_max_maps;
@@ -1363,8 +1447,6 @@ int ehca_unmap_one_fmr(struct ehca_shca *shca,
memset(&pginfo, 0, sizeof(pginfo));
pginfo.type = EHCA_MR_PGI_FMR;
- pginfo.num_kpages = 0;
- pginfo.num_hwpages = 0;
ret = ehca_reg_mr(shca, e_fmr, NULL,
(e_fmr->fmr_max_pages * e_fmr->fmr_page_size),
e_fmr->acl, e_pd, &pginfo, &tmp_lkey,
@@ -1373,7 +1455,6 @@ int ehca_unmap_one_fmr(struct ehca_shca *shca,
u32 offset = (u64)(&e_fmr->flags) - (u64)e_fmr;
memcpy(&e_fmr->flags, &(save_mr.flags),
sizeof(struct ehca_mr) - offset);
- goto ehca_unmap_one_fmr_exit0;
}
ehca_unmap_one_fmr_exit0:
@@ -1401,7 +1482,7 @@ int ehca_reg_smr(struct ehca_shca *shca,
struct ehca_mr_hipzout_parms hipzout;
ehca_mrmw_map_acl(acl, &hipz_acl);
- ehca_mrmw_set_pgsize_hipz_acl(&hipz_acl);
+ ehca_mrmw_set_pgsize_hipz_acl(e_origmr->hwpage_size, &hipz_acl);
h_ret = hipz_h_register_smr(shca->ipz_hca_handle, e_newmr, e_origmr,
(u64)iova_start, hipz_acl, e_pd->fw_pd,
@@ -1420,6 +1501,7 @@ int ehca_reg_smr(struct ehca_shca *shca,
/* successful registration */
e_newmr->num_kpages = e_origmr->num_kpages;
e_newmr->num_hwpages = e_origmr->num_hwpages;
+ e_newmr->hwpage_size = e_origmr->hwpage_size;
e_newmr->start = iova_start;
e_newmr->size = e_origmr->size;
e_newmr->acl = acl;
@@ -1452,6 +1534,7 @@ int ehca_reg_internal_maxmr(
struct ib_phys_buf ib_pbuf;
u32 num_kpages;
u32 num_hwpages;
+ u64 hw_pgsize;
e_mr = ehca_mr_new();
if (!e_mr) {
@@ -1468,13 +1551,15 @@ int ehca_reg_internal_maxmr(
ib_pbuf.size = size_maxmr;
num_kpages = NUM_CHUNKS(((u64)iova_start % PAGE_SIZE) + size_maxmr,
PAGE_SIZE);
- num_hwpages = NUM_CHUNKS(((u64)iova_start % EHCA_PAGESIZE) + size_maxmr,
- EHCA_PAGESIZE);
+ hw_pgsize = ehca_get_max_hwpage_size(shca);
+ num_hwpages = NUM_CHUNKS(((u64)iova_start % hw_pgsize) + size_maxmr,
+ hw_pgsize);
memset(&pginfo, 0, sizeof(pginfo));
pginfo.type = EHCA_MR_PGI_PHYS;
pginfo.num_kpages = num_kpages;
pginfo.num_hwpages = num_hwpages;
+ pginfo.hwpage_size = hw_pgsize;
pginfo.u.phy.num_phys_buf = 1;
pginfo.u.phy.phys_buf_array = &ib_pbuf;
@@ -1523,7 +1608,7 @@ int ehca_reg_maxmr(struct ehca_shca *shca,
struct ehca_mr_hipzout_parms hipzout;
ehca_mrmw_map_acl(acl, &hipz_acl);
- ehca_mrmw_set_pgsize_hipz_acl(&hipz_acl);
+ ehca_mrmw_set_pgsize_hipz_acl(e_origmr->hwpage_size, &hipz_acl);
h_ret = hipz_h_register_smr(shca->ipz_hca_handle, e_newmr, e_origmr,
(u64)iova_start, hipz_acl, e_pd->fw_pd,
@@ -1539,6 +1624,7 @@ int ehca_reg_maxmr(struct ehca_shca *shca,
/* successful registration */
e_newmr->num_kpages = e_origmr->num_kpages;
e_newmr->num_hwpages = e_origmr->num_hwpages;
+ e_newmr->hwpage_size = e_origmr->hwpage_size;
e_newmr->start = iova_start;
e_newmr->size = e_origmr->size;
e_newmr->acl = acl;
@@ -1684,6 +1770,7 @@ static int ehca_set_pagebuf_user1(struct ehca_mr_pginfo *pginfo,
u64 pgaddr;
u32 i = 0;
u32 j = 0;
+ int hwpages_per_kpage = PAGE_SIZE / pginfo->hwpage_size;
/* loop over desired chunk entries */
chunk = pginfo->u.usr.next_chunk;
@@ -1695,7 +1782,7 @@ static int ehca_set_pagebuf_user1(struct ehca_mr_pginfo *pginfo,
<< PAGE_SHIFT ;
*kpage = phys_to_abs(pgaddr +
(pginfo->next_hwpage *
- EHCA_PAGESIZE));
+ pginfo->hwpage_size));
if ( !(*kpage) ) {
ehca_gen_err("pgaddr=%lx "
"chunk->page_list[i]=%lx "
@@ -1708,8 +1795,7 @@ static int ehca_set_pagebuf_user1(struct ehca_mr_pginfo *pginfo,
(pginfo->hwpage_cnt)++;
(pginfo->next_hwpage)++;
kpage++;
- if (pginfo->next_hwpage %
- (PAGE_SIZE / EHCA_PAGESIZE) == 0) {
+ if (pginfo->next_hwpage % hwpages_per_kpage == 0) {
(pginfo->kpage_cnt)++;
(pginfo->u.usr.next_nmap)++;
pginfo->next_hwpage = 0;
@@ -1738,6 +1824,143 @@ static int ehca_set_pagebuf_user1(struct ehca_mr_pginfo *pginfo,
return ret;
}
+/*
+ * check given pages for contiguous layout
+ * last page addr is returned in prev_pgaddr for further check
+ */
+static int ehca_check_kpages_per_ate(struct scatterlist *page_list,
+ int start_idx, int end_idx,
+ u64 *prev_pgaddr)
+{
+ int t;
+ for (t = start_idx; t <= end_idx; t++) {
+ u64 pgaddr = page_to_pfn(page_list[t].page) << PAGE_SHIFT;
+ ehca_gen_dbg("chunk_page=%lx value=%016lx", pgaddr,
+ *(u64 *)abs_to_virt(phys_to_abs(pgaddr)));
+ if (pgaddr - PAGE_SIZE != *prev_pgaddr) {
+ ehca_gen_err("uncontiguous page found pgaddr=%lx "
+ "prev_pgaddr=%lx page_list_i=%x",
+ pgaddr, *prev_pgaddr, t);
+ return -EINVAL;
+ }
+ *prev_pgaddr = pgaddr;
+ }
+ return 0;
+}
+
+/* PAGE_SIZE < pginfo->hwpage_size */
+static int ehca_set_pagebuf_user2(struct ehca_mr_pginfo *pginfo,
+ u32 number,
+ u64 *kpage)
+{
+ int ret = 0;
+ struct ib_umem_chunk *prev_chunk;
+ struct ib_umem_chunk *chunk;
+ u64 pgaddr, prev_pgaddr;
+ u32 i = 0;
+ u32 j = 0;
+ int kpages_per_hwpage = pginfo->hwpage_size / PAGE_SIZE;
+ int nr_kpages = kpages_per_hwpage;
+
+ /* loop over desired chunk entries */
+ chunk = pginfo->u.usr.next_chunk;
+ prev_chunk = pginfo->u.usr.next_chunk;
+ list_for_each_entry_continue(
+ chunk, (&(pginfo->u.usr.region->chunk_list)), list) {
+ for (i = pginfo->u.usr.next_nmap; i < chunk->nmap; ) {
+ if (nr_kpages == kpages_per_hwpage) {
+ pgaddr = ( page_to_pfn(chunk->page_list[i].page)
+ << PAGE_SHIFT );
+ *kpage = phys_to_abs(pgaddr);
+ if ( !(*kpage) ) {
+ ehca_gen_err("pgaddr=%lx i=%x",
+ pgaddr, i);
+ ret = -EFAULT;
+ return ret;
+ }
+ /*
+ * The first page in a hwpage must be aligned;
+ * the first MR page is exempt from this rule.
+ */
+ if (pgaddr & (pginfo->hwpage_size - 1)) {
+ if (pginfo->hwpage_cnt) {
+ ehca_gen_err(
+ "invalid alignment "
+ "pgaddr=%lx i=%x "
+ "mr_pgsize=%lx",
+ pgaddr, i,
+ pginfo->hwpage_size);
+ ret = -EFAULT;
+ return ret;
+ }
+ /* first MR page */
+ pginfo->kpage_cnt =
+ (pgaddr &
+ (pginfo->hwpage_size - 1)) >>
+ PAGE_SHIFT;
+ nr_kpages -= pginfo->kpage_cnt;
+ *kpage = phys_to_abs(
+ pgaddr &
+ ~(pginfo->hwpage_size - 1));
+ }
+ ehca_gen_dbg("kpage=%lx chunk_page=%lx "
+ "value=%016lx", *kpage, pgaddr,
+ *(u64 *)abs_to_virt(
+ phys_to_abs(pgaddr)));
+ prev_pgaddr = pgaddr;
+ i++;
+ pginfo->kpage_cnt++;
+ pginfo->u.usr.next_nmap++;
+ nr_kpages--;
+ if (!nr_kpages)
+ goto next_kpage;
+ continue;
+ }
+ if (i + nr_kpages > chunk->nmap) {
+ ret = ehca_check_kpages_per_ate(
+ chunk->page_list, i,
+ chunk->nmap - 1, &prev_pgaddr);
+ if (ret) return ret;
+ pginfo->kpage_cnt += chunk->nmap - i;
+ pginfo->u.usr.next_nmap += chunk->nmap - i;
+ nr_kpages -= chunk->nmap - i;
+ break;
+ }
+
+ ret = ehca_check_kpages_per_ate(chunk->page_list, i,
+ i + nr_kpages - 1,
+ &prev_pgaddr);
+ if (ret) return ret;
+ i += nr_kpages;
+ pginfo->kpage_cnt += nr_kpages;
+ pginfo->u.usr.next_nmap += nr_kpages;
+next_kpage:
+ nr_kpages = kpages_per_hwpage;
+ (pginfo->hwpage_cnt)++;
+ kpage++;
+ j++;
+ if (j >= number) break;
+ }
+ if ((pginfo->u.usr.next_nmap >= chunk->nmap) &&
+ (j >= number)) {
+ pginfo->u.usr.next_nmap = 0;
+ prev_chunk = chunk;
+ break;
+ } else if (pginfo->u.usr.next_nmap >= chunk->nmap) {
+ pginfo->u.usr.next_nmap = 0;
+ prev_chunk = chunk;
+ } else if (j >= number)
+ break;
+ else
+ prev_chunk = chunk;
+ }
+ pginfo->u.usr.next_chunk =
+ list_prepare_entry(prev_chunk,
+ (&(pginfo->u.usr.region->chunk_list)),
+ list);
+ return ret;
+}
+
int ehca_set_pagebuf_phys(struct ehca_mr_pginfo *pginfo,
u32 number,
u64 *kpage)
@@ -1750,9 +1973,10 @@ int ehca_set_pagebuf_phys(struct ehca_mr_pginfo *pginfo,
/* loop over desired phys_buf_array entries */
while (i < number) {
pbuf = pginfo->u.phy.phys_buf_array + pginfo->u.phy.next_buf;
- num_hw = NUM_CHUNKS((pbuf->addr % EHCA_PAGESIZE) +
- pbuf->size, EHCA_PAGESIZE);
- offs_hw = (pbuf->addr & ~PAGE_MASK) / EHCA_PAGESIZE;
+ num_hw = NUM_CHUNKS((pbuf->addr % pginfo->hwpage_size) +
+ pbuf->size, pginfo->hwpage_size);
+ offs_hw = (pbuf->addr & ~(pginfo->hwpage_size - 1)) /
+ pginfo->hwpage_size;
while (pginfo->next_hwpage < offs_hw + num_hw) {
/* sanity check */
if ((pginfo->kpage_cnt >= pginfo->num_kpages) ||
@@ -1768,21 +1992,23 @@ int ehca_set_pagebuf_phys(struct ehca_mr_pginfo *pginfo,
return -EFAULT;
}
*kpage = phys_to_abs(
- (pbuf->addr & EHCA_PAGEMASK)
- + (pginfo->next_hwpage * EHCA_PAGESIZE));
+ (pbuf->addr & ~(pginfo->hwpage_size - 1)) +
+ (pginfo->next_hwpage * pginfo->hwpage_size));
if ( !(*kpage) && pbuf->addr ) {
- ehca_gen_err("pbuf->addr=%lx "
- "pbuf->size=%lx "
+ ehca_gen_err("pbuf->addr=%lx pbuf->size=%lx "
"next_hwpage=%lx", pbuf->addr,
- pbuf->size,
- pginfo->next_hwpage);
+ pbuf->size, pginfo->next_hwpage);
return -EFAULT;
}
(pginfo->hwpage_cnt)++;
(pginfo->next_hwpage)++;
- if (pginfo->next_hwpage %
- (PAGE_SIZE / EHCA_PAGESIZE) == 0)
- (pginfo->kpage_cnt)++;
+ if (PAGE_SIZE >= pginfo->hwpage_size) {
+ if (pginfo->next_hwpage %
+ (PAGE_SIZE / pginfo->hwpage_size) == 0)
+ (pginfo->kpage_cnt)++;
+ } else
+ pginfo->kpage_cnt += pginfo->hwpage_size /
+ PAGE_SIZE;
kpage++;
i++;
if (i >= number) break;
@@ -1806,8 +2032,8 @@ int ehca_set_pagebuf_fmr(struct ehca_mr_pginfo *pginfo,
/* loop over desired page_list entries */
fmrlist = pginfo->u.fmr.page_list + pginfo->u.fmr.next_listelem;
for (i = 0; i < number; i++) {
- *kpage = phys_to_abs((*fmrlist & EHCA_PAGEMASK) +
- pginfo->next_hwpage * EHCA_PAGESIZE);
+ *kpage = phys_to_abs((*fmrlist & ~(pginfo->hwpage_size - 1)) +
+ pginfo->next_hwpage * pginfo->hwpage_size);
if ( !(*kpage) ) {
ehca_gen_err("*fmrlist=%lx fmrlist=%p "
"next_listelem=%lx next_hwpage=%lx",
@@ -1817,15 +2043,38 @@ int ehca_set_pagebuf_fmr(struct ehca_mr_pginfo *pginfo,
return -EFAULT;
}
(pginfo->hwpage_cnt)++;
- (pginfo->next_hwpage)++;
- kpage++;
- if (pginfo->next_hwpage %
- (pginfo->u.fmr.fmr_pgsize / EHCA_PAGESIZE) == 0) {
- (pginfo->kpage_cnt)++;
- (pginfo->u.fmr.next_listelem)++;
- fmrlist++;
- pginfo->next_hwpage = 0;
+ if (pginfo->u.fmr.fmr_pgsize >= pginfo->hwpage_size) {
+ if (pginfo->next_hwpage %
+ (pginfo->u.fmr.fmr_pgsize /
+ pginfo->hwpage_size) == 0) {
+ (pginfo->kpage_cnt)++;
+ (pginfo->u.fmr.next_listelem)++;
+ fmrlist++;
+ pginfo->next_hwpage = 0;
+ } else
+ (pginfo->next_hwpage)++;
+ } else {
+ unsigned int cnt_per_hwpage = pginfo->hwpage_size /
+ pginfo->u.fmr.fmr_pgsize;
+ unsigned int j;
+ u64 prev = *kpage;
+ /* check if adrs are contiguous */
+ for (j = 1; j < cnt_per_hwpage; j++) {
+ u64 p = phys_to_abs(fmrlist[j] &
+ ~(pginfo->hwpage_size - 1));
+ if (prev + pginfo->u.fmr.fmr_pgsize != p) {
+ ehca_gen_err("uncontiguous fmr pages "
+ "found prev=%lx p=%lx "
+ "idx=%x", prev, p, i + j);
+ return -EINVAL;
+ }
+ prev = p;
+ }
+ pginfo->kpage_cnt += cnt_per_hwpage;
+ pginfo->u.fmr.next_listelem += cnt_per_hwpage;
+ fmrlist += cnt_per_hwpage;
}
+ kpage++;
}
return ret;
}
@@ -1842,7 +2091,9 @@ int ehca_set_pagebuf(struct ehca_mr_pginfo *pginfo,
ret = ehca_set_pagebuf_phys(pginfo, number, kpage);
break;
case EHCA_MR_PGI_USER:
- ret = ehca_set_pagebuf_user1(pginfo, number, kpage);
+ ret = PAGE_SIZE >= pginfo->hwpage_size ?
+ ehca_set_pagebuf_user1(pginfo, number, kpage) :
+ ehca_set_pagebuf_user2(pginfo, number, kpage);
break;
case EHCA_MR_PGI_FMR:
ret = ehca_set_pagebuf_fmr(pginfo, number, kpage);
@@ -1895,9 +2146,9 @@ void ehca_mrmw_map_acl(int ib_acl,
/*----------------------------------------------------------------------*/
/* sets page size in hipz access control for MR/MW. */
-void ehca_mrmw_set_pgsize_hipz_acl(u32 *hipz_acl) /*INOUT*/
+void ehca_mrmw_set_pgsize_hipz_acl(u32 pgsize, u32 *hipz_acl) /*INOUT*/
{
- return; /* HCA supports only 4k */
+ *hipz_acl |= (ehca_encode_hwpage_size(pgsize) << 24);
} /* end ehca_mrmw_set_pgsize_hipz_acl() */
/*----------------------------------------------------------------------*/
diff --git a/drivers/infiniband/hw/ehca/ehca_mrmw.h b/drivers/infiniband/hw/ehca/ehca_mrmw.h
index 24f13fe..bc8f4e3 100644
--- a/drivers/infiniband/hw/ehca/ehca_mrmw.h
+++ b/drivers/infiniband/hw/ehca/ehca_mrmw.h
@@ -111,7 +111,7 @@ int ehca_mr_is_maxmr(u64 size,
void ehca_mrmw_map_acl(int ib_acl,
u32 *hipz_acl);
-void ehca_mrmw_set_pgsize_hipz_acl(u32 *hipz_acl);
+void ehca_mrmw_set_pgsize_hipz_acl(u32 pgsize, u32 *hipz_acl);
void ehca_mrmw_reverse_map_acl(const u32 *hipz_acl,
int *ib_acl);
diff --git a/drivers/infiniband/hw/ehca/hcp_if.c b/drivers/infiniband/hw/ehca/hcp_if.c
index 3394e05..358796c 100644
--- a/drivers/infiniband/hw/ehca/hcp_if.c
+++ b/drivers/infiniband/hw/ehca/hcp_if.c
@@ -427,7 +427,8 @@ u64 hipz_h_register_rpage(const struct ipz_adapter_handle adapter_handle,
{
return ehca_plpar_hcall_norets(H_REGISTER_RPAGES,
adapter_handle.handle, /* r4 */
- queue_type | pagesize << 8, /* r5 */
+ (u64)queue_type | ((u64)pagesize) << 8,
+ /* r5 */
resource_handle, /* r6 */
logical_address_of_page, /* r7 */
count, /* r8 */
@@ -724,6 +725,9 @@ u64 hipz_h_alloc_resource_mr(const struct ipz_adapter_handle adapter_handle,
u64 ret;
u64 outs[PLPAR_HCALL9_BUFSIZE];
+ ehca_gen_dbg("kernel PAGE_SIZE=%x access_ctrl=%016x "
+ "vaddr=%lx length=%lx",
+ (u32)PAGE_SIZE, access_ctrl, vaddr, length);
ret = ehca_plpar_hcall9(H_ALLOC_RESOURCE, outs,
adapter_handle.handle, /* r4 */
5, /* r5 */
@@ -746,8 +750,22 @@ u64 hipz_h_register_rpage_mr(const struct ipz_adapter_handle adapter_handle,
const u64 logical_address_of_page,
const u64 count)
{
+ extern int ehca_debug_level;
u64 ret;
+ if (unlikely(ehca_debug_level >= 2)) {
+ if (count > 1) {
+ u64 *kpage;
+ int i;
+ kpage = (u64 *)abs_to_virt(logical_address_of_page);
+ for (i = 0; i < count; i++)
+ ehca_gen_dbg("kpage[%d]=%p",
+ i, (void *)kpage[i]);
+ } else
+ ehca_gen_dbg("kpage=%p",
+ (void *)logical_address_of_page);
+ }
+
if ((count > 1) && (logical_address_of_page & (EHCA_PAGESIZE-1))) {
ehca_gen_err("logical_address_of_page not on a 4k boundary "
"adapter_handle=%lx mr=%p mr_handle=%lx "
--
1.5.2
^ permalink raw reply related
* [PATCH 0/5] ehca: MR large page, small queue and fixes
From: Hoang-Nam Nguyen @ 2007-07-20 13:48 UTC (permalink / raw)
To: rolandd, linux-kernel, linuxppc-dev, openib-general
Cc: fenkes, raisch, stefan.roscher
Here is a patch set against Roland's git, branch for-2.6.23 for ehca.
It adds support for MR large page and small queues. In addition of that
it also contains various small fixes from previous comments and what
we found.
They are in details:
[1/5] adds support for MR large page
[2/5] generates event when SRQ limit reached
[3/5] makes ehca2ib_return_code() non inline
[4/5] makes internal_create/destroy_qp() static
[5/5] adds support for small queues
The patches should apply cleanly, in order, against Roland's git. Please
review the changes and apply the patches if they are okay.
Regards,
Nam & Stefan
^ permalink raw reply
* Re: Kmalloc returns which address
From: Alessandro Rubini @ 2007-07-20 12:13 UTC (permalink / raw)
To: linuxppc-embedded
In-Reply-To: <11705981.post@talk.nabble.com>
Hello.
> In linux kmelloc returns the pointer to virtual address not the physical
> address, to return to the physical address there is different function
> called ioremap
Not exactly. Both return virtual addresses. The kmalloc one is allocated
RAM memory, the ioremap is the virtual address built for you to
access a physical address you specified (i.e., a device memory area).
>> For Tx and Rx, hardware uses buffers, so I have to allocate buffers and
>> pass
>> the pointer to hardware. Can I pass the pointer returned kmalloc? or I
>> should convert it into physical address?
If you allocate with kmalloc, use __pa(addr) to turn the virtual
address addr into the physical address you need to pass to the device,
ad DMA access is outside of the virtual view of the address space.
However, please note that according to how your device is connected
and what bus it is using, the design can get more hairy, you may need
to study the consistent memory allocations.
Hope this helps
/alessandro
^ permalink raw reply
* RE: How to access physical memory from user space for MPC8260 chip
From: Misbah khan @ 2007-07-20 11:32 UTC (permalink / raw)
To: linuxppc-embedded
In-Reply-To: <0B45E93C5FF65740AEAE690BF3848B7A4AB178@rennsmail04.eu.thmulti.com>
Physical address you can map to kernel space using ioremap() function
but you are not clear whether you want to map it to the user space / kernel
space. To map kernel space to user space you should use mmap() functionality
in your driver.
I hope you got the answer to what you were expecting else send the clear
query
regard
misbah
Fillod Stephane wrote:
>
> suresh suresh wrote:
>>I have to map physical memory to user space or kernel space. I am
> writing >driver for MPC8260 chip and I want to know how to map any
> 32-bit address >space to user space and kernel space.
>
> Your question is a linuxppc-embedded FAQ. User-land access is documented
>
> in Denx's FAQ[1], and accessible through shorter URL[2]. For more
> information, please follow this thread[3] (not ppc specific actually).
>
> [1]
> http://www.denx.de/twiki/bin/view/PPCEmbedded/DeviceDrivers#Section_Acce
> ssingPeripheralsFromUserSpace
> [2] http://tinyurl.com/6c7th
> [3] http://article.gmane.org/gmane.linux.ports.ppc.embedded/5053
>
> In kernel land, ioremap() is all you need.
>
> Don't forget to use the 'eieio' asm instruction if you want explicit
> I/O ordering.
>
> Best Regards,
> --
> Stephane, the userland ioremap bot
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>
>
--
View this message in context: http://www.nabble.com/How-to-access-physical-memory-from-user-space-for-MPC8260-chip-tf4067159.html#a11706536
Sent from the linuxppc-embedded mailing list archive at Nabble.com.
^ permalink raw reply
* [PATCH] powerpc: mpc5200 low power mode, .globl fix
From: Domen Puncer @ 2007-07-20 10:55 UTC (permalink / raw)
To: Sylvain Munaut; +Cc: Linux PPC Mailing List, Paul Mackerras
In-Reply-To: <11847043512142-git-send-email-tnt@246tNt.com>
flush_data_cache doesn't need to be global, and can cause problems.
Thanks to Milton Miller for noticing this.
Signed-off-by: Domen Puncer <domen.puncer@telargo.com>
---
arch/powerpc/platforms/52xx/lite5200_sleep.S | 1 -
1 file changed, 1 deletion(-)
Index: work-powerpc.git/arch/powerpc/platforms/52xx/lite5200_sleep.S
===================================================================
--- work-powerpc.git.orig/arch/powerpc/platforms/52xx/lite5200_sleep.S
+++ work-powerpc.git/arch/powerpc/platforms/52xx/lite5200_sleep.S
@@ -400,7 +400,6 @@ restore_regs:
* Flush data cache
* Do this by just reading lots of stuff into the cache.
*/
- .globl flush_data_cache
flush_data_cache:
lis r3,CONFIG_KERNEL_START@h
ori r3,r3,CONFIG_KERNEL_START@l
^ permalink raw reply
* Re: Kmalloc returns which address
From: Misbah khan @ 2007-07-20 10:52 UTC (permalink / raw)
To: linuxppc-embedded
In-Reply-To: <e5eb93010707160659m6a7d0b02p623542739d28ab29@mail.gmail.com>
hi Suresh
In linux kmelloc returns the pointer to virtual address not the physical
address, to return to the physical address there is different function
called ioremap
for eg :-
char *buf_tx =kmalloc(100,GFP_KERNEL); // Tx buffer
char *buf_rx=kmalloc(100,GFP_KERNEL); // Rx buffer
ptr_tx=ioremap( buf_tx,100);
ptr_rx=ioremap(buf_rx,100);
To learn more about this go through Linux device driver by rubini
I hope this will work for you
regard
Misbah
suresh suresh wrote:
>
> Hi,
>
> I am porting MPC8280 driver from vxWorks to Linux.
>
> I want know the address return by kmalloc function? is it physical address
> or kernel virtual address.
>
> For Tx and Rx, hardware uses buffers, so I have to allocate buffers and
> pass
> the pointer to hardware. Can I pass the pointer returned kmalloc? or I
> should convert it into physical address?
>
> If it returns kernel virtual address, then how to convert into physical?
>
> Thanks & Regards-
> Suresh
>
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
>
--
View this message in context: http://www.nabble.com/Kmalloc-returns-which-address-tf4086826.html#a11705981
Sent from the linuxppc-embedded mailing list archive at Nabble.com.
^ permalink raw reply
* Problem faced while using workqueue in the character driver.
From: Misbah khan @ 2007-07-20 10:40 UTC (permalink / raw)
To: linuxppc-embedded
Hi all,
I am working on a character driver for FPGA, in which i am using a blocked
read call on workqueue. The read call will be unblocked by the Interrupt
from the Fpga to PPC Cpu.
The problem is that if the process is in blocked mode and then an Interrupt
occurs the system gives kernel Panic where as it get unblocked and start
reading the data but very soon it gets crashed.
Please send me your suggessins regarding the mentioned problem.
Thanks in advance
with regard
Misbah
--
View this message in context: http://www.nabble.com/Problem-faced-while-using-workqueue-in-the-character-driver.-tf4116327.html#a11705912
Sent from the linuxppc-embedded mailing list archive at Nabble.com.
^ permalink raw reply
* RE: ML403 Hard TEMAC, PLB and Linux 2.6
From: Mohammad Sadegh Sadri @ 2007-07-20 9:27 UTC (permalink / raw)
To: Rick Moleres, linux-ppc; +Cc: linuxppc-embedded
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="gb2312", Size: 10777 bytes --]
Dear Massimiliano
Well I do not know the solution to the problem, but just,
In your post, there is some thing interesting for me,
How could you implement such large circuit on one FX12 FPGA?
We have done many experineces with both of AVNET MM and ML403,
you have, DDR SDRAM Controller, GPIO, UART, SYSACE and two intances of Tri-mode TEMAC GMII,
In our tests, with one instance of Tri mode TEMAC and UART and DDR controller, the whole FPGA resource were consumed we could not even put sysace interface.
For me, another important question is, how many PHY chips are available on your board? Each of Tri-mode temac modules are connected to a separate phy chip, yes?
________________________________
> Subject: RE: ML403 Hard TEMAC, PLB and Linux 2.6
> Date: Thu, 19 Jul 2007 09:00:33 -0600
> From: Rick.Moleres@xilinx.com
> To: linux-ppc@eurotel.it
> CC: eemingliu@hotmail.com; linuxppc-embedded@ozlabs.org; mamsadegh@hotmail.com
>
> Hi,
> The first thing I would try is to make sure eth1 comes up without first bringing up eth0. In other words, does it come up independently? If not, there could be something wrong with its configuration in the kernel or perhaps a hardware design issue. If both interfaces come up independently, but not together, then there¡¯s likely a driver issue as we have not tested dual channel TEMAC with the Linux plb_temac driver. Perhaps others on the list have. The areas that I would look at would be: make sure each interface is getting a unique and correct PHY address; make sure any calls to the shared registers of the two channels are protected with semaphores/spinlocks. For example, I¡¯m pretty sure the PHY registers are shared, so any PHY accesses should be protected. I would think other Xilinx driver accesses like Start/Stop/Reset/Get or SetOptions/etc¡ should be protected as they may access shared registers.
> -Rick
> ________________________________
> From: Massimiliano.Patriarca@eurotel.it [mailto:Massimiliano.Patriarca@eurotel.it] On Behalf Of linux-ppc@eurotel.it
> Sent: Thursday, July 19, 2007 7:51 AM
> To: Rick Moleres
> Cc: Ming Liu; linuxppc-embedded@ozlabs.org; linuxppc-embedded-bounces+linux-ppc=eurotel.it@ozlabs.org; mamsadegh@hotmail.com
> Subject: RE: ML403 Hard TEMAC, PLB and Linux 2.6
> Hi, we are trying to use both side of an hard temac + 2 plb temac in a Virtex4FX12 project.
> we succesfull implemented a single temac Montavista tree with eth0.
> when trying to use both temac, devices are correctly registered with kernel-
> eth0 comes up and working ok-
> when manually start eth1 (/sbin/ifconfig eth1 up) kernel hang without any error or info in the console
> # dmesg
> Linux version 2.6.10_mvl401-ml40x (massimiliano@linux-yhbz) (gcc version 3.4.3 (MontaVista 3.4.3-25.0.135.0702900 2007-04-01)) #250 Wed Jul 18 12:20:43 CEST 2007
> Eurotel motherboard init
> Port by MontaVista Software, Inc. (source@mvista.com)
> On node 0 totalpages: 7812
> DMA zone: 7812 pages, LIFO batch:1
> Normal zone: 0 pages, LIFO batch:1
> HighMem zone: 0 pages, LIFO batch:1
> Built 1 zonelists
> Kernel command line: console=ttl0 root=/dev/xsysace2 rw ip=off
> Xilinx INTC #0 at 0x41200000 mapped to 0xFDFFF000
> PID hash table entries: 128 (order: 7, 2048 bytes)
> hr_time_init: arch_to_nsec = 8192000, nsec_to_arch = 1099511627
> Console: colour dummy device 80x25
> Dentry cache hash table entries: 4096 (order: 2, 16384 bytes)
> Inode-cache hash table entries: 2048 (order: 1, 8192 bytes)
> Memory: 28420k available (1864k kernel code, 528k data, 124k init, 0k highmem)
> Calibrating delay loop... 252.92 BogoMIPS (lpj=126464)
> Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
> spawn_desched_task(00000000)
> desched cpu_callback 3/00000000
> ksoftirqd started up.
> desched cpu_callback 2/00000000
> desched thread 0 started up.
> NET: Registered protocol family 16
> Registering platform device 'xilinx_temac.0'. Parent at platform
> Registering platform device 'xilinx_temac.1'. Parent at platform
> Registering platform device 'xilinx_sysace.0'. Parent at platform
> Registering platform device 'xilinx_gpio.0'. Parent at platform
> Registering platform device 'xilinx_gpio.1'. Parent at platform
> Registering platform device 'oled_fb.0'. Parent at platform
> Generic PHY: Registered new driver
> oled_fb: 4096 video memory mapped to c2011000
> Console: switching to colour frame buffer device 16x8
> xgpio #0 at 0x40000000 mapped to 0xC2020000
> xgpio #1 at 0x40020000 mapped to 0xC2040000
> io scheduler noop registered
> io scheduler anticipatory registered
> io scheduler deadline registered
> io scheduler cfq registered
> loop: loaded (max 8 devices)
> elevator: using anticipatory as default io scheduler
> System ACE at 0x41800000 mapped to 0xC2060000, irq=4, 1000944KB
> xsysace: xsysace1 xsysace2
> Universal TUN/TAP device driver 1.5 (C)1999-2002 Maxim Krasnyansky
> XTemac: using FIFO direct interrupt driven mode.
> eth0: Xilinx TEMAC #0 at 0x20000000 mapped to 0xC2080000, irq=0
> eth0: XTemac id 1.0f, block id 5, type 8
> XTemac: using FIFO direct interrupt driven mode.
> eth1: Xilinx TEMAC #1 at 0x20010000 mapped to 0xC20A0000, irq=10
> eth1: XTemac id 1.0f, block id 5, type 8
> i2c /dev entries driver
> i2c-xil_custom: registered with I2C (0)
> i2c-xil_custom: registered with I2C (1)
> mice: PS/2 mouse device common for all mice
> NET: Registered protocol family 2
> IP: routing cache hash table of 512 buckets, 4Kbytes
> TCP: Hash tables configured (established 2048 bind 4096)
> NET: Registered protocol family 1
> NET: Registered protocol family 17
> EXT3-fs warning: maximal mount count reached, running e2fsck is recommended
> kjournald starting. Commit interval 5 seconds
> EXT3 FS on xsysace2, internal journal
> EXT3-fs: recovery complete.
> EXT3-fs: mounted filesystem with ordered data mode.
> VFS: Mounted root (ext3 filesystem).
> Freeing unused kernel memory: 124k init
> eth0: XTemac: Options: 0xb8f2
> eth0: XTemac: speed set to 100Mb/s
> eth0: XTemac: PHY Link carrier lost.
> # /sbin/ifconfig eth1 up
> # eth1: XTemac: Options: 0xb8f2
> Any suggestion?
> "Rick Moleres"
> Sent by: linuxppc-embedded-bounces+linux-ppc=eurotel.it@ozlabs.org
> 08/02/2007 17.23
> To
> "Ming Liu" ,
> cc
> linuxppc-embedded@ozlabs.org
> Subject
> RE: ML403 Hard TEMAC, PLB and Linux 2.6
> Hi,
> As Ming says the Linux 2.6 TEMAC driver is made available in EDK 8.2.2 as part of the BSP and Libraries generation for Linux 2.6. Note that we made a recent fix for better PHY address detection and speed negotiation. I've attached the adapter here, and it will be available in EDK 9.1.1 when that's released.
> Thanks,
> Rick
> -----Original Message-----
> From: linuxppc-embedded-bounces+moleres=xilinx.com@ozlabs.org [mailto:linuxppc-embedded-bounces+moleres=xilinx.com@ozlabs.org] On Behalf Of Ming Liu
> Sent: Thursday, February 08, 2007 2:31 AM
> To: mamsadegh@hotmail.com
> Cc: linuxppc-embedded@ozlabs.org
> Subject: RE: ML403 Hard TEMAC, PLB and Linux 2.6
> Hi,
> In xapp902, they are using the old cores for Temac. So it will be easier to
> generate a project in EDK 8.2. Not only the cores there are new, but also
> it can generate the driver for linux 2.6.
> BR
> Ming
> >From: Mohammad Sadegh Sadri
> >To:
> >CC: linuxppc-embedded@ozlabs.org
> >Subject: RE: ML403 Hard TEMAC, PLB and Linux 2.6
> >Date: Thu, 8 Feb 2007 07:13:47 +0000
> >
> >
> >Hi
> >Thanks for reply
> >Well, regarding xapp902, there is a very simple question, Where can I find
> it? It seems that Xilinx has deleted all of the links to these files.
> >
> >
> >
> >
> >
> >----------------------------------------
> > > Subject: Re: ML403 Hard TEMAC, PLB and Linux 2.6
> > > From: christophe.alayrac@cresitt.com
> > > To: mamsadegh@hotmail.com
> > > CC: linuxppc-embedded@ozlabs.org
> > > Date: Thu, 8 Feb 2007 06:51:45 +0100
> > >
> > > Le mercredi 07 février 2007 ?22:30 +0000, Mohammad Sadegh Sadri a
> > > écrit :
> > > Hi Mohammad,
> > >
> > > Please find here after some answer
> > >
> > > > 1- I want to use, ML403, Hard TEMAC and PLB TEMAC cores. Is there any
> ready to use design, or any reference design available for this?
> > > > I'm using EDK 8.1 and I understood that, when I choose the ML403
> board , EDK does not use hard temac, so , should I add it manually? ( well
> this is funny, but the wizard does not use hard temac , is it true? )
> > > > Is there any ready EDK projects for ML403, with TEMAC and PLB TEMAC
> modules already inserted and cofigured? (I don't want to use GSRD ) ( If
> yes would you please put the link here )
> > > >
> > > You can start from XAPP 902 from Xilinx, this design demonstrate TEMAC
> > > use in loopback mode. Some memers from that community have tried from
> > > that design as a starting point. I did not nknow if the succeed.
> > > I would recommand to get EDK 8.2 SP2 and use the Wizard to build a new
> > > design with TEMAC.
> > >
> > > > 2- Simply, Is there any driver available for linux 2.6 , for PLB
> TEMAC and Hard TEMAC modules? If yes , can you put the link here, so that I
> can download it?
> > > >
> > > For the kernel you can get it from Montavista source code site using
> GIT
> > > to download it (linux-xilinx-26). This is 2.6.17.4 version (last week!)
> > >
> > > Then you will need to pacth the Kernel with TEMAC drivers (look for
> > > TEMAC PAULUS MVISTA with google, or follow my messages in that mailing
> > > list)
> > > NOTE: If you use that TEMAC patch do not use SGDMA on your TEMAC, use
> > > FIFO.
> > >
> > > > thanks
> > > Regards
> > >
> > > Chris
> > > > _________________________________________________________________
> > > > Personalize your Live.com homepage with the news, weather, and photos
> you care about.
> > > > http://www.live.com/getstarted.aspx?icid=T001MSN30A0701
> > > > _______________________________________________
> > > > Linuxppc-embedded mailing list
> > > > Linuxppc-embedded@ozlabs.org
> > > > https://ozlabs.org/mailman/listinfo/linuxppc-embedded
> > >
> >
> >_________________________________________________________________
> >Live Search: New search found
> >http://get.live.com/search/overview
> >_______________________________________________
> >Linuxppc-embedded mailing list
> >Linuxppc-embedded@ozlabs.org
> >https://ozlabs.org/mailman/listinfo/linuxppc-embedded
> _________________________________________________________________
> ÓëÁª»úµÄÅóÓѽøÐн»Á÷£¬ÇëʹÓà MSN Messenger: http://messenger.msn.com/cn
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded
_________________________________________________________________
Discover the new Windows Vista
http://search.msn.com/results.aspx?q=windows+vista&mkt=en-US&form=QBRE
^ permalink raw reply
* [PATCH] ucc_geth_mii: fix __exit called from __init
From: Domen Puncer @ 2007-07-20 8:19 UTC (permalink / raw)
To: leoli; +Cc: netdev, linuxppc-embedded
void __exit uec_mdio_exit(void) is called from
- static int __init ucc_geth_init(void)
- static void __exit ucc_geth_exit(void)
First one would make error path more than just an error.
Signed-off-by: Domen Puncer <domen.puncer@telargo.com>
---
drivers/net/ucc_geth_mii.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
Index: work-powerpc.git/drivers/net/ucc_geth_mii.c
===================================================================
--- work-powerpc.git.orig/drivers/net/ucc_geth_mii.c
+++ work-powerpc.git/drivers/net/ucc_geth_mii.c
@@ -272,7 +272,8 @@ int __init uec_mdio_init(void)
return of_register_platform_driver(&uec_mdio_driver);
}
-void __exit uec_mdio_exit(void)
+/* called from __init ucc_geth_init, therefore can not be __exit */
+void uec_mdio_exit(void)
{
of_unregister_platform_driver(&uec_mdio_driver);
}
^ permalink raw reply
* Re: Possible eHEA performance issue
From: Thomas Klein @ 2007-07-20 8:17 UTC (permalink / raw)
To: Michael Neuling
Cc: Thomas Klein, Jan-Bernd Themann, netdev, Stefan Roscher,
linux-ppc, Christoph Raisch, anton
In-Reply-To: <4634.1184900524@neuling.org>
Michael Neuling wrote:
> From ehea_start_xmit in ehea_main.c we have:
>
> if (unlikely(atomic_read(&pr->swqe_avail) <= 1)) {
> spin_lock_irqsave(&pr->netif_queue, flags);
> if (unlikely(atomic_read(&pr->swqe_avail) <= 1)) {
> pr->p_stats.queue_stopped++;
> netif_stop_queue(dev);
> pr->queue_stopped = 1;
> }
> spin_unlock_irqrestore(&pr->netif_queue, flags);
> }
>
> Since the conditions are the same, isn't it likely that the second 'if'
> is going to be taken. Hence, shouldn't the second 'unlikely' hint be
> removed or even changed to likely?
>
> Either way, some documentation here as to why it's done this way would
> be useful. I assume the atomic_read is cheap compared to the
> spin_unlock_irqsave, so we quickly check swqe_avail before we check it
> again properly with the lock on so we can change some stuff.
>
> Mikey
Hi Mike,
good point the second if could be a likely(). The impact isn't that big
because the if statement is true in the unlikely() case that the send queue
is full - which doesn't happen often. Anyway we will modify this in one of
the next driver versions. Thanks for the hint!
Thomas
^ permalink raw reply
* Re: [PATCH 2/2] fix showing xmon help
From: Ishizaki Kou @ 2007-07-20 8:15 UTC (permalink / raw)
To: miltonm; +Cc: paulus, linuxppc-dev
In-Reply-To: <664cbe49d76cfcba9d8a86e3a6946a2a@bga.com>
Milton Miller <miltonm@bga.com> wrote:
> On Jul 18, 2007, at 11:12 AM, Andreas Schwab wrote:
>
> > Milton Miller <miltonm@bga.com> writes:
> >
> >>> case '?':
> >>> - printf(help_string);
> >>> + xmon_puts(help_string);
> >>> break;
> >>>
> >>
> >> nonstdio.h #defines printf to xmon_printf. Please add a similar
line
> >> for puts, and use the define here. (It will avoid an unnecessary
> >> difference with the user space version).
> >
> > User space puts add a newline, which this xmon_puts doesn't.
> >
> > Andreas.
>
> Good point. This should be xmon_fputs #defined to fputs.
>
> milton
Should we change like below?
define fputs:
#define fputs(str,stream) xmon_fputs(str) /* stream is ignored. */
show the help string:
fputs(help_string, stdout);
Best regards,
Kou Ishizaki
^ permalink raw reply
* Re: [PATCH] of_detach_node()'s device node argument cannot be const
From: Segher Boessenkool @ 2007-07-20 7:57 UTC (permalink / raw)
To: Stephen Rothwell; +Cc: linuxppc-dev
In-Reply-To: <20070720161145.2ff6860c.sfr@canb.auug.org.au>
>> ...since it modifies it (when it sets the OF_DETACHED flag).
>
> http://patchwork.ozlabs.org/linuxppc/patch?q=Stephen%
> 20Rothwell&id=12212
Ah okay :-)
Segher
^ permalink raw reply
* Re: [PATCH] [updated] PHY fixed driver: rework release path and update phy_id notation
From: Andrew Morton @ 2007-07-20 7:57 UTC (permalink / raw)
To: Vitaly Bordug; +Cc: linuxppc-dev, linux-kernel, Jeff Garzik, netdev
In-Reply-To: <20070720115039.05f50f0f@localhost.localdomain>
On Fri, 20 Jul 2007 11:50:39 +0400 Vitaly Bordug <vitb@kernel.crashing.org> wrote:
> On Thu, 19 Jul 2007 23:23:37 -0700
> Andrew Morton wrote:
>
> > On Thu, 19 Jul 2007 03:38:04 +0400 Vitaly Bordug
> > <vitb@kernel.crashing.org> wrote:
> >
> > >
> > > device_bind_driver() error code returning has been fixed.
> > > release() function has been written, so that to free resources
> > > in correct way; the release path is now clean.
> > >
> > > Before the rework, it used to cause
> > > Device 'fixed@100:1' does not have a release() function, it is
> > > broken and must be fixed.
> > > BUG: at drivers/base/core.c:104 device_release()
> > >
> > > Call Trace:
> > > [<ffffffff802ec380>] kobject_cleanup+0x53/0x7e
> > > [<ffffffff802ec3ab>] kobject_release+0x0/0x9
> > > [<ffffffff802ecf3f>] kref_put+0x74/0x81
> > > [<ffffffff8035493b>] fixed_mdio_register_device+0x230/0x265
> > > [<ffffffff80564d31>] fixed_init+0x1f/0x35
> > > [<ffffffff802071a4>] init+0x147/0x2fb
> > > [<ffffffff80223b6e>] schedule_tail+0x36/0x92
> > > [<ffffffff8020a678>] child_rip+0xa/0x12
> > > [<ffffffff80311714>] acpi_ds_init_one_object+0x0/0x83
> > > [<ffffffff8020705d>] init+0x0/0x2fb
> > > [<ffffffff8020a66e>] child_rip+0x0/0x12
> > >
> > >
> > > Also changed the notation of the fixed phy definition on
> > > mdio bus to the form of <speed>+<duplex> to make it able to be used
> > > by gianfar and ucc_geth that define phy_id strictly as "%d:%d" and
> > > cleaned up the whitespace issues.
> > >
> >
> > Confused. Does the above refer to the difference between this patch
> > and the previous version, or does it just describe this patch?
> > Hopefully the latter, because the former isn't interesting, long-term.
> >
> Latter. IOW, that does mean, that mdio bus registered by this driver, now uses
> same naming conventioun that other PHYLIB things use. Hereby it will make it able to be used in
> NIC drivers other than fs_enet (and gianfar and ucc_geth are now points of interest).
>
> > If is _is_ a full standalone description of this patch then it's a
> > bit hard to follow ;)
> >
> Hmm -so what are my options - change the description and resubmit?
umm, I guess it's OK as-is. But it wasn't clear to me which sort of
changelog it was.
> > > +config FIXED_MII_1000_FDX
> > > + bool "Emulation for 1000M Fdx fixed PHY behavior"
> > > + depends on FIXED_PHY
> > > +
> > > +config FIXED_MII_AMNT
> > > + int "Number of emulated PHYs to allocate "
> > > + depends on FIXED_PHY
> > > + default "1"
> > > + ---help---
> > > + Sometimes it is required to have several independent
> > > emulated
> > > + PHYs on the bus (in case of multi-eth but phy-less HW for
> > > instance).
> > > + This control will have specified number allocated for each
> > > fixed
> > > + PHY type enabled.
> >
> > Shouldn't these be runtime options (ie: module parameters)?
> >
> I thought about it but this thing is more like the one that will never tend/required to change while\
> configured.. Will add if you see it appropriate though.
99% of users don't compile their own kernels: their vendor will have to
make this decision for them, and it sounds like any decision which they
make will be wrong for some of their users?
^ permalink raw reply
* Re: [PATCH] [updated] PHY fixed driver: rework release path and update phy_id notation
From: Vitaly Bordug @ 2007-07-20 7:50 UTC (permalink / raw)
To: Andrew Morton; +Cc: linuxppc-dev, linux-kernel, Jeff Garzik, netdev
In-Reply-To: <20070719232337.184942e0.akpm@linux-foundation.org>
On Thu, 19 Jul 2007 23:23:37 -0700
Andrew Morton wrote:
> On Thu, 19 Jul 2007 03:38:04 +0400 Vitaly Bordug
> <vitb@kernel.crashing.org> wrote:
>
> >
> > device_bind_driver() error code returning has been fixed.
> > release() function has been written, so that to free resources
> > in correct way; the release path is now clean.
> >
> > Before the rework, it used to cause
> > Device 'fixed@100:1' does not have a release() function, it is
> > broken and must be fixed.
> > BUG: at drivers/base/core.c:104 device_release()
> >
> > Call Trace:
> > [<ffffffff802ec380>] kobject_cleanup+0x53/0x7e
> > [<ffffffff802ec3ab>] kobject_release+0x0/0x9
> > [<ffffffff802ecf3f>] kref_put+0x74/0x81
> > [<ffffffff8035493b>] fixed_mdio_register_device+0x230/0x265
> > [<ffffffff80564d31>] fixed_init+0x1f/0x35
> > [<ffffffff802071a4>] init+0x147/0x2fb
> > [<ffffffff80223b6e>] schedule_tail+0x36/0x92
> > [<ffffffff8020a678>] child_rip+0xa/0x12
> > [<ffffffff80311714>] acpi_ds_init_one_object+0x0/0x83
> > [<ffffffff8020705d>] init+0x0/0x2fb
> > [<ffffffff8020a66e>] child_rip+0x0/0x12
> >
> >
> > Also changed the notation of the fixed phy definition on
> > mdio bus to the form of <speed>+<duplex> to make it able to be used
> > by gianfar and ucc_geth that define phy_id strictly as "%d:%d" and
> > cleaned up the whitespace issues.
> >
>
> Confused. Does the above refer to the difference between this patch
> and the previous version, or does it just describe this patch?
> Hopefully the latter, because the former isn't interesting, long-term.
>
Latter. IOW, that does mean, that mdio bus registered by this driver, now uses
same naming conventioun that other PHYLIB things use. Hereby it will make it able to be used in
NIC drivers other than fs_enet (and gianfar and ucc_geth are now points of interest).
> If is _is_ a full standalone description of this patch then it's a
> bit hard to follow ;)
>
Hmm -so what are my options - change the description and resubmit?
> > +config FIXED_MII_1000_FDX
> > + bool "Emulation for 1000M Fdx fixed PHY behavior"
> > + depends on FIXED_PHY
> > +
> > +config FIXED_MII_AMNT
> > + int "Number of emulated PHYs to allocate "
> > + depends on FIXED_PHY
> > + default "1"
> > + ---help---
> > + Sometimes it is required to have several independent
> > emulated
> > + PHYs on the bus (in case of multi-eth but phy-less HW for
> > instance).
> > + This control will have specified number allocated for each
> > fixed
> > + PHY type enabled.
>
> Shouldn't these be runtime options (ie: module parameters)?
>
I thought about it but this thing is more like the one that will never tend/required to change while\
configured.. Will add if you see it appropriate though.
> >
> > ...
> >
> > + * Private information hoder for mii_bus
>
> tpyo.
ok
--
Sincerely, Vitaly
^ permalink raw reply
* Re: [PATCH] Treat ISI faults as read faults on classic 32-bit PowerPC
From: Segher Boessenkool @ 2007-07-20 7:42 UTC (permalink / raw)
To: Paul Mackerras; +Cc: linuxppc-dev, Johannes Berg
In-Reply-To: <18079.63067.248965.356540@cargo.ozlabs.ibm.com>
>> Should you really be testing VM_READ|VM_WRITE, or should it just
>> be VM_READ?
>
> We test VM_READ | VM_WRITE | VM_EXEC in the read case below, and that
> is because we have no HPTE encoding to say "writable but not readable"
> or "executable but not readable". Similarly we have no encoding to
> say "writable but not executable" on classic processors, so if you
> have just VM_WRITE set, you get a page that is readable, writable and
> executable.
Ah yes. I thought "executable requires readable", but
that is with the CPU its flags, not the Linux flags.
Would it be a good idea to map Linux flags to CPU flags
somewhere early in this function? It might simplify some
code, and things certainly would become more readable.
Segher
^ permalink raw reply
* Re: [PATCH] powerpc: Add of_register_i2c_devices()
From: Segher Boessenkool @ 2007-07-20 7:26 UTC (permalink / raw)
To: Guennadi Liakhovetski; +Cc: linuxppc-dev, Paul Mackerras
In-Reply-To: <Pine.LNX.4.60.0707191932300.3365@poirot.grange>
>> You might want to put vendor names in the "compatible"
>> entries, dunno though, maybe these are "cross-vendor"
>> ICs?
>
> You mean like
>
> compatible = "ricoh,rs5c372a"
Yeah, like that. I'm not sure it is needed for these,
but it won't hurt either.
> + strncpy(info->driver_name, i2c_devices[i].i2c_driver,
> KOBJ_NAME_LEN);
> + strncpy(info->type, i2c_devices[i].i2c_type, I2C_NAME_SIZE);
Why not just strcpy(), btw?
> + addr = of_get_property(node, "reg", &len);
> + if (!addr || len < sizeof(int) || *addr > 0xffff)
> + continue;
Give a warning when the addr won't fit in 16 bits?
Segher
^ permalink raw reply
* Re: [PATCH] Add StorCenter DTS first draft.
From: Segher Boessenkool @ 2007-07-20 7:14 UTC (permalink / raw)
To: Benjamin Herrenschmidt; +Cc: linuxppc-dev, Jon Loeliger
In-Reply-To: <1184881198.25235.347.camel@localhost.localdomain>
>> So your plan is to prefer the device tree over the cputable, and
>> maybe even deprecate the cputable?
>
> Nah, you are fine not putting anything, but if it's in the DT it
> should
> be correct, not "0". Else, just don't put it in the DT.
Ah okay. So we should recommend to not put any of those
properties into DTS files at all? If so, I'll create a
patch to remove them.
> We use the DT to override cputable but that's mostly useful for things
> like pSeries where you can get some processors in "compatibility" mode
> over another one, or some virtual PVRs, and other fancy things like
> that
> where the cputable isn't very useful.
Yeah, and it's a good idea to have the device tree override
the cputable always, with real OF, anyway. (Well there are
buggy device trees out there, of course ;-) )
Segher
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox