* [PATCH] ehca: map 4k firmware context of cq, qp to user space
@ 2007-08-02 9:39 Hoang-Nam Nguyen
2007-08-02 14:08 ` Arnd Bergmann
0 siblings, 1 reply; 6+ messages in thread
From: Hoang-Nam Nguyen @ 2007-08-02 9:39 UTC (permalink / raw)
To: Roland Dreier, linux-kernel, linuxppc-dev, general; +Cc: raisch, paulus
From: Hoang-Nam Nguyen <hnguyen at de.ibm.com>
Date: Thu, 2 Aug 2007 10:08:30 +0200
Subject: [PATCH] ehca: map 4k firmware context of cq, qp to user space
This patch utilizes remap_4k_pfn() as introduced by Paul M.,
for details see http://patchwork.ozlabs.org/linuxppc/patch?id=10281,
to map ehca cq, qp firmware context (4k) to user space if kernel page
size is 64k. For reason, why this is required, see also Paul's patch.
In addition to that the kernel page offset of firmware context needs
to be set in cq and qp response block so that user space can assemble
the proper virtual address to use.
An appropriate patch for libehca will follow for ofed-1.3.
Signed-off-by: Hoang-Nam Nguyen <hnguyen@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_classes.h | 4 +++-
drivers/infiniband/hw/ehca/ehca_cq.c | 2 ++
drivers/infiniband/hw/ehca/ehca_qp.c | 2 ++
drivers/infiniband/hw/ehca/ehca_uverbs.c | 8 +++++++-
4 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h
index b5e9603..206d4eb 100644
--- a/drivers/infiniband/hw/ehca/ehca_classes.h
+++ b/drivers/infiniband/hw/ehca/ehca_classes.h
@@ -337,6 +337,8 @@ struct ehca_create_cq_resp {
u32 cq_number;
u32 token;
struct ipzu_queue_resp ipz_queue;
+ u32 fw_handle_ofs;
+ u32 dummy;
};
struct ehca_create_qp_resp {
@@ -347,7 +349,7 @@ struct ehca_create_qp_resp {
u32 qkey;
/* qp_num assigned by ehca: sqp0/1 may have got different numbers */
u32 real_qp_num;
- u32 dummy; /* padding for 8 byte alignment */
+ u32 fw_handle_ofs;
struct ipzu_queue_resp ipz_squeue;
struct ipzu_queue_resp ipz_rqueue;
};
diff --git a/drivers/infiniband/hw/ehca/ehca_cq.c b/drivers/infiniband/hw/ehca/ehca_cq.c
index 81aff36..ed5d67f 100644
--- a/drivers/infiniband/hw/ehca/ehca_cq.c
+++ b/drivers/infiniband/hw/ehca/ehca_cq.c
@@ -276,6 +276,8 @@ struct ib_cq *ehca_create_cq(struct ib_device *device, int cqe, int comp_vector,
resp.ipz_queue.queue_length = ipz_queue->queue_length;
resp.ipz_queue.pagesize = ipz_queue->pagesize;
resp.ipz_queue.toggle_state = ipz_queue->toggle_state;
+ resp.fw_handle_ofs = (u32)
+ (my_cq->galpas.user.fw_handle & (PAGE_SIZE - 1));
if (ib_copy_to_udata(udata, &resp, sizeof(resp))) {
ehca_err(device, "Copy to udata failed.");
goto create_cq_exit4;
diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c
index b178cba..66f632c 100644
--- a/drivers/infiniband/hw/ehca/ehca_qp.c
+++ b/drivers/infiniband/hw/ehca/ehca_qp.c
@@ -745,6 +745,8 @@ static struct ehca_qp *internal_create_qp(
queue2resp(&resp.ipz_squeue, &my_qp->ipz_squeue);
if (HAS_RQ(my_qp))
queue2resp(&resp.ipz_rqueue, &my_qp->ipz_rqueue);
+ resp.fw_handle_ofs = (u32)
+ (my_qp->galpas.user.fw_handle & (PAGE_SIZE - 1));
if (ib_copy_to_udata(udata, &resp, sizeof resp)) {
ehca_err(pd->device, "Copy to udata failed");
diff --git a/drivers/infiniband/hw/ehca/ehca_uverbs.c b/drivers/infiniband/hw/ehca/ehca_uverbs.c
index 4bc687f..1308efa 100644
--- a/drivers/infiniband/hw/ehca/ehca_uverbs.c
+++ b/drivers/infiniband/hw/ehca/ehca_uverbs.c
@@ -109,7 +109,7 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas,
u64 vsize, physical;
vsize = vma->vm_end - vma->vm_start;
- if (vsize != EHCA_PAGESIZE) {
+ if (vsize >= EHCA_PAGESIZE) {
ehca_gen_err("invalid vsize=%lx", vma->vm_end - vma->vm_start);
return -EINVAL;
}
@@ -118,8 +118,14 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas,
vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
ehca_gen_dbg("vsize=%lx physical=%lx", vsize, physical);
/* VM_IO | VM_RESERVED are set by remap_pfn_range() */
+#ifdef CONFIG_PPC_64K_PAGES
+ /* make sure we map only 4k for fw context */
+ ret = remap_4k_pfn(vma, vma->vm_start, physical >> EHCA_PAGESHIFT,
+ vma->vm_page_prot);
+#else
ret = remap_pfn_range(vma, vma->vm_start, physical >> PAGE_SHIFT,
vsize, vma->vm_page_prot);
+#endif
if (unlikely(ret)) {
ehca_gen_err("remap_pfn_range() failed ret=%x", ret);
return -ENOMEM;
--
1.5.2
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] ehca: map 4k firmware context of cq, qp to user space
2007-08-02 9:39 [PATCH] ehca: map 4k firmware context of cq, qp to user space Hoang-Nam Nguyen
@ 2007-08-02 14:08 ` Arnd Bergmann
2007-08-02 21:03 ` Roland Dreier
0 siblings, 1 reply; 6+ messages in thread
From: Arnd Bergmann @ 2007-08-02 14:08 UTC (permalink / raw)
To: linuxppc-dev
Cc: Roland Dreier, linux-kernel, raisch, paulus, general,
Hoang-Nam Nguyen
On Thursday 02 August 2007, Hoang-Nam Nguyen wrote:
> +#ifdef CONFIG_PPC_64K_PAGES
> +=A0=A0=A0=A0=A0=A0=A0/* make sure we map only 4k for fw context */
> +=A0=A0=A0=A0=A0=A0=A0ret =3D remap_4k_pfn(vma, vma->vm_start, physical >=
> EHCA_PAGESHIFT,
> +=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =
=A0 vma->vm_page_prot);
> +#else
> =A0=A0=A0=A0=A0=A0=A0=A0ret =3D remap_pfn_range(vma, vma->vm_start, physi=
cal >> PAGE_SHIFT,
> =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0 =
=A0 =A0 =A0vsize, vma->vm_page_prot);
> +#endif
remap_4k_pfn is defined in terms of remap_pfn_range if the base page
size if 4k, so you don't need this #ifdef afaics.
otherwise, the patch looks good.
Arnd <><
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] ehca: map 4k firmware context of cq, qp to user space
2007-08-02 14:08 ` Arnd Bergmann
@ 2007-08-02 21:03 ` Roland Dreier
2007-08-03 8:36 ` Hoang-Nam Nguyen
0 siblings, 1 reply; 6+ messages in thread
From: Roland Dreier @ 2007-08-02 21:03 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-kernel, linuxppc-dev, raisch, paulus, general,
Hoang-Nam Nguyen
> remap_4k_pfn is defined in terms of remap_pfn_range if the base page
> size if 4k, so you don't need this #ifdef afaics.
Good point. I'll wait for an updated patch.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] ehca: map 4k firmware context of cq, qp to user space
2007-08-02 21:03 ` Roland Dreier
@ 2007-08-03 8:36 ` Hoang-Nam Nguyen
2007-08-08 14:13 ` Hoang-Nam Nguyen
0 siblings, 1 reply; 6+ messages in thread
From: Hoang-Nam Nguyen @ 2007-08-03 8:36 UTC (permalink / raw)
To: Roland Dreier
Cc: Arnd Bergmann, linux-kernel, linuxppc-dev, raisch, paulus,
general
From: Hoang-Nam Nguyen <hnguyen at de.ibm.com>
Date: Fri, 3 Aug 2007 09:44:56 +0200
Subject: [PATCH] ehca: map 4k firmware context of cq, qp to user space
This patch utilizes remap_4k_pfn() as introduced by Paul M.,
for details see http://patchwork.ozlabs.org/linuxppc/patch?id=10281,
to map ehca cq, qp firmware context (4k) to user space if kernel page
size is 64k. For reason, why this is required, see also Paul's patch.
In addition to that the kernel page offset of firmware context needs
to be set in cq and qp response block so that user space can assemble
the proper virtual address to use.
An appropriate patch for libehca will follow for ofed-1.3.
Signed-off-by: Hoang-Nam Nguyen <hnguyen@de.ibm.com>
---
drivers/infiniband/hw/ehca/ehca_classes.h | 4 +++-
drivers/infiniband/hw/ehca/ehca_cq.c | 2 ++
drivers/infiniband/hw/ehca/ehca_qp.c | 2 ++
drivers/infiniband/hw/ehca/ehca_uverbs.c | 6 +++---
4 files changed, 10 insertions(+), 4 deletions(-)
diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h
index b5e9603..206d4eb 100644
--- a/drivers/infiniband/hw/ehca/ehca_classes.h
+++ b/drivers/infiniband/hw/ehca/ehca_classes.h
@@ -337,6 +337,8 @@ struct ehca_create_cq_resp {
u32 cq_number;
u32 token;
struct ipzu_queue_resp ipz_queue;
+ u32 fw_handle_ofs;
+ u32 dummy;
};
struct ehca_create_qp_resp {
@@ -347,7 +349,7 @@ struct ehca_create_qp_resp {
u32 qkey;
/* qp_num assigned by ehca: sqp0/1 may have got different numbers */
u32 real_qp_num;
- u32 dummy; /* padding for 8 byte alignment */
+ u32 fw_handle_ofs;
struct ipzu_queue_resp ipz_squeue;
struct ipzu_queue_resp ipz_rqueue;
};
diff --git a/drivers/infiniband/hw/ehca/ehca_cq.c b/drivers/infiniband/hw/ehca/ehca_cq.c
index 81aff36..ed5d67f 100644
--- a/drivers/infiniband/hw/ehca/ehca_cq.c
+++ b/drivers/infiniband/hw/ehca/ehca_cq.c
@@ -276,6 +276,8 @@ struct ib_cq *ehca_create_cq(struct ib_device *device, int cqe, int comp_vector,
resp.ipz_queue.queue_length = ipz_queue->queue_length;
resp.ipz_queue.pagesize = ipz_queue->pagesize;
resp.ipz_queue.toggle_state = ipz_queue->toggle_state;
+ resp.fw_handle_ofs = (u32)
+ (my_cq->galpas.user.fw_handle & (PAGE_SIZE - 1));
if (ib_copy_to_udata(udata, &resp, sizeof(resp))) {
ehca_err(device, "Copy to udata failed.");
goto create_cq_exit4;
diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c
index b178cba..66f632c 100644
--- a/drivers/infiniband/hw/ehca/ehca_qp.c
+++ b/drivers/infiniband/hw/ehca/ehca_qp.c
@@ -745,6 +745,8 @@ static struct ehca_qp *internal_create_qp(
queue2resp(&resp.ipz_squeue, &my_qp->ipz_squeue);
if (HAS_RQ(my_qp))
queue2resp(&resp.ipz_rqueue, &my_qp->ipz_rqueue);
+ resp.fw_handle_ofs = (u32)
+ (my_qp->galpas.user.fw_handle & (PAGE_SIZE - 1));
if (ib_copy_to_udata(udata, &resp, sizeof resp)) {
ehca_err(pd->device, "Copy to udata failed");
diff --git a/drivers/infiniband/hw/ehca/ehca_uverbs.c b/drivers/infiniband/hw/ehca/ehca_uverbs.c
index 4bc687f..be062f1 100644
--- a/drivers/infiniband/hw/ehca/ehca_uverbs.c
+++ b/drivers/infiniband/hw/ehca/ehca_uverbs.c
@@ -109,7 +109,7 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas,
u64 vsize, physical;
vsize = vma->vm_end - vma->vm_start;
- if (vsize != EHCA_PAGESIZE) {
+ if (vsize >= EHCA_PAGESIZE) {
ehca_gen_err("invalid vsize=%lx", vma->vm_end - vma->vm_start);
return -EINVAL;
}
@@ -118,8 +118,8 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas,
vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
ehca_gen_dbg("vsize=%lx physical=%lx", vsize, physical);
/* VM_IO | VM_RESERVED are set by remap_pfn_range() */
- ret = remap_pfn_range(vma, vma->vm_start, physical >> PAGE_SHIFT,
- vsize, vma->vm_page_prot);
+ ret = remap_4k_pfn(vma, vma->vm_start, physical >> EHCA_PAGESHIFT,
+ vma->vm_page_prot);
if (unlikely(ret)) {
ehca_gen_err("remap_pfn_range() failed ret=%x", ret);
return -ENOMEM;
--
1.5.2
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] ehca: map 4k firmware context of cq, qp to user space
2007-08-03 8:36 ` Hoang-Nam Nguyen
@ 2007-08-08 14:13 ` Hoang-Nam Nguyen
2007-08-08 16:08 ` Roland Dreier
0 siblings, 1 reply; 6+ messages in thread
From: Hoang-Nam Nguyen @ 2007-08-08 14:13 UTC (permalink / raw)
To: Roland Dreier
Cc: Arnd Bergmann, linux-kernel, linuxppc-dev, paulus, general,
stefan.roscher
Hello Roland!
Haven't got any ack for this updated patch yet. Anyway, since it contains
another bug as shown below, please ignore this patch. We'll send a patch
set that includes the proper version of this patch later.
> @@ -109,7 +109,7 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas,
> u64 vsize, physical;
>
> vsize = vma->vm_end - vma->vm_start;
> - if (vsize != EHCA_PAGESIZE) {
> + if (vsize >= EHCA_PAGESIZE) {
should be
> + if (vsize < EHCA_PAGESIZE) {
which is sort of invalid arg.
Thanks
Nam
On Friday 03 August 2007 10:36, Hoang-Nam Nguyen wrote:
> From: Hoang-Nam Nguyen <hnguyen at de.ibm.com>
> Date: Fri, 3 Aug 2007 09:44:56 +0200
> Subject: [PATCH] ehca: map 4k firmware context of cq, qp to user space
> This patch utilizes remap_4k_pfn() as introduced by Paul M.,
> for details see http://patchwork.ozlabs.org/linuxppc/patch?id=10281,
> to map ehca cq, qp firmware context (4k) to user space if kernel page
> size is 64k. For reason, why this is required, see also Paul's patch.
> In addition to that the kernel page offset of firmware context needs
> to be set in cq and qp response block so that user space can assemble
> the proper virtual address to use.
> An appropriate patch for libehca will follow for ofed-1.3.
>
> Signed-off-by: Hoang-Nam Nguyen <hnguyen@de.ibm.com>
> ---
> drivers/infiniband/hw/ehca/ehca_classes.h | 4 +++-
> drivers/infiniband/hw/ehca/ehca_cq.c | 2 ++
> drivers/infiniband/hw/ehca/ehca_qp.c | 2 ++
> drivers/infiniband/hw/ehca/ehca_uverbs.c | 6 +++---
> 4 files changed, 10 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h b/drivers/infiniband/hw/ehca/ehca_classes.h
> index b5e9603..206d4eb 100644
> --- a/drivers/infiniband/hw/ehca/ehca_classes.h
> +++ b/drivers/infiniband/hw/ehca/ehca_classes.h
> @@ -337,6 +337,8 @@ struct ehca_create_cq_resp {
> u32 cq_number;
> u32 token;
> struct ipzu_queue_resp ipz_queue;
> + u32 fw_handle_ofs;
> + u32 dummy;
> };
>
> struct ehca_create_qp_resp {
> @@ -347,7 +349,7 @@ struct ehca_create_qp_resp {
> u32 qkey;
> /* qp_num assigned by ehca: sqp0/1 may have got different numbers */
> u32 real_qp_num;
> - u32 dummy; /* padding for 8 byte alignment */
> + u32 fw_handle_ofs;
> struct ipzu_queue_resp ipz_squeue;
> struct ipzu_queue_resp ipz_rqueue;
> };
> diff --git a/drivers/infiniband/hw/ehca/ehca_cq.c b/drivers/infiniband/hw/ehca/ehca_cq.c
> index 81aff36..ed5d67f 100644
> --- a/drivers/infiniband/hw/ehca/ehca_cq.c
> +++ b/drivers/infiniband/hw/ehca/ehca_cq.c
> @@ -276,6 +276,8 @@ struct ib_cq *ehca_create_cq(struct ib_device *device, int cqe, int comp_vector,
> resp.ipz_queue.queue_length = ipz_queue->queue_length;
> resp.ipz_queue.pagesize = ipz_queue->pagesize;
> resp.ipz_queue.toggle_state = ipz_queue->toggle_state;
> + resp.fw_handle_ofs = (u32)
> + (my_cq->galpas.user.fw_handle & (PAGE_SIZE - 1));
> if (ib_copy_to_udata(udata, &resp, sizeof(resp))) {
> ehca_err(device, "Copy to udata failed.");
> goto create_cq_exit4;
> diff --git a/drivers/infiniband/hw/ehca/ehca_qp.c b/drivers/infiniband/hw/ehca/ehca_qp.c
> index b178cba..66f632c 100644
> --- a/drivers/infiniband/hw/ehca/ehca_qp.c
> +++ b/drivers/infiniband/hw/ehca/ehca_qp.c
> @@ -745,6 +745,8 @@ static struct ehca_qp *internal_create_qp(
> queue2resp(&resp.ipz_squeue, &my_qp->ipz_squeue);
> if (HAS_RQ(my_qp))
> queue2resp(&resp.ipz_rqueue, &my_qp->ipz_rqueue);
> + resp.fw_handle_ofs = (u32)
> + (my_qp->galpas.user.fw_handle & (PAGE_SIZE - 1));
>
> if (ib_copy_to_udata(udata, &resp, sizeof resp)) {
> ehca_err(pd->device, "Copy to udata failed");
> diff --git a/drivers/infiniband/hw/ehca/ehca_uverbs.c b/drivers/infiniband/hw/ehca/ehca_uverbs.c
> index 4bc687f..be062f1 100644
> --- a/drivers/infiniband/hw/ehca/ehca_uverbs.c
> +++ b/drivers/infiniband/hw/ehca/ehca_uverbs.c
> @@ -109,7 +109,7 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas,
> u64 vsize, physical;
>
> vsize = vma->vm_end - vma->vm_start;
> - if (vsize != EHCA_PAGESIZE) {
> + if (vsize >= EHCA_PAGESIZE) {
> ehca_gen_err("invalid vsize=%lx", vma->vm_end - vma->vm_start);
> return -EINVAL;
> }
> @@ -118,8 +118,8 @@ static int ehca_mmap_fw(struct vm_area_struct *vma, struct h_galpas *galpas,
> vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
> ehca_gen_dbg("vsize=%lx physical=%lx", vsize, physical);
> /* VM_IO | VM_RESERVED are set by remap_pfn_range() */
> - ret = remap_pfn_range(vma, vma->vm_start, physical >> PAGE_SHIFT,
> - vsize, vma->vm_page_prot);
> + ret = remap_4k_pfn(vma, vma->vm_start, physical >> EHCA_PAGESHIFT,
> + vma->vm_page_prot);
> if (unlikely(ret)) {
> ehca_gen_err("remap_pfn_range() failed ret=%x", ret);
> return -ENOMEM;
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] ehca: map 4k firmware context of cq, qp to user space
2007-08-08 14:13 ` Hoang-Nam Nguyen
@ 2007-08-08 16:08 ` Roland Dreier
0 siblings, 0 replies; 6+ messages in thread
From: Roland Dreier @ 2007-08-08 16:08 UTC (permalink / raw)
To: Hoang-Nam Nguyen
Cc: Arnd Bergmann, linux-kernel, linuxppc-dev, paulus, general,
stefan.roscher
> Haven't got any ack for this updated patch yet. Anyway, since it contains
> another bug as shown below, please ignore this patch. We'll send a patch
> set that includes the proper version of this patch later.
Yes, sorry, I'm a bit behind applying patches.
Anyway, OK I'll wait for a fixed patch and drop this one :)
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2007-08-08 16:08 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-08-02 9:39 [PATCH] ehca: map 4k firmware context of cq, qp to user space Hoang-Nam Nguyen
2007-08-02 14:08 ` Arnd Bergmann
2007-08-02 21:03 ` Roland Dreier
2007-08-03 8:36 ` Hoang-Nam Nguyen
2007-08-08 14:13 ` Hoang-Nam Nguyen
2007-08-08 16:08 ` Roland Dreier
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).