From: Oded Gabbay <ogabbay@kernel.org>
To: linux-kernel@vger.kernel.org
Cc: Tomer Tayar <ttayar@habana.ai>
Subject: [PATCH 07/12] habanalabs/gaudi2: map virtual MSI-X doorbell memory for user
Date: Mon, 11 Jul 2022 09:29:58 +0300 [thread overview]
Message-ID: <20220711063003.3182795-7-ogabbay@kernel.org> (raw)
In-Reply-To: <20220711063003.3182795-1-ogabbay@kernel.org>
From: Tomer Tayar <ttayar@habana.ai>
Upon the initialization of a user context, map the host memory page of
the virtual MSI-X doorbell in the device MMU.
A reserved VA is used for this purpose, so user can use it directly
without any allocation/map operation.
Signed-off-by: Tomer Tayar <ttayar@habana.ai>
Reviewed-by: Oded Gabbay <ogabbay@kernel.org>
Signed-off-by: Oded Gabbay <ogabbay@kernel.org>
---
drivers/misc/habanalabs/gaudi2/gaudi2.c | 43 ++++++++++++++++++-
.../misc/habanalabs/include/gaudi2/gaudi2.h | 3 ++
2 files changed, 44 insertions(+), 2 deletions(-)
diff --git a/drivers/misc/habanalabs/gaudi2/gaudi2.c b/drivers/misc/habanalabs/gaudi2/gaudi2.c
index 760ce5b7f955..eba8b0d674c3 100644
--- a/drivers/misc/habanalabs/gaudi2/gaudi2.c
+++ b/drivers/misc/habanalabs/gaudi2/gaudi2.c
@@ -1920,7 +1920,7 @@ static int gaudi2_set_fixed_properties(struct hl_device *hdev)
prop->pmmu.hop_table_size = prop->mmu_hop_table_size;
prop->pmmu.hop0_tables_total_size = prop->mmu_hop0_tables_total_size;
- prop->hints_host_reserved_va_range.start_addr = RESERVED_VA_RANGE_FOR_ARC_ON_HOST_START;
+ prop->hints_host_reserved_va_range.start_addr = RESERVED_VA_FOR_VIRTUAL_MSIX_DOORBELL_START;
prop->hints_host_reserved_va_range.end_addr = RESERVED_VA_RANGE_FOR_ARC_ON_HOST_END;
prop->hints_host_hpage_reserved_va_range.start_addr =
RESERVED_VA_RANGE_FOR_ARC_ON_HOST_HPAGE_START;
@@ -9241,6 +9241,35 @@ static void gaudi2_restore_user_registers(struct hl_device *hdev)
gaudi2_restore_user_qm_registers(hdev);
}
+static int gaudi2_map_virtual_msix_doorbell_memory(struct hl_ctx *ctx)
+{
+ struct hl_device *hdev = ctx->hdev;
+ struct asic_fixed_properties *prop = &hdev->asic_prop;
+ struct gaudi2_device *gaudi2 = hdev->asic_specific;
+ int rc;
+
+ rc = hl_mmu_map_page(ctx, RESERVED_VA_FOR_VIRTUAL_MSIX_DOORBELL_START,
+ gaudi2->virt_msix_db_dma_addr, prop->pmmu.page_size, true);
+ if (rc)
+ dev_err(hdev->dev, "Failed to map VA %#llx for virtual MSI-X doorbell memory\n",
+ RESERVED_VA_FOR_VIRTUAL_MSIX_DOORBELL_START);
+
+ return rc;
+}
+
+static void gaudi2_unmap_virtual_msix_doorbell_memory(struct hl_ctx *ctx)
+{
+ struct hl_device *hdev = ctx->hdev;
+ struct asic_fixed_properties *prop = &hdev->asic_prop;
+ int rc;
+
+ rc = hl_mmu_unmap_page(ctx, RESERVED_VA_FOR_VIRTUAL_MSIX_DOORBELL_START,
+ prop->pmmu.page_size, true);
+ if (rc)
+ dev_err(hdev->dev, "Failed to unmap VA %#llx of virtual MSI-X doorbell memory\n",
+ RESERVED_VA_FOR_VIRTUAL_MSIX_DOORBELL_START);
+}
+
static int gaudi2_ctx_init(struct hl_ctx *ctx)
{
int rc;
@@ -9257,7 +9286,15 @@ static int gaudi2_ctx_init(struct hl_ctx *ctx)
else
gaudi2_restore_user_registers(ctx->hdev);
- return gaudi2_internal_cb_pool_init(ctx->hdev, ctx);
+ rc = gaudi2_internal_cb_pool_init(ctx->hdev, ctx);
+ if (rc)
+ return rc;
+
+ rc = gaudi2_map_virtual_msix_doorbell_memory(ctx);
+ if (rc)
+ gaudi2_internal_cb_pool_fini(ctx->hdev, ctx);
+
+ return rc;
}
static void gaudi2_ctx_fini(struct hl_ctx *ctx)
@@ -9266,6 +9303,8 @@ static void gaudi2_ctx_fini(struct hl_ctx *ctx)
return;
gaudi2_internal_cb_pool_fini(ctx->hdev, ctx);
+
+ gaudi2_unmap_virtual_msix_doorbell_memory(ctx);
}
static int gaudi2_pre_schedule_cs(struct hl_cs *cs)
diff --git a/drivers/misc/habanalabs/include/gaudi2/gaudi2.h b/drivers/misc/habanalabs/include/gaudi2/gaudi2.h
index 071fc5a820f7..5b4f9e108798 100644
--- a/drivers/misc/habanalabs/include/gaudi2/gaudi2.h
+++ b/drivers/misc/habanalabs/include/gaudi2/gaudi2.h
@@ -54,6 +54,9 @@
#define RESERVED_VA_RANGE_FOR_ARC_ON_HBM_START 0x1001500000000000ull
#define RESERVED_VA_RANGE_FOR_ARC_ON_HBM_END 0x10016FFFFFFFFFFFull
+#define RESERVED_VA_FOR_VIRTUAL_MSIX_DOORBELL_START 0xFFF077FFFFFF0000ull
+#define RESERVED_VA_FOR_VIRTUAL_MSIX_DOORBELL_END 0xFFF077FFFFFFFFFFull
+
#define RESERVED_VA_RANGE_FOR_ARC_ON_HOST_START 0xFFF0780000000000ull
#define RESERVED_VA_RANGE_FOR_ARC_ON_HOST_END 0xFFF07FFFFFFFFFFFull
--
2.25.1
next prev parent reply other threads:[~2022-07-11 6:30 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-07-11 6:29 [PATCH 01/12] habanalabs: fixes to the poll-timeout macros Oded Gabbay
2022-07-11 6:29 ` [PATCH 02/12] habanalabs: add a value field to hl_fw_send_pci_access_msg() Oded Gabbay
2022-07-11 6:29 ` [PATCH 03/12] habanalabs/gaudi2: configure virtual MSI-X doorbell interface Oded Gabbay
2022-07-11 6:29 ` [PATCH 04/12] habanalabs/gaudi2: replace defines for reserved sob/mob with enums Oded Gabbay
2022-07-11 6:29 ` [PATCH 05/12] habanalabs/gaudi2: modify CS completion CQ to use virtual MSI-X doorbell Oded Gabbay
2022-07-11 6:29 ` [PATCH 06/12] habanalabs/gaudi2: modify decoder " Oded Gabbay
2022-07-11 6:29 ` Oded Gabbay [this message]
2022-07-11 6:29 ` [PATCH 08/12] habanalabs: expose only valid debugfs nodes Oded Gabbay
2022-07-11 6:30 ` [PATCH 09/12] habanalabs: fix update of is_in_soft_reset Oded Gabbay
2022-07-11 6:30 ` [PATCH 10/12] habanalabs: add status of reset after device release Oded Gabbay
2022-07-11 6:30 ` [PATCH 11/12] habanalabs: rename soft reset to compute reset Oded Gabbay
2022-07-11 6:30 ` [PATCH 12/12] habanalabs: move h/w dirty message to debug Oded Gabbay
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220711063003.3182795-7-ogabbay@kernel.org \
--to=ogabbay@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=ttayar@habana.ai \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox