* [PATCH 00/25] Parallel submission aka multi-bb execbuf
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
As discussed in [1] we are introducing a new parallel submission uAPI
for the i915 which allows more than 1 BB to be submitted in an execbuf
IOCTL. This is the implemenation for both GuC and execlists.
In addition to selftests in the series, an IGT is available implemented
in the first 4 patches [2].
The execbuf IOCTL changes have been done in a single large patch (#21)
as all the changes flow together and I believe a single patch will be
better if some one has to lookup this change in the future. Can split in
a series of smaller patches if desired.
This code is available in a public [3] repo for UMD teams to test there
code on.
v2: Drop complicated state machine to block in kernel if no guc_ids
available, perma-pin parallel contexts, reworker execbuf IOCTL to be a
series of loops inside the IOCTL rather than 1 large one on the outside,
address Daniel Vetter's comments
v3: Address John Harrison's comments, add a couple of patches which fix
bugs found internally
v4: Address John Harrison's latest round of comments
v5: Address John Harrison's latest round of comments, resend for CI
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
[1] https://patchwork.freedesktop.org/series/92028/
[2] https://patchwork.freedesktop.org/series/93071/
[3] https://gitlab.freedesktop.org/mbrost/mbrost-drm-intel/-/tree/drm-intel-parallel
Matthew Brost (25):
drm/i915/guc: Move GuC guc_id allocation under submission state
sub-struct
drm/i915/guc: Take GT PM ref when deregistering context
drm/i915/guc: Take engine PM when a context is pinned with GuC
submission
drm/i915/guc: Don't call switch_to_kernel_context with GuC submission
drm/i915: Add logical engine mapping
drm/i915: Expose logical engine instance to user
drm/i915/guc: Introduce context parent-child relationship
drm/i915/guc: Add multi-lrc context registration
drm/i915/guc: Ensure GuC schedule operations do not operate on child
contexts
drm/i915/guc: Assign contexts in parent-child relationship consecutive
guc_ids
drm/i915/guc: Implement parallel context pin / unpin functions
drm/i915/guc: Implement multi-lrc submission
drm/i915/guc: Insert submit fences between requests in parent-child
relationship
drm/i915/guc: Implement multi-lrc reset
drm/i915/guc: Update debugfs for GuC multi-lrc
drm/i915/guc: Connect UAPI to GuC multi-lrc interface
drm/i915/doc: Update parallel submit doc to point to i915_drm.h
drm/i915/guc: Add basic GuC multi-lrc selftest
drm/i915/guc: Implement no mid batch preemption for multi-lrc
drm/i915: Multi-BB execbuf
drm/i915/guc: Handle errors in multi-lrc requests
drm/i915: Make request conflict tracking understand parallel submits
drm/i915: Update I915_GEM_BUSY IOCTL to understand composite fences
drm/i915: Enable multi-bb execbuf
drm/i915/execlists: Weak parallel submission support for execlists
Documentation/gpu/rfc/i915_parallel_execbuf.h | 122 --
Documentation/gpu/rfc/i915_scheduler.rst | 4 +-
drivers/gpu/drm/i915/gem/i915_gem_busy.c | 57 +-
drivers/gpu/drm/i915/gem/i915_gem_context.c | 229 ++-
.../gpu/drm/i915/gem/i915_gem_context_types.h | 16 +-
.../gpu/drm/i915/gem/i915_gem_execbuffer.c | 786 ++++++---
drivers/gpu/drm/i915/gt/intel_context.c | 50 +-
drivers/gpu/drm/i915/gt/intel_context.h | 56 +-
drivers/gpu/drm/i915/gt/intel_context_types.h | 73 +-
drivers/gpu/drm/i915/gt/intel_engine.h | 12 +-
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 66 +-
drivers/gpu/drm/i915/gt/intel_engine_pm.c | 13 +
drivers/gpu/drm/i915/gt/intel_engine_pm.h | 37 +
drivers/gpu/drm/i915/gt/intel_engine_types.h | 7 +
.../drm/i915/gt/intel_execlists_submission.c | 63 +-
drivers/gpu/drm/i915/gt/intel_gt_pm.h | 14 +
drivers/gpu/drm/i915/gt/intel_lrc.c | 7 +
drivers/gpu/drm/i915/gt/selftest_execlists.c | 12 +-
.../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h | 1 +
drivers/gpu/drm/i915/gt/uc/intel_guc.c | 29 +
drivers/gpu/drm/i915/gt/uc/intel_guc.h | 54 +-
drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c | 2 +-
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 24 +-
drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 34 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 1444 ++++++++++++++---
.../drm/i915/gt/uc/selftest_guc_multi_lrc.c | 179 ++
drivers/gpu/drm/i915/i915_query.c | 2 +
drivers/gpu/drm/i915/i915_request.c | 143 +-
drivers/gpu/drm/i915/i915_request.h | 23 +
drivers/gpu/drm/i915/i915_vma.c | 21 +-
drivers/gpu/drm/i915/i915_vma.h | 13 +-
drivers/gpu/drm/i915/intel_wakeref.h | 12 +
.../drm/i915/selftests/i915_live_selftests.h | 1 +
include/uapi/drm/i915_drm.h | 139 +-
34 files changed, 3053 insertions(+), 692 deletions(-)
delete mode 100644 Documentation/gpu/rfc/i915_parallel_execbuf.h
create mode 100644 drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c
--
2.32.0
^ permalink raw reply [flat|nested] 69+ messages in thread* [Intel-gfx] [PATCH 01/25] drm/i915/guc: Move GuC guc_id allocation under submission state sub-struct
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Move guc_id allocation under submission state sub-struct as a future
patch will reuse the spin lock as a global submission state lock. Moving
this into sub-struct makes ownership of fields / lock clear.
v2:
(Docs)
- Add comment for submission_state sub-structure
v3:
(John Harrison)
- Fixup a few comments
v4:
(John Harrison)
- Fix typo
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_context_types.h | 6 +--
drivers/gpu/drm/i915/gt/uc/intel_guc.h | 28 +++++++----
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 46 ++++++++++---------
3 files changed, 46 insertions(+), 34 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 12252c411159..e7e3984aab78 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -197,18 +197,18 @@ struct intel_context {
struct {
/**
* @id: handle which is used to uniquely identify this context
- * with the GuC, protected by guc->contexts_lock
+ * with the GuC, protected by guc->submission_state.lock
*/
u16 id;
/**
* @ref: the number of references to the guc_id, when
* transitioning in and out of zero protected by
- * guc->contexts_lock
+ * guc->submission_state.lock
*/
atomic_t ref;
/**
* @link: in guc->guc_id_list when the guc_id has no refs but is
- * still valid, protected by guc->contexts_lock
+ * still valid, protected by guc->submission_state.lock
*/
struct list_head link;
} guc_id;
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
index 5dd174babf7a..82e248c2290c 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
@@ -71,16 +71,26 @@ struct intel_guc {
} interrupts;
/**
- * @contexts_lock: protects guc_ids, guc_id_list, ce->guc_id.id, and
- * ce->guc_id.ref when transitioning in and out of zero
+ * @submission_state: sub-structure for submission state protected by
+ * single lock
*/
- spinlock_t contexts_lock;
- /** @guc_ids: used to allocate unique ce->guc_id.id values */
- struct ida guc_ids;
- /**
- * @guc_id_list: list of intel_context with valid guc_ids but no refs
- */
- struct list_head guc_id_list;
+ struct {
+ /**
+ * @lock: protects everything in submission_state,
+ * ce->guc_id.id, and ce->guc_id.ref when transitioning in and
+ * out of zero
+ */
+ spinlock_t lock;
+ /**
+ * @guc_ids: used to allocate new guc_ids
+ */
+ struct ida guc_ids;
+ /**
+ * @guc_id_list: list of intel_context with valid guc_ids but no
+ * refs
+ */
+ struct list_head guc_id_list;
+ } submission_state;
/**
* @submission_supported: tracks whether we support GuC submission on
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index ba0de35f6323..48bdcdb823b3 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -68,7 +68,7 @@
* fence is used to stall all requests associated with this guc_id until the
* corresponding G2H returns indicating the guc_id has been deregistered.
*
- * guc_ids:
+ * submission_state.guc_ids:
* Unique number associated with private GuC context data passed in during
* context registration / submission / deregistration. 64k available. Simple ida
* is used for allocation.
@@ -89,7 +89,7 @@
* sched_engine can be submitting at a time. Currently only one sched_engine is
* used for all of GuC submission but that could change in the future.
*
- * guc->contexts_lock
+ * guc->submission_state.lock
* Protects guc_id allocation for the given GuC, i.e. only one context can be
* doing guc_id allocation operations at a time for each GuC in the system.
*
@@ -103,7 +103,7 @@
*
* Lock ordering rules:
* sched_engine->lock -> ce->guc_state.lock
- * guc->contexts_lock -> ce->guc_state.lock
+ * guc->submission_state.lock -> ce->guc_state.lock
*
* Reset races:
* When a full GT reset is triggered it is assumed that some G2H responses to
@@ -1148,9 +1148,9 @@ int intel_guc_submission_init(struct intel_guc *guc)
xa_init_flags(&guc->context_lookup, XA_FLAGS_LOCK_IRQ);
- spin_lock_init(&guc->contexts_lock);
- INIT_LIST_HEAD(&guc->guc_id_list);
- ida_init(&guc->guc_ids);
+ spin_lock_init(&guc->submission_state.lock);
+ INIT_LIST_HEAD(&guc->submission_state.guc_id_list);
+ ida_init(&guc->submission_state.guc_ids);
return 0;
}
@@ -1215,7 +1215,7 @@ static void guc_submit_request(struct i915_request *rq)
static int new_guc_id(struct intel_guc *guc)
{
- return ida_simple_get(&guc->guc_ids, 0,
+ return ida_simple_get(&guc->submission_state.guc_ids, 0,
GUC_MAX_LRC_DESCRIPTORS, GFP_KERNEL |
__GFP_RETRY_MAYFAIL | __GFP_NOWARN);
}
@@ -1223,7 +1223,8 @@ static int new_guc_id(struct intel_guc *guc)
static void __release_guc_id(struct intel_guc *guc, struct intel_context *ce)
{
if (!context_guc_id_invalid(ce)) {
- ida_simple_remove(&guc->guc_ids, ce->guc_id.id);
+ ida_simple_remove(&guc->submission_state.guc_ids,
+ ce->guc_id.id);
reset_lrc_desc(guc, ce->guc_id.id);
set_context_guc_id_invalid(ce);
}
@@ -1235,9 +1236,9 @@ static void release_guc_id(struct intel_guc *guc, struct intel_context *ce)
{
unsigned long flags;
- spin_lock_irqsave(&guc->contexts_lock, flags);
+ spin_lock_irqsave(&guc->submission_state.lock, flags);
__release_guc_id(guc, ce);
- spin_unlock_irqrestore(&guc->contexts_lock, flags);
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
}
static int steal_guc_id(struct intel_guc *guc)
@@ -1245,10 +1246,10 @@ static int steal_guc_id(struct intel_guc *guc)
struct intel_context *ce;
int guc_id;
- lockdep_assert_held(&guc->contexts_lock);
+ lockdep_assert_held(&guc->submission_state.lock);
- if (!list_empty(&guc->guc_id_list)) {
- ce = list_first_entry(&guc->guc_id_list,
+ if (!list_empty(&guc->submission_state.guc_id_list)) {
+ ce = list_first_entry(&guc->submission_state.guc_id_list,
struct intel_context,
guc_id.link);
@@ -1273,7 +1274,7 @@ static int assign_guc_id(struct intel_guc *guc, u16 *out)
{
int ret;
- lockdep_assert_held(&guc->contexts_lock);
+ lockdep_assert_held(&guc->submission_state.lock);
ret = new_guc_id(guc);
if (unlikely(ret < 0)) {
@@ -1295,7 +1296,7 @@ static int pin_guc_id(struct intel_guc *guc, struct intel_context *ce)
GEM_BUG_ON(atomic_read(&ce->guc_id.ref));
try_again:
- spin_lock_irqsave(&guc->contexts_lock, flags);
+ spin_lock_irqsave(&guc->submission_state.lock, flags);
might_lock(&ce->guc_state.lock);
@@ -1310,7 +1311,7 @@ static int pin_guc_id(struct intel_guc *guc, struct intel_context *ce)
atomic_inc(&ce->guc_id.ref);
out_unlock:
- spin_unlock_irqrestore(&guc->contexts_lock, flags);
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
/*
* -EAGAIN indicates no guc_id are available, let's retire any
@@ -1346,11 +1347,12 @@ static void unpin_guc_id(struct intel_guc *guc, struct intel_context *ce)
if (unlikely(context_guc_id_invalid(ce)))
return;
- spin_lock_irqsave(&guc->contexts_lock, flags);
+ spin_lock_irqsave(&guc->submission_state.lock, flags);
if (!context_guc_id_invalid(ce) && list_empty(&ce->guc_id.link) &&
!atomic_read(&ce->guc_id.ref))
- list_add_tail(&ce->guc_id.link, &guc->guc_id_list);
- spin_unlock_irqrestore(&guc->contexts_lock, flags);
+ list_add_tail(&ce->guc_id.link,
+ &guc->submission_state.guc_id_list);
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
}
static int __guc_action_register_context(struct intel_guc *guc,
@@ -1921,16 +1923,16 @@ static void guc_context_destroy(struct kref *kref)
* returns indicating this context has been deregistered the guc_id is
* returned to the pool of available guc_id.
*/
- spin_lock_irqsave(&guc->contexts_lock, flags);
+ spin_lock_irqsave(&guc->submission_state.lock, flags);
if (context_guc_id_invalid(ce)) {
- spin_unlock_irqrestore(&guc->contexts_lock, flags);
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
__guc_context_destroy(ce);
return;
}
if (!list_empty(&ce->guc_id.link))
list_del_init(&ce->guc_id.link);
- spin_unlock_irqrestore(&guc->contexts_lock, flags);
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
/* Seal race with Reset */
spin_lock_irqsave(&ce->guc_state.lock, flags);
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 01/25] drm/i915/guc: Move GuC guc_id allocation under submission state sub-struct
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Move guc_id allocation under submission state sub-struct as a future
patch will reuse the spin lock as a global submission state lock. Moving
this into sub-struct makes ownership of fields / lock clear.
v2:
(Docs)
- Add comment for submission_state sub-structure
v3:
(John Harrison)
- Fixup a few comments
v4:
(John Harrison)
- Fix typo
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_context_types.h | 6 +--
drivers/gpu/drm/i915/gt/uc/intel_guc.h | 28 +++++++----
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 46 ++++++++++---------
3 files changed, 46 insertions(+), 34 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 12252c411159..e7e3984aab78 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -197,18 +197,18 @@ struct intel_context {
struct {
/**
* @id: handle which is used to uniquely identify this context
- * with the GuC, protected by guc->contexts_lock
+ * with the GuC, protected by guc->submission_state.lock
*/
u16 id;
/**
* @ref: the number of references to the guc_id, when
* transitioning in and out of zero protected by
- * guc->contexts_lock
+ * guc->submission_state.lock
*/
atomic_t ref;
/**
* @link: in guc->guc_id_list when the guc_id has no refs but is
- * still valid, protected by guc->contexts_lock
+ * still valid, protected by guc->submission_state.lock
*/
struct list_head link;
} guc_id;
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
index 5dd174babf7a..82e248c2290c 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
@@ -71,16 +71,26 @@ struct intel_guc {
} interrupts;
/**
- * @contexts_lock: protects guc_ids, guc_id_list, ce->guc_id.id, and
- * ce->guc_id.ref when transitioning in and out of zero
+ * @submission_state: sub-structure for submission state protected by
+ * single lock
*/
- spinlock_t contexts_lock;
- /** @guc_ids: used to allocate unique ce->guc_id.id values */
- struct ida guc_ids;
- /**
- * @guc_id_list: list of intel_context with valid guc_ids but no refs
- */
- struct list_head guc_id_list;
+ struct {
+ /**
+ * @lock: protects everything in submission_state,
+ * ce->guc_id.id, and ce->guc_id.ref when transitioning in and
+ * out of zero
+ */
+ spinlock_t lock;
+ /**
+ * @guc_ids: used to allocate new guc_ids
+ */
+ struct ida guc_ids;
+ /**
+ * @guc_id_list: list of intel_context with valid guc_ids but no
+ * refs
+ */
+ struct list_head guc_id_list;
+ } submission_state;
/**
* @submission_supported: tracks whether we support GuC submission on
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index ba0de35f6323..48bdcdb823b3 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -68,7 +68,7 @@
* fence is used to stall all requests associated with this guc_id until the
* corresponding G2H returns indicating the guc_id has been deregistered.
*
- * guc_ids:
+ * submission_state.guc_ids:
* Unique number associated with private GuC context data passed in during
* context registration / submission / deregistration. 64k available. Simple ida
* is used for allocation.
@@ -89,7 +89,7 @@
* sched_engine can be submitting at a time. Currently only one sched_engine is
* used for all of GuC submission but that could change in the future.
*
- * guc->contexts_lock
+ * guc->submission_state.lock
* Protects guc_id allocation for the given GuC, i.e. only one context can be
* doing guc_id allocation operations at a time for each GuC in the system.
*
@@ -103,7 +103,7 @@
*
* Lock ordering rules:
* sched_engine->lock -> ce->guc_state.lock
- * guc->contexts_lock -> ce->guc_state.lock
+ * guc->submission_state.lock -> ce->guc_state.lock
*
* Reset races:
* When a full GT reset is triggered it is assumed that some G2H responses to
@@ -1148,9 +1148,9 @@ int intel_guc_submission_init(struct intel_guc *guc)
xa_init_flags(&guc->context_lookup, XA_FLAGS_LOCK_IRQ);
- spin_lock_init(&guc->contexts_lock);
- INIT_LIST_HEAD(&guc->guc_id_list);
- ida_init(&guc->guc_ids);
+ spin_lock_init(&guc->submission_state.lock);
+ INIT_LIST_HEAD(&guc->submission_state.guc_id_list);
+ ida_init(&guc->submission_state.guc_ids);
return 0;
}
@@ -1215,7 +1215,7 @@ static void guc_submit_request(struct i915_request *rq)
static int new_guc_id(struct intel_guc *guc)
{
- return ida_simple_get(&guc->guc_ids, 0,
+ return ida_simple_get(&guc->submission_state.guc_ids, 0,
GUC_MAX_LRC_DESCRIPTORS, GFP_KERNEL |
__GFP_RETRY_MAYFAIL | __GFP_NOWARN);
}
@@ -1223,7 +1223,8 @@ static int new_guc_id(struct intel_guc *guc)
static void __release_guc_id(struct intel_guc *guc, struct intel_context *ce)
{
if (!context_guc_id_invalid(ce)) {
- ida_simple_remove(&guc->guc_ids, ce->guc_id.id);
+ ida_simple_remove(&guc->submission_state.guc_ids,
+ ce->guc_id.id);
reset_lrc_desc(guc, ce->guc_id.id);
set_context_guc_id_invalid(ce);
}
@@ -1235,9 +1236,9 @@ static void release_guc_id(struct intel_guc *guc, struct intel_context *ce)
{
unsigned long flags;
- spin_lock_irqsave(&guc->contexts_lock, flags);
+ spin_lock_irqsave(&guc->submission_state.lock, flags);
__release_guc_id(guc, ce);
- spin_unlock_irqrestore(&guc->contexts_lock, flags);
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
}
static int steal_guc_id(struct intel_guc *guc)
@@ -1245,10 +1246,10 @@ static int steal_guc_id(struct intel_guc *guc)
struct intel_context *ce;
int guc_id;
- lockdep_assert_held(&guc->contexts_lock);
+ lockdep_assert_held(&guc->submission_state.lock);
- if (!list_empty(&guc->guc_id_list)) {
- ce = list_first_entry(&guc->guc_id_list,
+ if (!list_empty(&guc->submission_state.guc_id_list)) {
+ ce = list_first_entry(&guc->submission_state.guc_id_list,
struct intel_context,
guc_id.link);
@@ -1273,7 +1274,7 @@ static int assign_guc_id(struct intel_guc *guc, u16 *out)
{
int ret;
- lockdep_assert_held(&guc->contexts_lock);
+ lockdep_assert_held(&guc->submission_state.lock);
ret = new_guc_id(guc);
if (unlikely(ret < 0)) {
@@ -1295,7 +1296,7 @@ static int pin_guc_id(struct intel_guc *guc, struct intel_context *ce)
GEM_BUG_ON(atomic_read(&ce->guc_id.ref));
try_again:
- spin_lock_irqsave(&guc->contexts_lock, flags);
+ spin_lock_irqsave(&guc->submission_state.lock, flags);
might_lock(&ce->guc_state.lock);
@@ -1310,7 +1311,7 @@ static int pin_guc_id(struct intel_guc *guc, struct intel_context *ce)
atomic_inc(&ce->guc_id.ref);
out_unlock:
- spin_unlock_irqrestore(&guc->contexts_lock, flags);
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
/*
* -EAGAIN indicates no guc_id are available, let's retire any
@@ -1346,11 +1347,12 @@ static void unpin_guc_id(struct intel_guc *guc, struct intel_context *ce)
if (unlikely(context_guc_id_invalid(ce)))
return;
- spin_lock_irqsave(&guc->contexts_lock, flags);
+ spin_lock_irqsave(&guc->submission_state.lock, flags);
if (!context_guc_id_invalid(ce) && list_empty(&ce->guc_id.link) &&
!atomic_read(&ce->guc_id.ref))
- list_add_tail(&ce->guc_id.link, &guc->guc_id_list);
- spin_unlock_irqrestore(&guc->contexts_lock, flags);
+ list_add_tail(&ce->guc_id.link,
+ &guc->submission_state.guc_id_list);
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
}
static int __guc_action_register_context(struct intel_guc *guc,
@@ -1921,16 +1923,16 @@ static void guc_context_destroy(struct kref *kref)
* returns indicating this context has been deregistered the guc_id is
* returned to the pool of available guc_id.
*/
- spin_lock_irqsave(&guc->contexts_lock, flags);
+ spin_lock_irqsave(&guc->submission_state.lock, flags);
if (context_guc_id_invalid(ce)) {
- spin_unlock_irqrestore(&guc->contexts_lock, flags);
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
__guc_context_destroy(ce);
return;
}
if (!list_empty(&ce->guc_id.link))
list_del_init(&ce->guc_id.link);
- spin_unlock_irqrestore(&guc->contexts_lock, flags);
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
/* Seal race with Reset */
spin_lock_irqsave(&ce->guc_state.lock, flags);
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 02/25] drm/i915/guc: Take GT PM ref when deregistering context
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Taking a PM reference to prevent intel_gt_wait_for_idle from short
circuiting while a deregister context H2G is in flight. To do this must
issue the deregister H2G from a worker as context can be destroyed from
an atomic context and taking GT PM ref blows up. Previously we took a
runtime PM from this atomic context which worked but will stop working
once runtime pm autosuspend in enabled.
So this patch is two fold, stop intel_gt_wait_for_idle from short
circuting and fix runtime pm autosuspend.
v2:
(John Harrison)
- Split structure changes out in different patch
(Tvrtko)
- Don't drop lock in deregister_destroyed_contexts
v3:
(John Harrison)
- Flush destroyed contexts before destroying context reg pool
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.c | 2 +
drivers/gpu/drm/i915/gt/intel_context_types.h | 7 +
drivers/gpu/drm/i915/gt/intel_engine_pm.h | 5 +
drivers/gpu/drm/i915/gt/intel_gt_pm.h | 4 +
drivers/gpu/drm/i915/gt/uc/intel_guc.h | 11 ++
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 146 +++++++++++-------
6 files changed, 121 insertions(+), 54 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index 35babd02ddfe..d008ef8623ce 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -400,6 +400,8 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)
ce->guc_id.id = GUC_INVALID_LRC_ID;
INIT_LIST_HEAD(&ce->guc_id.link);
+ INIT_LIST_HEAD(&ce->destroyed_link);
+
/*
* Initialize fence to be complete as this is expected to be complete
* unless there is a pending schedule disable outstanding.
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index e7e3984aab78..4613d027cbc3 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -213,6 +213,13 @@ struct intel_context {
struct list_head link;
} guc_id;
+ /**
+ * @destroyed_link: link in guc->submission_state.destroyed_contexts, in
+ * list when context is pending to be destroyed (deregistered with the
+ * GuC), protected by guc->submission_state.lock
+ */
+ struct list_head destroyed_link;
+
#ifdef CONFIG_DRM_I915_SELFTEST
/**
* @drop_schedule_enable: Force drop of schedule enable G2H for selftest
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.h b/drivers/gpu/drm/i915/gt/intel_engine_pm.h
index 8520c595f5e1..6fdeae668e6e 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_pm.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.h
@@ -16,6 +16,11 @@ intel_engine_pm_is_awake(const struct intel_engine_cs *engine)
return intel_wakeref_is_active(&engine->wakeref);
}
+static inline void __intel_engine_pm_get(struct intel_engine_cs *engine)
+{
+ __intel_wakeref_get(&engine->wakeref);
+}
+
static inline void intel_engine_pm_get(struct intel_engine_cs *engine)
{
intel_wakeref_get(&engine->wakeref);
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_pm.h b/drivers/gpu/drm/i915/gt/intel_gt_pm.h
index d0588d8aaa44..05de6c1af25b 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_pm.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_pm.h
@@ -41,6 +41,10 @@ static inline void intel_gt_pm_put_async(struct intel_gt *gt)
intel_wakeref_put_async(>->wakeref);
}
+#define with_intel_gt_pm(gt, tmp) \
+ for (tmp = 1, intel_gt_pm_get(gt); tmp; \
+ intel_gt_pm_put(gt), tmp = 0)
+
static inline int intel_gt_pm_wait_for_idle(struct intel_gt *gt)
{
return intel_wakeref_wait_for_idle(>->wakeref);
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
index 82e248c2290c..74f071a0b6d5 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
@@ -90,6 +90,17 @@ struct intel_guc {
* refs
*/
struct list_head guc_id_list;
+ /**
+ * @destroyed_contexts: list of contexts waiting to be destroyed
+ * (deregistered with the GuC)
+ */
+ struct list_head destroyed_contexts;
+ /**
+ * @destroyed_worker: worker to deregister contexts, need as we
+ * need to take a GT PM reference and can't from destroy
+ * function as it might be in an atomic context (no sleeping)
+ */
+ struct work_struct destroyed_worker;
} submission_state;
/**
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 48bdcdb823b3..5c53428032b5 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -90,8 +90,8 @@
* used for all of GuC submission but that could change in the future.
*
* guc->submission_state.lock
- * Protects guc_id allocation for the given GuC, i.e. only one context can be
- * doing guc_id allocation operations at a time for each GuC in the system.
+ * Global lock for GuC submission state. Protects guc_ids and destroyed contexts
+ * list.
*
* ce->guc_state.lock
* Protects everything under ce->guc_state. Ensures that a context is in the
@@ -719,6 +719,7 @@ static void scrub_guc_desc_for_outstanding_g2h(struct intel_guc *guc)
if (deregister)
guc_signal_context_fence(ce);
if (destroyed) {
+ intel_gt_pm_put_async(guc_to_gt(guc));
release_guc_id(guc, ce);
__guc_context_destroy(ce);
}
@@ -797,6 +798,8 @@ static void guc_flush_submissions(struct intel_guc *guc)
spin_unlock_irqrestore(&sched_engine->lock, flags);
}
+static void guc_flush_destroyed_contexts(struct intel_guc *guc);
+
void intel_guc_submission_reset_prepare(struct intel_guc *guc)
{
int i;
@@ -815,6 +818,7 @@ void intel_guc_submission_reset_prepare(struct intel_guc *guc)
spin_unlock_irq(&guc_to_gt(guc)->irq_lock);
guc_flush_submissions(guc);
+ guc_flush_destroyed_contexts(guc);
/*
* Handle any outstanding G2Hs before reset. Call IRQ handler directly
@@ -1126,6 +1130,8 @@ void intel_guc_submission_reset_finish(struct intel_guc *guc)
intel_gt_unpark_heartbeats(guc_to_gt(guc));
}
+static void destroyed_worker_func(struct work_struct *w);
+
/*
* Set up the memory resources to be shared with the GuC (via the GGTT)
* at firmware loading time.
@@ -1151,6 +1157,9 @@ int intel_guc_submission_init(struct intel_guc *guc)
spin_lock_init(&guc->submission_state.lock);
INIT_LIST_HEAD(&guc->submission_state.guc_id_list);
ida_init(&guc->submission_state.guc_ids);
+ INIT_LIST_HEAD(&guc->submission_state.destroyed_contexts);
+ INIT_WORK(&guc->submission_state.destroyed_worker,
+ destroyed_worker_func);
return 0;
}
@@ -1160,6 +1169,7 @@ void intel_guc_submission_fini(struct intel_guc *guc)
if (!guc->lrc_desc_pool)
return;
+ guc_flush_destroyed_contexts(guc);
guc_lrc_desc_pool_destroy(guc);
i915_sched_engine_put(guc->sched_engine);
}
@@ -1859,11 +1869,30 @@ static void guc_context_sched_disable(struct intel_context *ce)
static inline void guc_lrc_desc_unpin(struct intel_context *ce)
{
struct intel_guc *guc = ce_to_guc(ce);
+ struct intel_gt *gt = guc_to_gt(guc);
+ unsigned long flags;
+ bool disabled;
+ GEM_BUG_ON(!intel_gt_pm_is_awake(gt));
GEM_BUG_ON(!lrc_desc_registered(guc, ce->guc_id.id));
GEM_BUG_ON(ce != __get_context(guc, ce->guc_id.id));
GEM_BUG_ON(context_enabled(ce));
+ /* Seal race with Reset */
+ spin_lock_irqsave(&ce->guc_state.lock, flags);
+ disabled = submission_disabled(guc);
+ if (likely(!disabled)) {
+ __intel_gt_pm_get(gt);
+ set_context_destroyed(ce);
+ clr_context_registered(ce);
+ }
+ spin_unlock_irqrestore(&ce->guc_state.lock, flags);
+ if (unlikely(disabled)) {
+ release_guc_id(guc, ce);
+ __guc_context_destroy(ce);
+ return;
+ }
+
deregister_context(ce, ce->guc_id.id);
}
@@ -1891,78 +1920,86 @@ static void __guc_context_destroy(struct intel_context *ce)
}
}
+static void guc_flush_destroyed_contexts(struct intel_guc *guc)
+{
+ struct intel_context *ce, *cn;
+ unsigned long flags;
+
+ GEM_BUG_ON(!submission_disabled(guc) &&
+ guc_submission_initialized(guc));
+
+ spin_lock_irqsave(&guc->submission_state.lock, flags);
+ list_for_each_entry_safe(ce, cn,
+ &guc->submission_state.destroyed_contexts,
+ destroyed_link) {
+ list_del_init(&ce->destroyed_link);
+ __release_guc_id(guc, ce);
+ __guc_context_destroy(ce);
+ }
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
+}
+
+static void deregister_destroyed_contexts(struct intel_guc *guc)
+{
+ struct intel_context *ce, *cn;
+ unsigned long flags;
+
+ spin_lock_irqsave(&guc->submission_state.lock, flags);
+ list_for_each_entry_safe(ce, cn,
+ &guc->submission_state.destroyed_contexts,
+ destroyed_link) {
+ list_del_init(&ce->destroyed_link);
+ guc_lrc_desc_unpin(ce);
+ }
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
+}
+
+static void destroyed_worker_func(struct work_struct *w)
+{
+ struct intel_guc *guc = container_of(w, struct intel_guc,
+ submission_state.destroyed_worker);
+ struct intel_gt *gt = guc_to_gt(guc);
+ int tmp;
+
+ with_intel_gt_pm(gt, tmp)
+ deregister_destroyed_contexts(guc);
+}
+
static void guc_context_destroy(struct kref *kref)
{
struct intel_context *ce = container_of(kref, typeof(*ce), ref);
- struct intel_runtime_pm *runtime_pm = ce->engine->uncore->rpm;
struct intel_guc *guc = ce_to_guc(ce);
- intel_wakeref_t wakeref;
unsigned long flags;
- bool disabled;
+ bool destroy;
/*
* If the guc_id is invalid this context has been stolen and we can free
* it immediately. Also can be freed immediately if the context is not
* registered with the GuC or the GuC is in the middle of a reset.
*/
- if (context_guc_id_invalid(ce)) {
- __guc_context_destroy(ce);
- return;
- } else if (submission_disabled(guc) ||
- !lrc_desc_registered(guc, ce->guc_id.id)) {
- release_guc_id(guc, ce);
- __guc_context_destroy(ce);
- return;
- }
-
- /*
- * We have to acquire the context spinlock and check guc_id again, if it
- * is valid it hasn't been stolen and needs to be deregistered. We
- * delete this context from the list of unpinned guc_id available to
- * steal to seal a race with guc_lrc_desc_pin(). When the G2H CTB
- * returns indicating this context has been deregistered the guc_id is
- * returned to the pool of available guc_id.
- */
spin_lock_irqsave(&guc->submission_state.lock, flags);
- if (context_guc_id_invalid(ce)) {
- spin_unlock_irqrestore(&guc->submission_state.lock, flags);
- __guc_context_destroy(ce);
- return;
+ destroy = submission_disabled(guc) || context_guc_id_invalid(ce) ||
+ !lrc_desc_registered(guc, ce->guc_id.id);
+ if (likely(!destroy)) {
+ if (!list_empty(&ce->guc_id.link))
+ list_del_init(&ce->guc_id.link);
+ list_add_tail(&ce->destroyed_link,
+ &guc->submission_state.destroyed_contexts);
+ } else {
+ __release_guc_id(guc, ce);
}
-
- if (!list_empty(&ce->guc_id.link))
- list_del_init(&ce->guc_id.link);
spin_unlock_irqrestore(&guc->submission_state.lock, flags);
-
- /* Seal race with Reset */
- spin_lock_irqsave(&ce->guc_state.lock, flags);
- disabled = submission_disabled(guc);
- if (likely(!disabled)) {
- set_context_destroyed(ce);
- clr_context_registered(ce);
- }
- spin_unlock_irqrestore(&ce->guc_state.lock, flags);
- if (unlikely(disabled)) {
- release_guc_id(guc, ce);
+ if (unlikely(destroy)) {
__guc_context_destroy(ce);
return;
}
/*
- * We defer GuC context deregistration until the context is destroyed
- * in order to save on CTBs. With this optimization ideally we only need
- * 1 CTB to register the context during the first pin and 1 CTB to
- * deregister the context when the context is destroyed. Without this
- * optimization, a CTB would be needed every pin & unpin.
- *
- * XXX: Need to acqiure the runtime wakeref as this can be triggered
- * from context_free_worker when runtime wakeref is not held.
- * guc_lrc_desc_unpin requires the runtime as a GuC register is written
- * in H2G CTB to deregister the context. A future patch may defer this
- * H2G CTB if the runtime wakeref is zero.
+ * We use a worker to issue the H2G to deregister the context as we can
+ * take the GT PM for the first time which isn't allowed from an atomic
+ * context.
*/
- with_intel_runtime_pm(runtime_pm, wakeref)
- guc_lrc_desc_unpin(ce);
+ queue_work(system_unbound_wq, &guc->submission_state.destroyed_worker);
}
static int guc_context_alloc(struct intel_context *ce)
@@ -2798,6 +2835,7 @@ int intel_guc_deregister_done_process_msg(struct intel_guc *guc,
intel_context_put(ce);
} else if (context_destroyed(ce)) {
/* Context has been destroyed */
+ intel_gt_pm_put_async(guc_to_gt(guc));
release_guc_id(guc, ce);
__guc_context_destroy(ce);
}
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 02/25] drm/i915/guc: Take GT PM ref when deregistering context
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Taking a PM reference to prevent intel_gt_wait_for_idle from short
circuiting while a deregister context H2G is in flight. To do this must
issue the deregister H2G from a worker as context can be destroyed from
an atomic context and taking GT PM ref blows up. Previously we took a
runtime PM from this atomic context which worked but will stop working
once runtime pm autosuspend in enabled.
So this patch is two fold, stop intel_gt_wait_for_idle from short
circuting and fix runtime pm autosuspend.
v2:
(John Harrison)
- Split structure changes out in different patch
(Tvrtko)
- Don't drop lock in deregister_destroyed_contexts
v3:
(John Harrison)
- Flush destroyed contexts before destroying context reg pool
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.c | 2 +
drivers/gpu/drm/i915/gt/intel_context_types.h | 7 +
drivers/gpu/drm/i915/gt/intel_engine_pm.h | 5 +
drivers/gpu/drm/i915/gt/intel_gt_pm.h | 4 +
drivers/gpu/drm/i915/gt/uc/intel_guc.h | 11 ++
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 146 +++++++++++-------
6 files changed, 121 insertions(+), 54 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index 35babd02ddfe..d008ef8623ce 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -400,6 +400,8 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)
ce->guc_id.id = GUC_INVALID_LRC_ID;
INIT_LIST_HEAD(&ce->guc_id.link);
+ INIT_LIST_HEAD(&ce->destroyed_link);
+
/*
* Initialize fence to be complete as this is expected to be complete
* unless there is a pending schedule disable outstanding.
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index e7e3984aab78..4613d027cbc3 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -213,6 +213,13 @@ struct intel_context {
struct list_head link;
} guc_id;
+ /**
+ * @destroyed_link: link in guc->submission_state.destroyed_contexts, in
+ * list when context is pending to be destroyed (deregistered with the
+ * GuC), protected by guc->submission_state.lock
+ */
+ struct list_head destroyed_link;
+
#ifdef CONFIG_DRM_I915_SELFTEST
/**
* @drop_schedule_enable: Force drop of schedule enable G2H for selftest
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.h b/drivers/gpu/drm/i915/gt/intel_engine_pm.h
index 8520c595f5e1..6fdeae668e6e 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_pm.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.h
@@ -16,6 +16,11 @@ intel_engine_pm_is_awake(const struct intel_engine_cs *engine)
return intel_wakeref_is_active(&engine->wakeref);
}
+static inline void __intel_engine_pm_get(struct intel_engine_cs *engine)
+{
+ __intel_wakeref_get(&engine->wakeref);
+}
+
static inline void intel_engine_pm_get(struct intel_engine_cs *engine)
{
intel_wakeref_get(&engine->wakeref);
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_pm.h b/drivers/gpu/drm/i915/gt/intel_gt_pm.h
index d0588d8aaa44..05de6c1af25b 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_pm.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_pm.h
@@ -41,6 +41,10 @@ static inline void intel_gt_pm_put_async(struct intel_gt *gt)
intel_wakeref_put_async(>->wakeref);
}
+#define with_intel_gt_pm(gt, tmp) \
+ for (tmp = 1, intel_gt_pm_get(gt); tmp; \
+ intel_gt_pm_put(gt), tmp = 0)
+
static inline int intel_gt_pm_wait_for_idle(struct intel_gt *gt)
{
return intel_wakeref_wait_for_idle(>->wakeref);
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
index 82e248c2290c..74f071a0b6d5 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
@@ -90,6 +90,17 @@ struct intel_guc {
* refs
*/
struct list_head guc_id_list;
+ /**
+ * @destroyed_contexts: list of contexts waiting to be destroyed
+ * (deregistered with the GuC)
+ */
+ struct list_head destroyed_contexts;
+ /**
+ * @destroyed_worker: worker to deregister contexts, need as we
+ * need to take a GT PM reference and can't from destroy
+ * function as it might be in an atomic context (no sleeping)
+ */
+ struct work_struct destroyed_worker;
} submission_state;
/**
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 48bdcdb823b3..5c53428032b5 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -90,8 +90,8 @@
* used for all of GuC submission but that could change in the future.
*
* guc->submission_state.lock
- * Protects guc_id allocation for the given GuC, i.e. only one context can be
- * doing guc_id allocation operations at a time for each GuC in the system.
+ * Global lock for GuC submission state. Protects guc_ids and destroyed contexts
+ * list.
*
* ce->guc_state.lock
* Protects everything under ce->guc_state. Ensures that a context is in the
@@ -719,6 +719,7 @@ static void scrub_guc_desc_for_outstanding_g2h(struct intel_guc *guc)
if (deregister)
guc_signal_context_fence(ce);
if (destroyed) {
+ intel_gt_pm_put_async(guc_to_gt(guc));
release_guc_id(guc, ce);
__guc_context_destroy(ce);
}
@@ -797,6 +798,8 @@ static void guc_flush_submissions(struct intel_guc *guc)
spin_unlock_irqrestore(&sched_engine->lock, flags);
}
+static void guc_flush_destroyed_contexts(struct intel_guc *guc);
+
void intel_guc_submission_reset_prepare(struct intel_guc *guc)
{
int i;
@@ -815,6 +818,7 @@ void intel_guc_submission_reset_prepare(struct intel_guc *guc)
spin_unlock_irq(&guc_to_gt(guc)->irq_lock);
guc_flush_submissions(guc);
+ guc_flush_destroyed_contexts(guc);
/*
* Handle any outstanding G2Hs before reset. Call IRQ handler directly
@@ -1126,6 +1130,8 @@ void intel_guc_submission_reset_finish(struct intel_guc *guc)
intel_gt_unpark_heartbeats(guc_to_gt(guc));
}
+static void destroyed_worker_func(struct work_struct *w);
+
/*
* Set up the memory resources to be shared with the GuC (via the GGTT)
* at firmware loading time.
@@ -1151,6 +1157,9 @@ int intel_guc_submission_init(struct intel_guc *guc)
spin_lock_init(&guc->submission_state.lock);
INIT_LIST_HEAD(&guc->submission_state.guc_id_list);
ida_init(&guc->submission_state.guc_ids);
+ INIT_LIST_HEAD(&guc->submission_state.destroyed_contexts);
+ INIT_WORK(&guc->submission_state.destroyed_worker,
+ destroyed_worker_func);
return 0;
}
@@ -1160,6 +1169,7 @@ void intel_guc_submission_fini(struct intel_guc *guc)
if (!guc->lrc_desc_pool)
return;
+ guc_flush_destroyed_contexts(guc);
guc_lrc_desc_pool_destroy(guc);
i915_sched_engine_put(guc->sched_engine);
}
@@ -1859,11 +1869,30 @@ static void guc_context_sched_disable(struct intel_context *ce)
static inline void guc_lrc_desc_unpin(struct intel_context *ce)
{
struct intel_guc *guc = ce_to_guc(ce);
+ struct intel_gt *gt = guc_to_gt(guc);
+ unsigned long flags;
+ bool disabled;
+ GEM_BUG_ON(!intel_gt_pm_is_awake(gt));
GEM_BUG_ON(!lrc_desc_registered(guc, ce->guc_id.id));
GEM_BUG_ON(ce != __get_context(guc, ce->guc_id.id));
GEM_BUG_ON(context_enabled(ce));
+ /* Seal race with Reset */
+ spin_lock_irqsave(&ce->guc_state.lock, flags);
+ disabled = submission_disabled(guc);
+ if (likely(!disabled)) {
+ __intel_gt_pm_get(gt);
+ set_context_destroyed(ce);
+ clr_context_registered(ce);
+ }
+ spin_unlock_irqrestore(&ce->guc_state.lock, flags);
+ if (unlikely(disabled)) {
+ release_guc_id(guc, ce);
+ __guc_context_destroy(ce);
+ return;
+ }
+
deregister_context(ce, ce->guc_id.id);
}
@@ -1891,78 +1920,86 @@ static void __guc_context_destroy(struct intel_context *ce)
}
}
+static void guc_flush_destroyed_contexts(struct intel_guc *guc)
+{
+ struct intel_context *ce, *cn;
+ unsigned long flags;
+
+ GEM_BUG_ON(!submission_disabled(guc) &&
+ guc_submission_initialized(guc));
+
+ spin_lock_irqsave(&guc->submission_state.lock, flags);
+ list_for_each_entry_safe(ce, cn,
+ &guc->submission_state.destroyed_contexts,
+ destroyed_link) {
+ list_del_init(&ce->destroyed_link);
+ __release_guc_id(guc, ce);
+ __guc_context_destroy(ce);
+ }
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
+}
+
+static void deregister_destroyed_contexts(struct intel_guc *guc)
+{
+ struct intel_context *ce, *cn;
+ unsigned long flags;
+
+ spin_lock_irqsave(&guc->submission_state.lock, flags);
+ list_for_each_entry_safe(ce, cn,
+ &guc->submission_state.destroyed_contexts,
+ destroyed_link) {
+ list_del_init(&ce->destroyed_link);
+ guc_lrc_desc_unpin(ce);
+ }
+ spin_unlock_irqrestore(&guc->submission_state.lock, flags);
+}
+
+static void destroyed_worker_func(struct work_struct *w)
+{
+ struct intel_guc *guc = container_of(w, struct intel_guc,
+ submission_state.destroyed_worker);
+ struct intel_gt *gt = guc_to_gt(guc);
+ int tmp;
+
+ with_intel_gt_pm(gt, tmp)
+ deregister_destroyed_contexts(guc);
+}
+
static void guc_context_destroy(struct kref *kref)
{
struct intel_context *ce = container_of(kref, typeof(*ce), ref);
- struct intel_runtime_pm *runtime_pm = ce->engine->uncore->rpm;
struct intel_guc *guc = ce_to_guc(ce);
- intel_wakeref_t wakeref;
unsigned long flags;
- bool disabled;
+ bool destroy;
/*
* If the guc_id is invalid this context has been stolen and we can free
* it immediately. Also can be freed immediately if the context is not
* registered with the GuC or the GuC is in the middle of a reset.
*/
- if (context_guc_id_invalid(ce)) {
- __guc_context_destroy(ce);
- return;
- } else if (submission_disabled(guc) ||
- !lrc_desc_registered(guc, ce->guc_id.id)) {
- release_guc_id(guc, ce);
- __guc_context_destroy(ce);
- return;
- }
-
- /*
- * We have to acquire the context spinlock and check guc_id again, if it
- * is valid it hasn't been stolen and needs to be deregistered. We
- * delete this context from the list of unpinned guc_id available to
- * steal to seal a race with guc_lrc_desc_pin(). When the G2H CTB
- * returns indicating this context has been deregistered the guc_id is
- * returned to the pool of available guc_id.
- */
spin_lock_irqsave(&guc->submission_state.lock, flags);
- if (context_guc_id_invalid(ce)) {
- spin_unlock_irqrestore(&guc->submission_state.lock, flags);
- __guc_context_destroy(ce);
- return;
+ destroy = submission_disabled(guc) || context_guc_id_invalid(ce) ||
+ !lrc_desc_registered(guc, ce->guc_id.id);
+ if (likely(!destroy)) {
+ if (!list_empty(&ce->guc_id.link))
+ list_del_init(&ce->guc_id.link);
+ list_add_tail(&ce->destroyed_link,
+ &guc->submission_state.destroyed_contexts);
+ } else {
+ __release_guc_id(guc, ce);
}
-
- if (!list_empty(&ce->guc_id.link))
- list_del_init(&ce->guc_id.link);
spin_unlock_irqrestore(&guc->submission_state.lock, flags);
-
- /* Seal race with Reset */
- spin_lock_irqsave(&ce->guc_state.lock, flags);
- disabled = submission_disabled(guc);
- if (likely(!disabled)) {
- set_context_destroyed(ce);
- clr_context_registered(ce);
- }
- spin_unlock_irqrestore(&ce->guc_state.lock, flags);
- if (unlikely(disabled)) {
- release_guc_id(guc, ce);
+ if (unlikely(destroy)) {
__guc_context_destroy(ce);
return;
}
/*
- * We defer GuC context deregistration until the context is destroyed
- * in order to save on CTBs. With this optimization ideally we only need
- * 1 CTB to register the context during the first pin and 1 CTB to
- * deregister the context when the context is destroyed. Without this
- * optimization, a CTB would be needed every pin & unpin.
- *
- * XXX: Need to acqiure the runtime wakeref as this can be triggered
- * from context_free_worker when runtime wakeref is not held.
- * guc_lrc_desc_unpin requires the runtime as a GuC register is written
- * in H2G CTB to deregister the context. A future patch may defer this
- * H2G CTB if the runtime wakeref is zero.
+ * We use a worker to issue the H2G to deregister the context as we can
+ * take the GT PM for the first time which isn't allowed from an atomic
+ * context.
*/
- with_intel_runtime_pm(runtime_pm, wakeref)
- guc_lrc_desc_unpin(ce);
+ queue_work(system_unbound_wq, &guc->submission_state.destroyed_worker);
}
static int guc_context_alloc(struct intel_context *ce)
@@ -2798,6 +2835,7 @@ int intel_guc_deregister_done_process_msg(struct intel_guc *guc,
intel_context_put(ce);
} else if (context_destroyed(ce)) {
/* Context has been destroyed */
+ intel_gt_pm_put_async(guc_to_gt(guc));
release_guc_id(guc, ce);
__guc_context_destroy(ce);
}
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 03/25] drm/i915/guc: Take engine PM when a context is pinned with GuC submission
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Taking a PM reference to prevent intel_gt_wait_for_idle from short
circuiting while any user context has scheduling enabled. Returning GT
idle when it is not can cause all sorts of issues throughout the stack.
v2:
(Daniel Vetter)
- Add might_lock annotations to pin / unpin function
v3:
(CI)
- Drop intel_engine_pm_might_put from unpin path as an async put is
used
v4:
(John Harrison)
- Make intel_engine_pm_might_get/put work with GuC virtual engines
- Update commit message
v5:
- Update commit message again
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.c | 2 ++
drivers/gpu/drm/i915/gt/intel_engine_pm.h | 32 +++++++++++++++++
drivers/gpu/drm/i915/gt/intel_gt_pm.h | 10 ++++++
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 36 +++++++++++++++++--
drivers/gpu/drm/i915/intel_wakeref.h | 12 +++++++
5 files changed, 89 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index d008ef8623ce..f98c9f470ba1 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -240,6 +240,8 @@ int __intel_context_do_pin_ww(struct intel_context *ce,
if (err)
goto err_post_unpin;
+ intel_engine_pm_might_get(ce->engine);
+
if (unlikely(intel_context_is_closed(ce))) {
err = -ENOENT;
goto err_unlock;
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.h b/drivers/gpu/drm/i915/gt/intel_engine_pm.h
index 6fdeae668e6e..d68675925b79 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_pm.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.h
@@ -6,9 +6,11 @@
#ifndef INTEL_ENGINE_PM_H
#define INTEL_ENGINE_PM_H
+#include "i915_drv.h"
#include "i915_request.h"
#include "intel_engine_types.h"
#include "intel_wakeref.h"
+#include "intel_gt_pm.h"
static inline bool
intel_engine_pm_is_awake(const struct intel_engine_cs *engine)
@@ -31,6 +33,21 @@ static inline bool intel_engine_pm_get_if_awake(struct intel_engine_cs *engine)
return intel_wakeref_get_if_active(&engine->wakeref);
}
+static inline void intel_engine_pm_might_get(struct intel_engine_cs *engine)
+{
+ if (!intel_engine_is_virtual(engine)) {
+ intel_wakeref_might_get(&engine->wakeref);
+ } else {
+ struct intel_gt *gt = engine->gt;
+ struct intel_engine_cs *tengine;
+ intel_engine_mask_t tmp, mask = engine->mask;
+
+ for_each_engine_masked(tengine, gt, mask, tmp)
+ intel_wakeref_might_get(&tengine->wakeref);
+ }
+ intel_gt_pm_might_get(engine->gt);
+}
+
static inline void intel_engine_pm_put(struct intel_engine_cs *engine)
{
intel_wakeref_put(&engine->wakeref);
@@ -52,6 +69,21 @@ static inline void intel_engine_pm_flush(struct intel_engine_cs *engine)
intel_wakeref_unlock_wait(&engine->wakeref);
}
+static inline void intel_engine_pm_might_put(struct intel_engine_cs *engine)
+{
+ if (!intel_engine_is_virtual(engine)) {
+ intel_wakeref_might_put(&engine->wakeref);
+ } else {
+ struct intel_gt *gt = engine->gt;
+ struct intel_engine_cs *tengine;
+ intel_engine_mask_t tmp, mask = engine->mask;
+
+ for_each_engine_masked(tengine, gt, mask, tmp)
+ intel_wakeref_might_put(&tengine->wakeref);
+ }
+ intel_gt_pm_might_put(engine->gt);
+}
+
static inline struct i915_request *
intel_engine_create_kernel_request(struct intel_engine_cs *engine)
{
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_pm.h b/drivers/gpu/drm/i915/gt/intel_gt_pm.h
index 05de6c1af25b..bc898df7a48c 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_pm.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_pm.h
@@ -31,6 +31,11 @@ static inline bool intel_gt_pm_get_if_awake(struct intel_gt *gt)
return intel_wakeref_get_if_active(>->wakeref);
}
+static inline void intel_gt_pm_might_get(struct intel_gt *gt)
+{
+ intel_wakeref_might_get(>->wakeref);
+}
+
static inline void intel_gt_pm_put(struct intel_gt *gt)
{
intel_wakeref_put(>->wakeref);
@@ -41,6 +46,11 @@ static inline void intel_gt_pm_put_async(struct intel_gt *gt)
intel_wakeref_put_async(>->wakeref);
}
+static inline void intel_gt_pm_might_put(struct intel_gt *gt)
+{
+ intel_wakeref_might_put(>->wakeref);
+}
+
#define with_intel_gt_pm(gt, tmp) \
for (tmp = 1, intel_gt_pm_get(gt); tmp; \
intel_gt_pm_put(gt), tmp = 0)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 5c53428032b5..795cc071d05d 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -1571,7 +1571,12 @@ static int guc_context_pre_pin(struct intel_context *ce,
static int guc_context_pin(struct intel_context *ce, void *vaddr)
{
- return __guc_context_pin(ce, ce->engine, vaddr);
+ int ret = __guc_context_pin(ce, ce->engine, vaddr);
+
+ if (likely(!ret && !intel_context_is_barrier(ce)))
+ intel_engine_pm_get(ce->engine);
+
+ return ret;
}
static void guc_context_unpin(struct intel_context *ce)
@@ -1580,6 +1585,9 @@ static void guc_context_unpin(struct intel_context *ce)
unpin_guc_id(guc, ce);
lrc_unpin(ce);
+
+ if (likely(!intel_context_is_barrier(ce)))
+ intel_engine_pm_put_async(ce->engine);
}
static void guc_context_post_unpin(struct intel_context *ce)
@@ -2341,8 +2349,30 @@ static int guc_virtual_context_pre_pin(struct intel_context *ce,
static int guc_virtual_context_pin(struct intel_context *ce, void *vaddr)
{
struct intel_engine_cs *engine = guc_virtual_get_sibling(ce->engine, 0);
+ int ret = __guc_context_pin(ce, engine, vaddr);
+ intel_engine_mask_t tmp, mask = ce->engine->mask;
+
+ if (likely(!ret))
+ for_each_engine_masked(engine, ce->engine->gt, mask, tmp)
+ intel_engine_pm_get(engine);
- return __guc_context_pin(ce, engine, vaddr);
+ return ret;
+}
+
+static void guc_virtual_context_unpin(struct intel_context *ce)
+{
+ intel_engine_mask_t tmp, mask = ce->engine->mask;
+ struct intel_engine_cs *engine;
+ struct intel_guc *guc = ce_to_guc(ce);
+
+ GEM_BUG_ON(context_enabled(ce));
+ GEM_BUG_ON(intel_context_is_barrier(ce));
+
+ unpin_guc_id(guc, ce);
+ lrc_unpin(ce);
+
+ for_each_engine_masked(engine, ce->engine->gt, mask, tmp)
+ intel_engine_pm_put_async(engine);
}
static void guc_virtual_context_enter(struct intel_context *ce)
@@ -2379,7 +2409,7 @@ static const struct intel_context_ops virtual_guc_context_ops = {
.pre_pin = guc_virtual_context_pre_pin,
.pin = guc_virtual_context_pin,
- .unpin = guc_context_unpin,
+ .unpin = guc_virtual_context_unpin,
.post_unpin = guc_context_post_unpin,
.ban = guc_context_ban,
diff --git a/drivers/gpu/drm/i915/intel_wakeref.h b/drivers/gpu/drm/i915/intel_wakeref.h
index 545c8f277c46..4f4c2e15e736 100644
--- a/drivers/gpu/drm/i915/intel_wakeref.h
+++ b/drivers/gpu/drm/i915/intel_wakeref.h
@@ -123,6 +123,12 @@ enum {
__INTEL_WAKEREF_PUT_LAST_BIT__
};
+static inline void
+intel_wakeref_might_get(struct intel_wakeref *wf)
+{
+ might_lock(&wf->mutex);
+}
+
/**
* intel_wakeref_put_flags: Release the wakeref
* @wf: the wakeref
@@ -170,6 +176,12 @@ intel_wakeref_put_delay(struct intel_wakeref *wf, unsigned long delay)
FIELD_PREP(INTEL_WAKEREF_PUT_DELAY, delay));
}
+static inline void
+intel_wakeref_might_put(struct intel_wakeref *wf)
+{
+ might_lock(&wf->mutex);
+}
+
/**
* intel_wakeref_lock: Lock the wakeref (mutex)
* @wf: the wakeref
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 03/25] drm/i915/guc: Take engine PM when a context is pinned with GuC submission
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Taking a PM reference to prevent intel_gt_wait_for_idle from short
circuiting while any user context has scheduling enabled. Returning GT
idle when it is not can cause all sorts of issues throughout the stack.
v2:
(Daniel Vetter)
- Add might_lock annotations to pin / unpin function
v3:
(CI)
- Drop intel_engine_pm_might_put from unpin path as an async put is
used
v4:
(John Harrison)
- Make intel_engine_pm_might_get/put work with GuC virtual engines
- Update commit message
v5:
- Update commit message again
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.c | 2 ++
drivers/gpu/drm/i915/gt/intel_engine_pm.h | 32 +++++++++++++++++
drivers/gpu/drm/i915/gt/intel_gt_pm.h | 10 ++++++
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 36 +++++++++++++++++--
drivers/gpu/drm/i915/intel_wakeref.h | 12 +++++++
5 files changed, 89 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index d008ef8623ce..f98c9f470ba1 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -240,6 +240,8 @@ int __intel_context_do_pin_ww(struct intel_context *ce,
if (err)
goto err_post_unpin;
+ intel_engine_pm_might_get(ce->engine);
+
if (unlikely(intel_context_is_closed(ce))) {
err = -ENOENT;
goto err_unlock;
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.h b/drivers/gpu/drm/i915/gt/intel_engine_pm.h
index 6fdeae668e6e..d68675925b79 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_pm.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.h
@@ -6,9 +6,11 @@
#ifndef INTEL_ENGINE_PM_H
#define INTEL_ENGINE_PM_H
+#include "i915_drv.h"
#include "i915_request.h"
#include "intel_engine_types.h"
#include "intel_wakeref.h"
+#include "intel_gt_pm.h"
static inline bool
intel_engine_pm_is_awake(const struct intel_engine_cs *engine)
@@ -31,6 +33,21 @@ static inline bool intel_engine_pm_get_if_awake(struct intel_engine_cs *engine)
return intel_wakeref_get_if_active(&engine->wakeref);
}
+static inline void intel_engine_pm_might_get(struct intel_engine_cs *engine)
+{
+ if (!intel_engine_is_virtual(engine)) {
+ intel_wakeref_might_get(&engine->wakeref);
+ } else {
+ struct intel_gt *gt = engine->gt;
+ struct intel_engine_cs *tengine;
+ intel_engine_mask_t tmp, mask = engine->mask;
+
+ for_each_engine_masked(tengine, gt, mask, tmp)
+ intel_wakeref_might_get(&tengine->wakeref);
+ }
+ intel_gt_pm_might_get(engine->gt);
+}
+
static inline void intel_engine_pm_put(struct intel_engine_cs *engine)
{
intel_wakeref_put(&engine->wakeref);
@@ -52,6 +69,21 @@ static inline void intel_engine_pm_flush(struct intel_engine_cs *engine)
intel_wakeref_unlock_wait(&engine->wakeref);
}
+static inline void intel_engine_pm_might_put(struct intel_engine_cs *engine)
+{
+ if (!intel_engine_is_virtual(engine)) {
+ intel_wakeref_might_put(&engine->wakeref);
+ } else {
+ struct intel_gt *gt = engine->gt;
+ struct intel_engine_cs *tengine;
+ intel_engine_mask_t tmp, mask = engine->mask;
+
+ for_each_engine_masked(tengine, gt, mask, tmp)
+ intel_wakeref_might_put(&tengine->wakeref);
+ }
+ intel_gt_pm_might_put(engine->gt);
+}
+
static inline struct i915_request *
intel_engine_create_kernel_request(struct intel_engine_cs *engine)
{
diff --git a/drivers/gpu/drm/i915/gt/intel_gt_pm.h b/drivers/gpu/drm/i915/gt/intel_gt_pm.h
index 05de6c1af25b..bc898df7a48c 100644
--- a/drivers/gpu/drm/i915/gt/intel_gt_pm.h
+++ b/drivers/gpu/drm/i915/gt/intel_gt_pm.h
@@ -31,6 +31,11 @@ static inline bool intel_gt_pm_get_if_awake(struct intel_gt *gt)
return intel_wakeref_get_if_active(>->wakeref);
}
+static inline void intel_gt_pm_might_get(struct intel_gt *gt)
+{
+ intel_wakeref_might_get(>->wakeref);
+}
+
static inline void intel_gt_pm_put(struct intel_gt *gt)
{
intel_wakeref_put(>->wakeref);
@@ -41,6 +46,11 @@ static inline void intel_gt_pm_put_async(struct intel_gt *gt)
intel_wakeref_put_async(>->wakeref);
}
+static inline void intel_gt_pm_might_put(struct intel_gt *gt)
+{
+ intel_wakeref_might_put(>->wakeref);
+}
+
#define with_intel_gt_pm(gt, tmp) \
for (tmp = 1, intel_gt_pm_get(gt); tmp; \
intel_gt_pm_put(gt), tmp = 0)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 5c53428032b5..795cc071d05d 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -1571,7 +1571,12 @@ static int guc_context_pre_pin(struct intel_context *ce,
static int guc_context_pin(struct intel_context *ce, void *vaddr)
{
- return __guc_context_pin(ce, ce->engine, vaddr);
+ int ret = __guc_context_pin(ce, ce->engine, vaddr);
+
+ if (likely(!ret && !intel_context_is_barrier(ce)))
+ intel_engine_pm_get(ce->engine);
+
+ return ret;
}
static void guc_context_unpin(struct intel_context *ce)
@@ -1580,6 +1585,9 @@ static void guc_context_unpin(struct intel_context *ce)
unpin_guc_id(guc, ce);
lrc_unpin(ce);
+
+ if (likely(!intel_context_is_barrier(ce)))
+ intel_engine_pm_put_async(ce->engine);
}
static void guc_context_post_unpin(struct intel_context *ce)
@@ -2341,8 +2349,30 @@ static int guc_virtual_context_pre_pin(struct intel_context *ce,
static int guc_virtual_context_pin(struct intel_context *ce, void *vaddr)
{
struct intel_engine_cs *engine = guc_virtual_get_sibling(ce->engine, 0);
+ int ret = __guc_context_pin(ce, engine, vaddr);
+ intel_engine_mask_t tmp, mask = ce->engine->mask;
+
+ if (likely(!ret))
+ for_each_engine_masked(engine, ce->engine->gt, mask, tmp)
+ intel_engine_pm_get(engine);
- return __guc_context_pin(ce, engine, vaddr);
+ return ret;
+}
+
+static void guc_virtual_context_unpin(struct intel_context *ce)
+{
+ intel_engine_mask_t tmp, mask = ce->engine->mask;
+ struct intel_engine_cs *engine;
+ struct intel_guc *guc = ce_to_guc(ce);
+
+ GEM_BUG_ON(context_enabled(ce));
+ GEM_BUG_ON(intel_context_is_barrier(ce));
+
+ unpin_guc_id(guc, ce);
+ lrc_unpin(ce);
+
+ for_each_engine_masked(engine, ce->engine->gt, mask, tmp)
+ intel_engine_pm_put_async(engine);
}
static void guc_virtual_context_enter(struct intel_context *ce)
@@ -2379,7 +2409,7 @@ static const struct intel_context_ops virtual_guc_context_ops = {
.pre_pin = guc_virtual_context_pre_pin,
.pin = guc_virtual_context_pin,
- .unpin = guc_context_unpin,
+ .unpin = guc_virtual_context_unpin,
.post_unpin = guc_context_post_unpin,
.ban = guc_context_ban,
diff --git a/drivers/gpu/drm/i915/intel_wakeref.h b/drivers/gpu/drm/i915/intel_wakeref.h
index 545c8f277c46..4f4c2e15e736 100644
--- a/drivers/gpu/drm/i915/intel_wakeref.h
+++ b/drivers/gpu/drm/i915/intel_wakeref.h
@@ -123,6 +123,12 @@ enum {
__INTEL_WAKEREF_PUT_LAST_BIT__
};
+static inline void
+intel_wakeref_might_get(struct intel_wakeref *wf)
+{
+ might_lock(&wf->mutex);
+}
+
/**
* intel_wakeref_put_flags: Release the wakeref
* @wf: the wakeref
@@ -170,6 +176,12 @@ intel_wakeref_put_delay(struct intel_wakeref *wf, unsigned long delay)
FIELD_PREP(INTEL_WAKEREF_PUT_DELAY, delay));
}
+static inline void
+intel_wakeref_might_put(struct intel_wakeref *wf)
+{
+ might_lock(&wf->mutex);
+}
+
/**
* intel_wakeref_lock: Lock the wakeref (mutex)
* @wf: the wakeref
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 04/25] drm/i915/guc: Don't call switch_to_kernel_context with GuC submission
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Calling switch_to_kernel_context isn't needed if the engine PM reference
is taken while all user contexts are pinned as if don't have PM ref that
guarantees that all user contexts scheduling is disabled. By not calling
switch_to_kernel_context we save on issuing a request to the engine.
v2:
(Daniel Vetter)
- Add FIXME comment about pushing switch_to_kernel_context to backend
v3:
(John Harrison)
- Update commit message
- Fix workding comment
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_engine_pm.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
index dacd62773735..a1334b48dde7 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
@@ -162,6 +162,19 @@ static bool switch_to_kernel_context(struct intel_engine_cs *engine)
unsigned long flags;
bool result = true;
+ /*
+ * This is execlist specific behaviour intended to ensure the GPU is
+ * idle by switching to a known 'safe' context. With GuC submission, the
+ * same idle guarantee is achieved by other means (disabling
+ * scheduling). Further, switching to a 'safe' context has no effect
+ * with GuC submission as the scheduler can just switch back again.
+ *
+ * FIXME: Move this backend scheduler specific behaviour into the
+ * scheduler backend.
+ */
+ if (intel_engine_uses_guc(engine))
+ return true;
+
/* GPU is pointing to the void, as good as in the kernel context. */
if (intel_gt_is_wedged(engine->gt))
return true;
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH 04/25] drm/i915/guc: Don't call switch_to_kernel_context with GuC submission
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Calling switch_to_kernel_context isn't needed if the engine PM reference
is taken while all user contexts are pinned as if don't have PM ref that
guarantees that all user contexts scheduling is disabled. By not calling
switch_to_kernel_context we save on issuing a request to the engine.
v2:
(Daniel Vetter)
- Add FIXME comment about pushing switch_to_kernel_context to backend
v3:
(John Harrison)
- Update commit message
- Fix workding comment
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_engine_pm.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_pm.c b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
index dacd62773735..a1334b48dde7 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_pm.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_pm.c
@@ -162,6 +162,19 @@ static bool switch_to_kernel_context(struct intel_engine_cs *engine)
unsigned long flags;
bool result = true;
+ /*
+ * This is execlist specific behaviour intended to ensure the GPU is
+ * idle by switching to a known 'safe' context. With GuC submission, the
+ * same idle guarantee is achieved by other means (disabling
+ * scheduling). Further, switching to a 'safe' context has no effect
+ * with GuC submission as the scheduler can just switch back again.
+ *
+ * FIXME: Move this backend scheduler specific behaviour into the
+ * scheduler backend.
+ */
+ if (intel_engine_uses_guc(engine))
+ return true;
+
/* GPU is pointing to the void, as good as in the kernel context. */
if (intel_gt_is_wedged(engine->gt))
return true;
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 05/25] drm/i915: Add logical engine mapping
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Add logical engine mapping. This is required for split-frame, as
workloads need to be placed on engines in a logically contiguous manner.
v2:
(Daniel Vetter)
- Add kernel doc for new fields
v3:
(Tvrtko)
- Update comment for new logical_mask field
v4:
(John Harrison)
- Update comment for new logical_mask field
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 60 ++++++++++++++++---
drivers/gpu/drm/i915/gt/intel_engine_types.h | 7 +++
.../drm/i915/gt/intel_execlists_submission.c | 1 +
drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c | 2 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 21 +------
5 files changed, 62 insertions(+), 29 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index 2ae57e4656a3..2eb798ad068b 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -290,7 +290,8 @@ static void nop_irq_handler(struct intel_engine_cs *engine, u16 iir)
GEM_DEBUG_WARN_ON(iir);
}
-static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id)
+static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id,
+ u8 logical_instance)
{
const struct engine_info *info = &intel_engines[id];
struct drm_i915_private *i915 = gt->i915;
@@ -335,6 +336,7 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id)
engine->class = info->class;
engine->instance = info->instance;
+ engine->logical_mask = BIT(logical_instance);
__sprint_engine_name(engine);
engine->props.heartbeat_interval_ms =
@@ -588,6 +590,37 @@ static intel_engine_mask_t init_engine_mask(struct intel_gt *gt)
return info->engine_mask;
}
+static void populate_logical_ids(struct intel_gt *gt, u8 *logical_ids,
+ u8 class, const u8 *map, u8 num_instances)
+{
+ int i, j;
+ u8 current_logical_id = 0;
+
+ for (j = 0; j < num_instances; ++j) {
+ for (i = 0; i < ARRAY_SIZE(intel_engines); ++i) {
+ if (!HAS_ENGINE(gt, i) ||
+ intel_engines[i].class != class)
+ continue;
+
+ if (intel_engines[i].instance == map[j]) {
+ logical_ids[intel_engines[i].instance] =
+ current_logical_id++;
+ break;
+ }
+ }
+ }
+}
+
+static void setup_logical_ids(struct intel_gt *gt, u8 *logical_ids, u8 class)
+{
+ int i;
+ u8 map[MAX_ENGINE_INSTANCE + 1];
+
+ for (i = 0; i < MAX_ENGINE_INSTANCE + 1; ++i)
+ map[i] = i;
+ populate_logical_ids(gt, logical_ids, class, map, ARRAY_SIZE(map));
+}
+
/**
* intel_engines_init_mmio() - allocate and prepare the Engine Command Streamers
* @gt: pointer to struct intel_gt
@@ -599,7 +632,8 @@ int intel_engines_init_mmio(struct intel_gt *gt)
struct drm_i915_private *i915 = gt->i915;
const unsigned int engine_mask = init_engine_mask(gt);
unsigned int mask = 0;
- unsigned int i;
+ unsigned int i, class;
+ u8 logical_ids[MAX_ENGINE_INSTANCE + 1];
int err;
drm_WARN_ON(&i915->drm, engine_mask == 0);
@@ -609,15 +643,23 @@ int intel_engines_init_mmio(struct intel_gt *gt)
if (i915_inject_probe_failure(i915))
return -ENODEV;
- for (i = 0; i < ARRAY_SIZE(intel_engines); i++) {
- if (!HAS_ENGINE(gt, i))
- continue;
+ for (class = 0; class < MAX_ENGINE_CLASS + 1; ++class) {
+ setup_logical_ids(gt, logical_ids, class);
- err = intel_engine_setup(gt, i);
- if (err)
- goto cleanup;
+ for (i = 0; i < ARRAY_SIZE(intel_engines); ++i) {
+ u8 instance = intel_engines[i].instance;
+
+ if (intel_engines[i].class != class ||
+ !HAS_ENGINE(gt, i))
+ continue;
- mask |= BIT(i);
+ err = intel_engine_setup(gt, i,
+ logical_ids[instance]);
+ if (err)
+ goto cleanup;
+
+ mask |= BIT(i);
+ }
}
/*
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
index 9167ce52487c..e0f773585c29 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
@@ -269,6 +269,13 @@ struct intel_engine_cs {
unsigned int guc_id;
intel_engine_mask_t mask;
+ /**
+ * @logical_mask: logical mask of engine, reported to user space via
+ * query IOCTL and used to communicate with the GuC in logical space.
+ * The logical instance of a physical engine can change based on product
+ * and fusing.
+ */
+ intel_engine_mask_t logical_mask;
u8 class;
u8 instance;
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 73a79c2acd3a..43a74b216efb 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -3877,6 +3877,7 @@ execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count)
ve->siblings[ve->num_siblings++] = sibling;
ve->base.mask |= sibling->mask;
+ ve->base.logical_mask |= sibling->logical_mask;
/*
* All physical engines must be compatible for their emission
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c
index 2c6ea64af7ec..621c893a009f 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c
@@ -176,7 +176,7 @@ static void guc_mapping_table_init(struct intel_gt *gt,
for_each_engine(engine, gt, id) {
u8 guc_class = engine_class_to_guc_class(engine->class);
- system_info->mapping_table[guc_class][engine->instance] =
+ system_info->mapping_table[guc_class][ilog2(engine->logical_mask)] =
engine->instance;
}
}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 795cc071d05d..fd6594bc1b96 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -1423,23 +1423,6 @@ static int deregister_context(struct intel_context *ce, u32 guc_id)
return __guc_action_deregister_context(guc, guc_id);
}
-static intel_engine_mask_t adjust_engine_mask(u8 class, intel_engine_mask_t mask)
-{
- switch (class) {
- case RENDER_CLASS:
- return mask >> RCS0;
- case VIDEO_ENHANCEMENT_CLASS:
- return mask >> VECS0;
- case VIDEO_DECODE_CLASS:
- return mask >> VCS0;
- case COPY_ENGINE_CLASS:
- return mask >> BCS0;
- default:
- MISSING_CASE(class);
- return 0;
- }
-}
-
static void guc_context_policy_init(struct intel_engine_cs *engine,
struct guc_lrc_desc *desc)
{
@@ -1481,8 +1464,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)
desc = __get_lrc_desc(guc, desc_idx);
desc->engine_class = engine_class_to_guc_class(engine->class);
- desc->engine_submit_mask = adjust_engine_mask(engine->class,
- engine->mask);
+ desc->engine_submit_mask = engine->logical_mask;
desc->hw_context_desc = ce->lrc.lrca;
desc->priority = ce->guc_state.prio;
desc->context_flags = CONTEXT_REGISTRATION_FLAG_KMD;
@@ -3271,6 +3253,7 @@ guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count)
}
ve->base.mask |= sibling->mask;
+ ve->base.logical_mask |= sibling->logical_mask;
if (n != 0 && ve->base.class != sibling->class) {
DRM_DEBUG("invalid mixing of engine class, sibling %d, already %d\n",
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 05/25] drm/i915: Add logical engine mapping
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Add logical engine mapping. This is required for split-frame, as
workloads need to be placed on engines in a logically contiguous manner.
v2:
(Daniel Vetter)
- Add kernel doc for new fields
v3:
(Tvrtko)
- Update comment for new logical_mask field
v4:
(John Harrison)
- Update comment for new logical_mask field
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 60 ++++++++++++++++---
drivers/gpu/drm/i915/gt/intel_engine_types.h | 7 +++
.../drm/i915/gt/intel_execlists_submission.c | 1 +
drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c | 2 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 21 +------
5 files changed, 62 insertions(+), 29 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index 2ae57e4656a3..2eb798ad068b 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -290,7 +290,8 @@ static void nop_irq_handler(struct intel_engine_cs *engine, u16 iir)
GEM_DEBUG_WARN_ON(iir);
}
-static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id)
+static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id,
+ u8 logical_instance)
{
const struct engine_info *info = &intel_engines[id];
struct drm_i915_private *i915 = gt->i915;
@@ -335,6 +336,7 @@ static int intel_engine_setup(struct intel_gt *gt, enum intel_engine_id id)
engine->class = info->class;
engine->instance = info->instance;
+ engine->logical_mask = BIT(logical_instance);
__sprint_engine_name(engine);
engine->props.heartbeat_interval_ms =
@@ -588,6 +590,37 @@ static intel_engine_mask_t init_engine_mask(struct intel_gt *gt)
return info->engine_mask;
}
+static void populate_logical_ids(struct intel_gt *gt, u8 *logical_ids,
+ u8 class, const u8 *map, u8 num_instances)
+{
+ int i, j;
+ u8 current_logical_id = 0;
+
+ for (j = 0; j < num_instances; ++j) {
+ for (i = 0; i < ARRAY_SIZE(intel_engines); ++i) {
+ if (!HAS_ENGINE(gt, i) ||
+ intel_engines[i].class != class)
+ continue;
+
+ if (intel_engines[i].instance == map[j]) {
+ logical_ids[intel_engines[i].instance] =
+ current_logical_id++;
+ break;
+ }
+ }
+ }
+}
+
+static void setup_logical_ids(struct intel_gt *gt, u8 *logical_ids, u8 class)
+{
+ int i;
+ u8 map[MAX_ENGINE_INSTANCE + 1];
+
+ for (i = 0; i < MAX_ENGINE_INSTANCE + 1; ++i)
+ map[i] = i;
+ populate_logical_ids(gt, logical_ids, class, map, ARRAY_SIZE(map));
+}
+
/**
* intel_engines_init_mmio() - allocate and prepare the Engine Command Streamers
* @gt: pointer to struct intel_gt
@@ -599,7 +632,8 @@ int intel_engines_init_mmio(struct intel_gt *gt)
struct drm_i915_private *i915 = gt->i915;
const unsigned int engine_mask = init_engine_mask(gt);
unsigned int mask = 0;
- unsigned int i;
+ unsigned int i, class;
+ u8 logical_ids[MAX_ENGINE_INSTANCE + 1];
int err;
drm_WARN_ON(&i915->drm, engine_mask == 0);
@@ -609,15 +643,23 @@ int intel_engines_init_mmio(struct intel_gt *gt)
if (i915_inject_probe_failure(i915))
return -ENODEV;
- for (i = 0; i < ARRAY_SIZE(intel_engines); i++) {
- if (!HAS_ENGINE(gt, i))
- continue;
+ for (class = 0; class < MAX_ENGINE_CLASS + 1; ++class) {
+ setup_logical_ids(gt, logical_ids, class);
- err = intel_engine_setup(gt, i);
- if (err)
- goto cleanup;
+ for (i = 0; i < ARRAY_SIZE(intel_engines); ++i) {
+ u8 instance = intel_engines[i].instance;
+
+ if (intel_engines[i].class != class ||
+ !HAS_ENGINE(gt, i))
+ continue;
- mask |= BIT(i);
+ err = intel_engine_setup(gt, i,
+ logical_ids[instance]);
+ if (err)
+ goto cleanup;
+
+ mask |= BIT(i);
+ }
}
/*
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_types.h b/drivers/gpu/drm/i915/gt/intel_engine_types.h
index 9167ce52487c..e0f773585c29 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine_types.h
@@ -269,6 +269,13 @@ struct intel_engine_cs {
unsigned int guc_id;
intel_engine_mask_t mask;
+ /**
+ * @logical_mask: logical mask of engine, reported to user space via
+ * query IOCTL and used to communicate with the GuC in logical space.
+ * The logical instance of a physical engine can change based on product
+ * and fusing.
+ */
+ intel_engine_mask_t logical_mask;
u8 class;
u8 instance;
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 73a79c2acd3a..43a74b216efb 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -3877,6 +3877,7 @@ execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count)
ve->siblings[ve->num_siblings++] = sibling;
ve->base.mask |= sibling->mask;
+ ve->base.logical_mask |= sibling->logical_mask;
/*
* All physical engines must be compatible for their emission
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c
index 2c6ea64af7ec..621c893a009f 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c
@@ -176,7 +176,7 @@ static void guc_mapping_table_init(struct intel_gt *gt,
for_each_engine(engine, gt, id) {
u8 guc_class = engine_class_to_guc_class(engine->class);
- system_info->mapping_table[guc_class][engine->instance] =
+ system_info->mapping_table[guc_class][ilog2(engine->logical_mask)] =
engine->instance;
}
}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 795cc071d05d..fd6594bc1b96 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -1423,23 +1423,6 @@ static int deregister_context(struct intel_context *ce, u32 guc_id)
return __guc_action_deregister_context(guc, guc_id);
}
-static intel_engine_mask_t adjust_engine_mask(u8 class, intel_engine_mask_t mask)
-{
- switch (class) {
- case RENDER_CLASS:
- return mask >> RCS0;
- case VIDEO_ENHANCEMENT_CLASS:
- return mask >> VECS0;
- case VIDEO_DECODE_CLASS:
- return mask >> VCS0;
- case COPY_ENGINE_CLASS:
- return mask >> BCS0;
- default:
- MISSING_CASE(class);
- return 0;
- }
-}
-
static void guc_context_policy_init(struct intel_engine_cs *engine,
struct guc_lrc_desc *desc)
{
@@ -1481,8 +1464,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)
desc = __get_lrc_desc(guc, desc_idx);
desc->engine_class = engine_class_to_guc_class(engine->class);
- desc->engine_submit_mask = adjust_engine_mask(engine->class,
- engine->mask);
+ desc->engine_submit_mask = engine->logical_mask;
desc->hw_context_desc = ce->lrc.lrca;
desc->priority = ce->guc_state.prio;
desc->context_flags = CONTEXT_REGISTRATION_FLAG_KMD;
@@ -3271,6 +3253,7 @@ guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count)
}
ve->base.mask |= sibling->mask;
+ ve->base.logical_mask |= sibling->logical_mask;
if (n != 0 && ve->base.class != sibling->class) {
DRM_DEBUG("invalid mixing of engine class, sibling %d, already %d\n",
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 06/25] drm/i915: Expose logical engine instance to user
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Expose logical engine instance to user via query engine info IOCTL. This
is required for split-frame workloads as these needs to be placed on
engines in a logically contiguous order. The logical mapping can change
based on fusing. Rather than having user have knowledge of the fusing we
simply just expose the logical mapping with the existing query engine
info IOCTL.
IGT: https://patchwork.freedesktop.org/patch/445637/?series=92854&rev=1
media UMD: https://github.com/intel/media-driver/pull/1252
v2:
(Daniel Vetter)
- Add IGT link, placeholder for media UMD
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_query.c | 2 ++
include/uapi/drm/i915_drm.h | 8 +++++++-
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/i915_query.c b/drivers/gpu/drm/i915/i915_query.c
index 5e2b909827f4..51b368be0fc4 100644
--- a/drivers/gpu/drm/i915/i915_query.c
+++ b/drivers/gpu/drm/i915/i915_query.c
@@ -124,7 +124,9 @@ query_engine_info(struct drm_i915_private *i915,
for_each_uabi_engine(engine, i915) {
info.engine.engine_class = engine->uabi_class;
info.engine.engine_instance = engine->uabi_instance;
+ info.flags = I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE;
info.capabilities = engine->uabi_capabilities;
+ info.logical_instance = ilog2(engine->logical_mask);
if (copy_to_user(info_ptr, &info, sizeof(info)))
return -EFAULT;
diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
index aa2a7eccfb94..0179f92e0916 100644
--- a/include/uapi/drm/i915_drm.h
+++ b/include/uapi/drm/i915_drm.h
@@ -2775,14 +2775,20 @@ struct drm_i915_engine_info {
/** @flags: Engine flags. */
__u64 flags;
+#define I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE (1 << 0)
/** @capabilities: Capabilities of this engine. */
__u64 capabilities;
#define I915_VIDEO_CLASS_CAPABILITY_HEVC (1 << 0)
#define I915_VIDEO_AND_ENHANCE_CLASS_CAPABILITY_SFC (1 << 1)
+ /** @logical_instance: Logical instance of engine */
+ __u16 logical_instance;
+
/** @rsvd1: Reserved fields. */
- __u64 rsvd1[4];
+ __u16 rsvd1[3];
+ /** @rsvd2: Reserved fields. */
+ __u64 rsvd2[3];
};
/**
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 06/25] drm/i915: Expose logical engine instance to user
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Expose logical engine instance to user via query engine info IOCTL. This
is required for split-frame workloads as these needs to be placed on
engines in a logically contiguous order. The logical mapping can change
based on fusing. Rather than having user have knowledge of the fusing we
simply just expose the logical mapping with the existing query engine
info IOCTL.
IGT: https://patchwork.freedesktop.org/patch/445637/?series=92854&rev=1
media UMD: https://github.com/intel/media-driver/pull/1252
v2:
(Daniel Vetter)
- Add IGT link, placeholder for media UMD
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_query.c | 2 ++
include/uapi/drm/i915_drm.h | 8 +++++++-
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/i915_query.c b/drivers/gpu/drm/i915/i915_query.c
index 5e2b909827f4..51b368be0fc4 100644
--- a/drivers/gpu/drm/i915/i915_query.c
+++ b/drivers/gpu/drm/i915/i915_query.c
@@ -124,7 +124,9 @@ query_engine_info(struct drm_i915_private *i915,
for_each_uabi_engine(engine, i915) {
info.engine.engine_class = engine->uabi_class;
info.engine.engine_instance = engine->uabi_instance;
+ info.flags = I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE;
info.capabilities = engine->uabi_capabilities;
+ info.logical_instance = ilog2(engine->logical_mask);
if (copy_to_user(info_ptr, &info, sizeof(info)))
return -EFAULT;
diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
index aa2a7eccfb94..0179f92e0916 100644
--- a/include/uapi/drm/i915_drm.h
+++ b/include/uapi/drm/i915_drm.h
@@ -2775,14 +2775,20 @@ struct drm_i915_engine_info {
/** @flags: Engine flags. */
__u64 flags;
+#define I915_ENGINE_INFO_HAS_LOGICAL_INSTANCE (1 << 0)
/** @capabilities: Capabilities of this engine. */
__u64 capabilities;
#define I915_VIDEO_CLASS_CAPABILITY_HEVC (1 << 0)
#define I915_VIDEO_AND_ENHANCE_CLASS_CAPABILITY_SFC (1 << 1)
+ /** @logical_instance: Logical instance of engine */
+ __u16 logical_instance;
+
/** @rsvd1: Reserved fields. */
- __u64 rsvd1[4];
+ __u16 rsvd1[3];
+ /** @rsvd2: Reserved fields. */
+ __u64 rsvd2[3];
};
/**
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 07/25] drm/i915/guc: Introduce context parent-child relationship
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Introduce context parent-child relationship. Once this relationship is
created all pinning / unpinning operations are directed to the parent
context. The parent context is responsible for pinning all of its
children and itself.
This is a precursor to the full GuC multi-lrc implementation but aligns
to how GuC mutli-lrc interface is defined - a single H2G is used
register / deregister all of the contexts simultaneously.
Subsequent patches in the series will implement the pinning / unpinning
operations for parent / child contexts.
v2:
(Daniel Vetter)
- Add kernel doc, add wrapper to access parent to ensure safety
v3:
(John Harrison)
- Fix comment explaing GEM_BUG_ON in to_parent()
- Make variable names generic (non-GuC specific)
v4:
(John Harrison)
- s/its'/its/g
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.c | 29 +++++++++++++
drivers/gpu/drm/i915/gt/intel_context.h | 41 +++++++++++++++++++
drivers/gpu/drm/i915/gt/intel_context_types.h | 21 ++++++++++
3 files changed, 91 insertions(+)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index f98c9f470ba1..79f321c6c008 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -404,6 +404,8 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)
INIT_LIST_HEAD(&ce->destroyed_link);
+ INIT_LIST_HEAD(&ce->parallel.child_list);
+
/*
* Initialize fence to be complete as this is expected to be complete
* unless there is a pending schedule disable outstanding.
@@ -418,10 +420,17 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)
void intel_context_fini(struct intel_context *ce)
{
+ struct intel_context *child, *next;
+
if (ce->timeline)
intel_timeline_put(ce->timeline);
i915_vm_put(ce->vm);
+ /* Need to put the creation ref for the children */
+ if (intel_context_is_parent(ce))
+ for_each_child_safe(ce, child, next)
+ intel_context_put(child);
+
mutex_destroy(&ce->pin_mutex);
i915_active_fini(&ce->active);
i915_sw_fence_fini(&ce->guc_state.blocked);
@@ -538,6 +547,26 @@ struct i915_request *intel_context_find_active_request(struct intel_context *ce)
return active;
}
+void intel_context_bind_parent_child(struct intel_context *parent,
+ struct intel_context *child)
+{
+ /*
+ * Callers responsibility to validate that this function is used
+ * correctly but we use GEM_BUG_ON here ensure that they do.
+ */
+ GEM_BUG_ON(!intel_engine_uses_guc(parent->engine));
+ GEM_BUG_ON(intel_context_is_pinned(parent));
+ GEM_BUG_ON(intel_context_is_child(parent));
+ GEM_BUG_ON(intel_context_is_pinned(child));
+ GEM_BUG_ON(intel_context_is_child(child));
+ GEM_BUG_ON(intel_context_is_parent(child));
+
+ parent->parallel.number_children++;
+ list_add_tail(&child->parallel.child_link,
+ &parent->parallel.child_list);
+ child->parallel.parent = parent;
+}
+
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
#include "selftest_context.c"
#endif
diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
index c41098950746..b63c10a144af 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.h
+++ b/drivers/gpu/drm/i915/gt/intel_context.h
@@ -44,6 +44,47 @@ void intel_context_free(struct intel_context *ce);
int intel_context_reconfigure_sseu(struct intel_context *ce,
const struct intel_sseu sseu);
+static inline bool intel_context_is_child(struct intel_context *ce)
+{
+ return !!ce->parallel.parent;
+}
+
+static inline bool intel_context_is_parent(struct intel_context *ce)
+{
+ return !!ce->parallel.number_children;
+}
+
+static inline bool intel_context_is_pinned(struct intel_context *ce);
+
+static inline struct intel_context *
+intel_context_to_parent(struct intel_context *ce)
+{
+ if (intel_context_is_child(ce)) {
+ /*
+ * The parent holds ref count to the child so it is always safe
+ * for the parent to access the child, but the child has a
+ * pointer to the parent without a ref. To ensure this is safe
+ * the child should only access the parent pointer while the
+ * parent is pinned.
+ */
+ GEM_BUG_ON(!intel_context_is_pinned(ce->parallel.parent));
+
+ return ce->parallel.parent;
+ } else {
+ return ce;
+ }
+}
+
+void intel_context_bind_parent_child(struct intel_context *parent,
+ struct intel_context *child);
+
+#define for_each_child(parent, ce)\
+ list_for_each_entry(ce, &(parent)->parallel.child_list,\
+ parallel.child_link)
+#define for_each_child_safe(parent, ce, cn)\
+ list_for_each_entry_safe(ce, cn, &(parent)->parallel.child_list,\
+ parallel.child_link)
+
/**
* intel_context_lock_pinned - Stablises the 'pinned' status of the HW context
* @ce - the context
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 4613d027cbc3..76dfca57cb45 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -220,6 +220,27 @@ struct intel_context {
*/
struct list_head destroyed_link;
+ /** @parallel: sub-structure for parallel submission members */
+ struct {
+ union {
+ /**
+ * @child_list: parent's list of children
+ * contexts, no protection as immutable after context
+ * creation
+ */
+ struct list_head child_list;
+ /**
+ * @child_link: child's link into parent's list of
+ * children
+ */
+ struct list_head child_link;
+ };
+ /** @parent: pointer to parent if child */
+ struct intel_context *parent;
+ /** @number_children: number of children if parent */
+ u8 number_children;
+ } parallel;
+
#ifdef CONFIG_DRM_I915_SELFTEST
/**
* @drop_schedule_enable: Force drop of schedule enable G2H for selftest
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 07/25] drm/i915/guc: Introduce context parent-child relationship
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Introduce context parent-child relationship. Once this relationship is
created all pinning / unpinning operations are directed to the parent
context. The parent context is responsible for pinning all of its
children and itself.
This is a precursor to the full GuC multi-lrc implementation but aligns
to how GuC mutli-lrc interface is defined - a single H2G is used
register / deregister all of the contexts simultaneously.
Subsequent patches in the series will implement the pinning / unpinning
operations for parent / child contexts.
v2:
(Daniel Vetter)
- Add kernel doc, add wrapper to access parent to ensure safety
v3:
(John Harrison)
- Fix comment explaing GEM_BUG_ON in to_parent()
- Make variable names generic (non-GuC specific)
v4:
(John Harrison)
- s/its'/its/g
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.c | 29 +++++++++++++
drivers/gpu/drm/i915/gt/intel_context.h | 41 +++++++++++++++++++
drivers/gpu/drm/i915/gt/intel_context_types.h | 21 ++++++++++
3 files changed, 91 insertions(+)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index f98c9f470ba1..79f321c6c008 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -404,6 +404,8 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)
INIT_LIST_HEAD(&ce->destroyed_link);
+ INIT_LIST_HEAD(&ce->parallel.child_list);
+
/*
* Initialize fence to be complete as this is expected to be complete
* unless there is a pending schedule disable outstanding.
@@ -418,10 +420,17 @@ intel_context_init(struct intel_context *ce, struct intel_engine_cs *engine)
void intel_context_fini(struct intel_context *ce)
{
+ struct intel_context *child, *next;
+
if (ce->timeline)
intel_timeline_put(ce->timeline);
i915_vm_put(ce->vm);
+ /* Need to put the creation ref for the children */
+ if (intel_context_is_parent(ce))
+ for_each_child_safe(ce, child, next)
+ intel_context_put(child);
+
mutex_destroy(&ce->pin_mutex);
i915_active_fini(&ce->active);
i915_sw_fence_fini(&ce->guc_state.blocked);
@@ -538,6 +547,26 @@ struct i915_request *intel_context_find_active_request(struct intel_context *ce)
return active;
}
+void intel_context_bind_parent_child(struct intel_context *parent,
+ struct intel_context *child)
+{
+ /*
+ * Callers responsibility to validate that this function is used
+ * correctly but we use GEM_BUG_ON here ensure that they do.
+ */
+ GEM_BUG_ON(!intel_engine_uses_guc(parent->engine));
+ GEM_BUG_ON(intel_context_is_pinned(parent));
+ GEM_BUG_ON(intel_context_is_child(parent));
+ GEM_BUG_ON(intel_context_is_pinned(child));
+ GEM_BUG_ON(intel_context_is_child(child));
+ GEM_BUG_ON(intel_context_is_parent(child));
+
+ parent->parallel.number_children++;
+ list_add_tail(&child->parallel.child_link,
+ &parent->parallel.child_list);
+ child->parallel.parent = parent;
+}
+
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
#include "selftest_context.c"
#endif
diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
index c41098950746..b63c10a144af 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.h
+++ b/drivers/gpu/drm/i915/gt/intel_context.h
@@ -44,6 +44,47 @@ void intel_context_free(struct intel_context *ce);
int intel_context_reconfigure_sseu(struct intel_context *ce,
const struct intel_sseu sseu);
+static inline bool intel_context_is_child(struct intel_context *ce)
+{
+ return !!ce->parallel.parent;
+}
+
+static inline bool intel_context_is_parent(struct intel_context *ce)
+{
+ return !!ce->parallel.number_children;
+}
+
+static inline bool intel_context_is_pinned(struct intel_context *ce);
+
+static inline struct intel_context *
+intel_context_to_parent(struct intel_context *ce)
+{
+ if (intel_context_is_child(ce)) {
+ /*
+ * The parent holds ref count to the child so it is always safe
+ * for the parent to access the child, but the child has a
+ * pointer to the parent without a ref. To ensure this is safe
+ * the child should only access the parent pointer while the
+ * parent is pinned.
+ */
+ GEM_BUG_ON(!intel_context_is_pinned(ce->parallel.parent));
+
+ return ce->parallel.parent;
+ } else {
+ return ce;
+ }
+}
+
+void intel_context_bind_parent_child(struct intel_context *parent,
+ struct intel_context *child);
+
+#define for_each_child(parent, ce)\
+ list_for_each_entry(ce, &(parent)->parallel.child_list,\
+ parallel.child_link)
+#define for_each_child_safe(parent, ce, cn)\
+ list_for_each_entry_safe(ce, cn, &(parent)->parallel.child_list,\
+ parallel.child_link)
+
/**
* intel_context_lock_pinned - Stablises the 'pinned' status of the HW context
* @ce - the context
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 4613d027cbc3..76dfca57cb45 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -220,6 +220,27 @@ struct intel_context {
*/
struct list_head destroyed_link;
+ /** @parallel: sub-structure for parallel submission members */
+ struct {
+ union {
+ /**
+ * @child_list: parent's list of children
+ * contexts, no protection as immutable after context
+ * creation
+ */
+ struct list_head child_list;
+ /**
+ * @child_link: child's link into parent's list of
+ * children
+ */
+ struct list_head child_link;
+ };
+ /** @parent: pointer to parent if child */
+ struct intel_context *parent;
+ /** @number_children: number of children if parent */
+ u8 number_children;
+ } parallel;
+
#ifdef CONFIG_DRM_I915_SELFTEST
/**
* @drop_schedule_enable: Force drop of schedule enable G2H for selftest
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 08/25] drm/i915/guc: Add multi-lrc context registration
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Add multi-lrc context registration H2G. In addition a workqueue and
process descriptor are setup during multi-lrc context registration as
these data structures are needed for multi-lrc submission.
v2:
(John Harrison)
- Move GuC specific fields into sub-struct
- Clean up WQ defines
- Add comment explaining math to derive WQ / PD address
v3:
(John Harrison)
- Add PARENT_SCRATCH_SIZE define
- Update comment explaining multi-lrc register
v4:
(John Harrison)
- Move PARENT_SCRATCH_SIZE to common file
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.h | 2 +
drivers/gpu/drm/i915/gt/intel_context_types.h | 12 ++
drivers/gpu/drm/i915/gt/intel_lrc.c | 5 +
.../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h | 1 +
drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 2 -
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 115 +++++++++++++++++-
6 files changed, 134 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
index b63c10a144af..9f0995150a7a 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.h
+++ b/drivers/gpu/drm/i915/gt/intel_context.h
@@ -44,6 +44,8 @@ void intel_context_free(struct intel_context *ce);
int intel_context_reconfigure_sseu(struct intel_context *ce,
const struct intel_sseu sseu);
+#define PARENT_SCRATCH_SIZE PAGE_SIZE
+
static inline bool intel_context_is_child(struct intel_context *ce)
{
return !!ce->parallel.parent;
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 76dfca57cb45..48decb5ee954 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -239,6 +239,18 @@ struct intel_context {
struct intel_context *parent;
/** @number_children: number of children if parent */
u8 number_children;
+ /** @guc: GuC specific members for parallel submission */
+ struct {
+ /** @wqi_head: head pointer in work queue */
+ u16 wqi_head;
+ /** @wqi_tail: tail pointer in work queue */
+ u16 wqi_tail;
+ /**
+ * @parent_page: page in context state (ce->state) used
+ * by parent for work queue, process descriptor
+ */
+ u8 parent_page;
+ } guc;
} parallel;
#ifdef CONFIG_DRM_I915_SELFTEST
diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
index 3ef9eaf8c50e..56156cf18c41 100644
--- a/drivers/gpu/drm/i915/gt/intel_lrc.c
+++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
@@ -942,6 +942,11 @@ __lrc_alloc_state(struct intel_context *ce, struct intel_engine_cs *engine)
context_size += PAGE_SIZE;
}
+ if (intel_context_is_parent(ce) && intel_engine_uses_guc(engine)) {
+ ce->parallel.guc.parent_page = context_size / PAGE_SIZE;
+ context_size += PARENT_SCRATCH_SIZE;
+ }
+
obj = i915_gem_object_create_lmem(engine->i915, context_size,
I915_BO_ALLOC_PM_VOLATILE);
if (IS_ERR(obj))
diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
index 8ff582222aff..ba10bd374cee 100644
--- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
+++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
@@ -142,6 +142,7 @@ enum intel_guc_action {
INTEL_GUC_ACTION_REGISTER_COMMAND_TRANSPORT_BUFFER = 0x4505,
INTEL_GUC_ACTION_DEREGISTER_COMMAND_TRANSPORT_BUFFER = 0x4506,
INTEL_GUC_ACTION_DEREGISTER_CONTEXT_DONE = 0x4600,
+ INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC = 0x4601,
INTEL_GUC_ACTION_RESET_CLIENT = 0x5507,
INTEL_GUC_ACTION_LIMIT
};
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
index fa4be13c8854..0eeb2a9feeed 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
@@ -52,8 +52,6 @@
#define GUC_DOORBELL_INVALID 256
-#define GUC_WQ_SIZE (PAGE_SIZE * 2)
-
/* Work queue item header definitions */
#define WQ_STATUS_ACTIVE 1
#define WQ_STATUS_SUSPENDED 2
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index fd6594bc1b96..d9f5be00e586 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -344,6 +344,46 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
return rb_entry(rb, struct i915_priolist, node);
}
+/*
+ * When using multi-lrc submission a scratch memory area is reserved in the
+ * parent's context state for the process descriptor and work queue. Currently
+ * the scratch area is sized to a page.
+ *
+ * The layout of this scratch area is below:
+ * 0 guc_process_desc
+ * ... unused
+ * PARENT_SCRATCH_SIZE / 2 work queue start
+ * ... work queue
+ * PARENT_SCRATCH_SIZE - 1 work queue end
+ */
+#define WQ_SIZE (PARENT_SCRATCH_SIZE / 2)
+#define WQ_OFFSET (PARENT_SCRATCH_SIZE - WQ_SIZE)
+static u32 __get_process_desc_offset(struct intel_context *ce)
+{
+ GEM_BUG_ON(!ce->parallel.guc.parent_page);
+
+ return ce->parallel.guc.parent_page * PAGE_SIZE;
+}
+
+static u32 __get_wq_offset(struct intel_context *ce)
+{
+ return __get_process_desc_offset(ce) + WQ_OFFSET;
+}
+
+static struct guc_process_desc *
+__get_process_desc(struct intel_context *ce)
+{
+ /*
+ * Need to subtract LRC_STATE_OFFSET here as the
+ * parallel.guc.parent_page is the offset into ce->state while
+ * ce->lrc_reg_reg is ce->state + LRC_STATE_OFFSET.
+ */
+ return (struct guc_process_desc *)
+ (ce->lrc_reg_state +
+ ((__get_process_desc_offset(ce) -
+ LRC_STATE_OFFSET) / sizeof(u32)));
+}
+
static struct guc_lrc_desc *__get_lrc_desc(struct intel_guc *guc, u32 index)
{
struct guc_lrc_desc *base = guc->lrc_desc_pool_vaddr;
@@ -1365,6 +1405,30 @@ static void unpin_guc_id(struct intel_guc *guc, struct intel_context *ce)
spin_unlock_irqrestore(&guc->submission_state.lock, flags);
}
+static int __guc_action_register_multi_lrc(struct intel_guc *guc,
+ struct intel_context *ce,
+ u32 guc_id,
+ u32 offset,
+ bool loop)
+{
+ struct intel_context *child;
+ u32 action[4 + MAX_ENGINE_INSTANCE];
+ int len = 0;
+
+ GEM_BUG_ON(ce->parallel.number_children > MAX_ENGINE_INSTANCE);
+
+ action[len++] = INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC;
+ action[len++] = guc_id;
+ action[len++] = ce->parallel.number_children + 1;
+ action[len++] = offset;
+ for_each_child(ce, child) {
+ offset += sizeof(struct guc_lrc_desc);
+ action[len++] = offset;
+ }
+
+ return guc_submission_send_busy_loop(guc, action, len, 0, loop);
+}
+
static int __guc_action_register_context(struct intel_guc *guc,
u32 guc_id,
u32 offset,
@@ -1387,9 +1451,15 @@ static int register_context(struct intel_context *ce, bool loop)
ce->guc_id.id * sizeof(struct guc_lrc_desc);
int ret;
+ GEM_BUG_ON(intel_context_is_child(ce));
trace_intel_context_register(ce);
- ret = __guc_action_register_context(guc, ce->guc_id.id, offset, loop);
+ if (intel_context_is_parent(ce))
+ ret = __guc_action_register_multi_lrc(guc, ce, ce->guc_id.id,
+ offset, loop);
+ else
+ ret = __guc_action_register_context(guc, ce->guc_id.id, offset,
+ loop);
if (likely(!ret)) {
unsigned long flags;
@@ -1418,6 +1488,7 @@ static int deregister_context(struct intel_context *ce, u32 guc_id)
{
struct intel_guc *guc = ce_to_guc(ce);
+ GEM_BUG_ON(intel_context_is_child(ce));
trace_intel_context_deregister(ce);
return __guc_action_deregister_context(guc, guc_id);
@@ -1445,6 +1516,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)
struct guc_lrc_desc *desc;
bool context_registered;
intel_wakeref_t wakeref;
+ struct intel_context *child;
int ret = 0;
GEM_BUG_ON(!engine->mask);
@@ -1470,6 +1542,41 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)
desc->context_flags = CONTEXT_REGISTRATION_FLAG_KMD;
guc_context_policy_init(engine, desc);
+ /*
+ * If context is a parent, we need to register a process descriptor
+ * describing a work queue and register all child contexts.
+ */
+ if (intel_context_is_parent(ce)) {
+ struct guc_process_desc *pdesc;
+
+ ce->parallel.guc.wqi_tail = 0;
+ ce->parallel.guc.wqi_head = 0;
+
+ desc->process_desc = i915_ggtt_offset(ce->state) +
+ __get_process_desc_offset(ce);
+ desc->wq_addr = i915_ggtt_offset(ce->state) +
+ __get_wq_offset(ce);
+ desc->wq_size = WQ_SIZE;
+
+ pdesc = __get_process_desc(ce);
+ memset(pdesc, 0, sizeof(*(pdesc)));
+ pdesc->stage_id = ce->guc_id.id;
+ pdesc->wq_base_addr = desc->wq_addr;
+ pdesc->wq_size_bytes = desc->wq_size;
+ pdesc->wq_status = WQ_STATUS_ACTIVE;
+
+ for_each_child(ce, child) {
+ desc = __get_lrc_desc(guc, child->guc_id.id);
+
+ desc->engine_class =
+ engine_class_to_guc_class(engine->class);
+ desc->hw_context_desc = child->lrc.lrca;
+ desc->priority = ce->guc_state.prio;
+ desc->context_flags = CONTEXT_REGISTRATION_FLAG_KMD;
+ guc_context_policy_init(engine, desc);
+ }
+ }
+
/*
* The context_lookup xarray is used to determine if the hardware
* context is currently registered. There are two cases in which it
@@ -2804,6 +2911,12 @@ g2h_context_lookup(struct intel_guc *guc, u32 desc_idx)
return NULL;
}
+ if (unlikely(intel_context_is_child(ce))) {
+ drm_err(&guc_to_gt(guc)->i915->drm,
+ "Context is child, desc_idx %u", desc_idx);
+ return NULL;
+ }
+
return ce;
}
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 08/25] drm/i915/guc: Add multi-lrc context registration
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Add multi-lrc context registration H2G. In addition a workqueue and
process descriptor are setup during multi-lrc context registration as
these data structures are needed for multi-lrc submission.
v2:
(John Harrison)
- Move GuC specific fields into sub-struct
- Clean up WQ defines
- Add comment explaining math to derive WQ / PD address
v3:
(John Harrison)
- Add PARENT_SCRATCH_SIZE define
- Update comment explaining multi-lrc register
v4:
(John Harrison)
- Move PARENT_SCRATCH_SIZE to common file
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.h | 2 +
drivers/gpu/drm/i915/gt/intel_context_types.h | 12 ++
drivers/gpu/drm/i915/gt/intel_lrc.c | 5 +
.../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h | 1 +
drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 2 -
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 115 +++++++++++++++++-
6 files changed, 134 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
index b63c10a144af..9f0995150a7a 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.h
+++ b/drivers/gpu/drm/i915/gt/intel_context.h
@@ -44,6 +44,8 @@ void intel_context_free(struct intel_context *ce);
int intel_context_reconfigure_sseu(struct intel_context *ce,
const struct intel_sseu sseu);
+#define PARENT_SCRATCH_SIZE PAGE_SIZE
+
static inline bool intel_context_is_child(struct intel_context *ce)
{
return !!ce->parallel.parent;
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 76dfca57cb45..48decb5ee954 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -239,6 +239,18 @@ struct intel_context {
struct intel_context *parent;
/** @number_children: number of children if parent */
u8 number_children;
+ /** @guc: GuC specific members for parallel submission */
+ struct {
+ /** @wqi_head: head pointer in work queue */
+ u16 wqi_head;
+ /** @wqi_tail: tail pointer in work queue */
+ u16 wqi_tail;
+ /**
+ * @parent_page: page in context state (ce->state) used
+ * by parent for work queue, process descriptor
+ */
+ u8 parent_page;
+ } guc;
} parallel;
#ifdef CONFIG_DRM_I915_SELFTEST
diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
index 3ef9eaf8c50e..56156cf18c41 100644
--- a/drivers/gpu/drm/i915/gt/intel_lrc.c
+++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
@@ -942,6 +942,11 @@ __lrc_alloc_state(struct intel_context *ce, struct intel_engine_cs *engine)
context_size += PAGE_SIZE;
}
+ if (intel_context_is_parent(ce) && intel_engine_uses_guc(engine)) {
+ ce->parallel.guc.parent_page = context_size / PAGE_SIZE;
+ context_size += PARENT_SCRATCH_SIZE;
+ }
+
obj = i915_gem_object_create_lmem(engine->i915, context_size,
I915_BO_ALLOC_PM_VOLATILE);
if (IS_ERR(obj))
diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
index 8ff582222aff..ba10bd374cee 100644
--- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
+++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
@@ -142,6 +142,7 @@ enum intel_guc_action {
INTEL_GUC_ACTION_REGISTER_COMMAND_TRANSPORT_BUFFER = 0x4505,
INTEL_GUC_ACTION_DEREGISTER_COMMAND_TRANSPORT_BUFFER = 0x4506,
INTEL_GUC_ACTION_DEREGISTER_CONTEXT_DONE = 0x4600,
+ INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC = 0x4601,
INTEL_GUC_ACTION_RESET_CLIENT = 0x5507,
INTEL_GUC_ACTION_LIMIT
};
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
index fa4be13c8854..0eeb2a9feeed 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
@@ -52,8 +52,6 @@
#define GUC_DOORBELL_INVALID 256
-#define GUC_WQ_SIZE (PAGE_SIZE * 2)
-
/* Work queue item header definitions */
#define WQ_STATUS_ACTIVE 1
#define WQ_STATUS_SUSPENDED 2
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index fd6594bc1b96..d9f5be00e586 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -344,6 +344,46 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
return rb_entry(rb, struct i915_priolist, node);
}
+/*
+ * When using multi-lrc submission a scratch memory area is reserved in the
+ * parent's context state for the process descriptor and work queue. Currently
+ * the scratch area is sized to a page.
+ *
+ * The layout of this scratch area is below:
+ * 0 guc_process_desc
+ * ... unused
+ * PARENT_SCRATCH_SIZE / 2 work queue start
+ * ... work queue
+ * PARENT_SCRATCH_SIZE - 1 work queue end
+ */
+#define WQ_SIZE (PARENT_SCRATCH_SIZE / 2)
+#define WQ_OFFSET (PARENT_SCRATCH_SIZE - WQ_SIZE)
+static u32 __get_process_desc_offset(struct intel_context *ce)
+{
+ GEM_BUG_ON(!ce->parallel.guc.parent_page);
+
+ return ce->parallel.guc.parent_page * PAGE_SIZE;
+}
+
+static u32 __get_wq_offset(struct intel_context *ce)
+{
+ return __get_process_desc_offset(ce) + WQ_OFFSET;
+}
+
+static struct guc_process_desc *
+__get_process_desc(struct intel_context *ce)
+{
+ /*
+ * Need to subtract LRC_STATE_OFFSET here as the
+ * parallel.guc.parent_page is the offset into ce->state while
+ * ce->lrc_reg_reg is ce->state + LRC_STATE_OFFSET.
+ */
+ return (struct guc_process_desc *)
+ (ce->lrc_reg_state +
+ ((__get_process_desc_offset(ce) -
+ LRC_STATE_OFFSET) / sizeof(u32)));
+}
+
static struct guc_lrc_desc *__get_lrc_desc(struct intel_guc *guc, u32 index)
{
struct guc_lrc_desc *base = guc->lrc_desc_pool_vaddr;
@@ -1365,6 +1405,30 @@ static void unpin_guc_id(struct intel_guc *guc, struct intel_context *ce)
spin_unlock_irqrestore(&guc->submission_state.lock, flags);
}
+static int __guc_action_register_multi_lrc(struct intel_guc *guc,
+ struct intel_context *ce,
+ u32 guc_id,
+ u32 offset,
+ bool loop)
+{
+ struct intel_context *child;
+ u32 action[4 + MAX_ENGINE_INSTANCE];
+ int len = 0;
+
+ GEM_BUG_ON(ce->parallel.number_children > MAX_ENGINE_INSTANCE);
+
+ action[len++] = INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC;
+ action[len++] = guc_id;
+ action[len++] = ce->parallel.number_children + 1;
+ action[len++] = offset;
+ for_each_child(ce, child) {
+ offset += sizeof(struct guc_lrc_desc);
+ action[len++] = offset;
+ }
+
+ return guc_submission_send_busy_loop(guc, action, len, 0, loop);
+}
+
static int __guc_action_register_context(struct intel_guc *guc,
u32 guc_id,
u32 offset,
@@ -1387,9 +1451,15 @@ static int register_context(struct intel_context *ce, bool loop)
ce->guc_id.id * sizeof(struct guc_lrc_desc);
int ret;
+ GEM_BUG_ON(intel_context_is_child(ce));
trace_intel_context_register(ce);
- ret = __guc_action_register_context(guc, ce->guc_id.id, offset, loop);
+ if (intel_context_is_parent(ce))
+ ret = __guc_action_register_multi_lrc(guc, ce, ce->guc_id.id,
+ offset, loop);
+ else
+ ret = __guc_action_register_context(guc, ce->guc_id.id, offset,
+ loop);
if (likely(!ret)) {
unsigned long flags;
@@ -1418,6 +1488,7 @@ static int deregister_context(struct intel_context *ce, u32 guc_id)
{
struct intel_guc *guc = ce_to_guc(ce);
+ GEM_BUG_ON(intel_context_is_child(ce));
trace_intel_context_deregister(ce);
return __guc_action_deregister_context(guc, guc_id);
@@ -1445,6 +1516,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)
struct guc_lrc_desc *desc;
bool context_registered;
intel_wakeref_t wakeref;
+ struct intel_context *child;
int ret = 0;
GEM_BUG_ON(!engine->mask);
@@ -1470,6 +1542,41 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)
desc->context_flags = CONTEXT_REGISTRATION_FLAG_KMD;
guc_context_policy_init(engine, desc);
+ /*
+ * If context is a parent, we need to register a process descriptor
+ * describing a work queue and register all child contexts.
+ */
+ if (intel_context_is_parent(ce)) {
+ struct guc_process_desc *pdesc;
+
+ ce->parallel.guc.wqi_tail = 0;
+ ce->parallel.guc.wqi_head = 0;
+
+ desc->process_desc = i915_ggtt_offset(ce->state) +
+ __get_process_desc_offset(ce);
+ desc->wq_addr = i915_ggtt_offset(ce->state) +
+ __get_wq_offset(ce);
+ desc->wq_size = WQ_SIZE;
+
+ pdesc = __get_process_desc(ce);
+ memset(pdesc, 0, sizeof(*(pdesc)));
+ pdesc->stage_id = ce->guc_id.id;
+ pdesc->wq_base_addr = desc->wq_addr;
+ pdesc->wq_size_bytes = desc->wq_size;
+ pdesc->wq_status = WQ_STATUS_ACTIVE;
+
+ for_each_child(ce, child) {
+ desc = __get_lrc_desc(guc, child->guc_id.id);
+
+ desc->engine_class =
+ engine_class_to_guc_class(engine->class);
+ desc->hw_context_desc = child->lrc.lrca;
+ desc->priority = ce->guc_state.prio;
+ desc->context_flags = CONTEXT_REGISTRATION_FLAG_KMD;
+ guc_context_policy_init(engine, desc);
+ }
+ }
+
/*
* The context_lookup xarray is used to determine if the hardware
* context is currently registered. There are two cases in which it
@@ -2804,6 +2911,12 @@ g2h_context_lookup(struct intel_guc *guc, u32 desc_idx)
return NULL;
}
+ if (unlikely(intel_context_is_child(ce))) {
+ drm_err(&guc_to_gt(guc)->i915->drm,
+ "Context is child, desc_idx %u", desc_idx);
+ return NULL;
+ }
+
return ce;
}
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [Intel-gfx] [PATCH 08/25] drm/i915/guc: Add multi-lrc context registration
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 18:18 ` John Harrison
-1 siblings, 0 replies; 69+ messages in thread
From: John Harrison @ 2021-10-14 18:18 UTC (permalink / raw)
To: Matthew Brost, intel-gfx, dri-devel
On 10/14/2021 10:19, Matthew Brost wrote:
> Add multi-lrc context registration H2G. In addition a workqueue and
> process descriptor are setup during multi-lrc context registration as
> these data structures are needed for multi-lrc submission.
>
> v2:
> (John Harrison)
> - Move GuC specific fields into sub-struct
> - Clean up WQ defines
> - Add comment explaining math to derive WQ / PD address
> v3:
> (John Harrison)
> - Add PARENT_SCRATCH_SIZE define
> - Update comment explaining multi-lrc register
> v4:
> (John Harrison)
> - Move PARENT_SCRATCH_SIZE to common file
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gt/intel_context.h | 2 +
> drivers/gpu/drm/i915/gt/intel_context_types.h | 12 ++
> drivers/gpu/drm/i915/gt/intel_lrc.c | 5 +
> .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h | 1 +
> drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 2 -
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 115 +++++++++++++++++-
> 6 files changed, 134 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
> index b63c10a144af..9f0995150a7a 100644
> --- a/drivers/gpu/drm/i915/gt/intel_context.h
> +++ b/drivers/gpu/drm/i915/gt/intel_context.h
> @@ -44,6 +44,8 @@ void intel_context_free(struct intel_context *ce);
> int intel_context_reconfigure_sseu(struct intel_context *ce,
> const struct intel_sseu sseu);
>
> +#define PARENT_SCRATCH_SIZE PAGE_SIZE
Would have been nice to have a comment. At least something like 'For
multi-LRC submission, see uc/intel_guc_submission.c for details'. But
the description is there in the other file for those who want to look.
So either way:
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
> +
> static inline bool intel_context_is_child(struct intel_context *ce)
> {
> return !!ce->parallel.parent;
> diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
> index 76dfca57cb45..48decb5ee954 100644
> --- a/drivers/gpu/drm/i915/gt/intel_context_types.h
> +++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
> @@ -239,6 +239,18 @@ struct intel_context {
> struct intel_context *parent;
> /** @number_children: number of children if parent */
> u8 number_children;
> + /** @guc: GuC specific members for parallel submission */
> + struct {
> + /** @wqi_head: head pointer in work queue */
> + u16 wqi_head;
> + /** @wqi_tail: tail pointer in work queue */
> + u16 wqi_tail;
> + /**
> + * @parent_page: page in context state (ce->state) used
> + * by parent for work queue, process descriptor
> + */
> + u8 parent_page;
> + } guc;
> } parallel;
>
> #ifdef CONFIG_DRM_I915_SELFTEST
> diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
> index 3ef9eaf8c50e..56156cf18c41 100644
> --- a/drivers/gpu/drm/i915/gt/intel_lrc.c
> +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
> @@ -942,6 +942,11 @@ __lrc_alloc_state(struct intel_context *ce, struct intel_engine_cs *engine)
> context_size += PAGE_SIZE;
> }
>
> + if (intel_context_is_parent(ce) && intel_engine_uses_guc(engine)) {
> + ce->parallel.guc.parent_page = context_size / PAGE_SIZE;
> + context_size += PARENT_SCRATCH_SIZE;
> + }
> +
> obj = i915_gem_object_create_lmem(engine->i915, context_size,
> I915_BO_ALLOC_PM_VOLATILE);
> if (IS_ERR(obj))
> diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
> index 8ff582222aff..ba10bd374cee 100644
> --- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
> +++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
> @@ -142,6 +142,7 @@ enum intel_guc_action {
> INTEL_GUC_ACTION_REGISTER_COMMAND_TRANSPORT_BUFFER = 0x4505,
> INTEL_GUC_ACTION_DEREGISTER_COMMAND_TRANSPORT_BUFFER = 0x4506,
> INTEL_GUC_ACTION_DEREGISTER_CONTEXT_DONE = 0x4600,
> + INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC = 0x4601,
> INTEL_GUC_ACTION_RESET_CLIENT = 0x5507,
> INTEL_GUC_ACTION_LIMIT
> };
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
> index fa4be13c8854..0eeb2a9feeed 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
> @@ -52,8 +52,6 @@
>
> #define GUC_DOORBELL_INVALID 256
>
> -#define GUC_WQ_SIZE (PAGE_SIZE * 2)
> -
> /* Work queue item header definitions */
> #define WQ_STATUS_ACTIVE 1
> #define WQ_STATUS_SUSPENDED 2
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index fd6594bc1b96..d9f5be00e586 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -344,6 +344,46 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
> return rb_entry(rb, struct i915_priolist, node);
> }
>
> +/*
> + * When using multi-lrc submission a scratch memory area is reserved in the
> + * parent's context state for the process descriptor and work queue. Currently
> + * the scratch area is sized to a page.
> + *
> + * The layout of this scratch area is below:
> + * 0 guc_process_desc
> + * ... unused
> + * PARENT_SCRATCH_SIZE / 2 work queue start
> + * ... work queue
> + * PARENT_SCRATCH_SIZE - 1 work queue end
> + */
> +#define WQ_SIZE (PARENT_SCRATCH_SIZE / 2)
> +#define WQ_OFFSET (PARENT_SCRATCH_SIZE - WQ_SIZE)
> +static u32 __get_process_desc_offset(struct intel_context *ce)
> +{
> + GEM_BUG_ON(!ce->parallel.guc.parent_page);
> +
> + return ce->parallel.guc.parent_page * PAGE_SIZE;
> +}
> +
> +static u32 __get_wq_offset(struct intel_context *ce)
> +{
> + return __get_process_desc_offset(ce) + WQ_OFFSET;
> +}
> +
> +static struct guc_process_desc *
> +__get_process_desc(struct intel_context *ce)
> +{
> + /*
> + * Need to subtract LRC_STATE_OFFSET here as the
> + * parallel.guc.parent_page is the offset into ce->state while
> + * ce->lrc_reg_reg is ce->state + LRC_STATE_OFFSET.
> + */
> + return (struct guc_process_desc *)
> + (ce->lrc_reg_state +
> + ((__get_process_desc_offset(ce) -
> + LRC_STATE_OFFSET) / sizeof(u32)));
> +}
> +
> static struct guc_lrc_desc *__get_lrc_desc(struct intel_guc *guc, u32 index)
> {
> struct guc_lrc_desc *base = guc->lrc_desc_pool_vaddr;
> @@ -1365,6 +1405,30 @@ static void unpin_guc_id(struct intel_guc *guc, struct intel_context *ce)
> spin_unlock_irqrestore(&guc->submission_state.lock, flags);
> }
>
> +static int __guc_action_register_multi_lrc(struct intel_guc *guc,
> + struct intel_context *ce,
> + u32 guc_id,
> + u32 offset,
> + bool loop)
> +{
> + struct intel_context *child;
> + u32 action[4 + MAX_ENGINE_INSTANCE];
> + int len = 0;
> +
> + GEM_BUG_ON(ce->parallel.number_children > MAX_ENGINE_INSTANCE);
> +
> + action[len++] = INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC;
> + action[len++] = guc_id;
> + action[len++] = ce->parallel.number_children + 1;
> + action[len++] = offset;
> + for_each_child(ce, child) {
> + offset += sizeof(struct guc_lrc_desc);
> + action[len++] = offset;
> + }
> +
> + return guc_submission_send_busy_loop(guc, action, len, 0, loop);
> +}
> +
> static int __guc_action_register_context(struct intel_guc *guc,
> u32 guc_id,
> u32 offset,
> @@ -1387,9 +1451,15 @@ static int register_context(struct intel_context *ce, bool loop)
> ce->guc_id.id * sizeof(struct guc_lrc_desc);
> int ret;
>
> + GEM_BUG_ON(intel_context_is_child(ce));
> trace_intel_context_register(ce);
>
> - ret = __guc_action_register_context(guc, ce->guc_id.id, offset, loop);
> + if (intel_context_is_parent(ce))
> + ret = __guc_action_register_multi_lrc(guc, ce, ce->guc_id.id,
> + offset, loop);
> + else
> + ret = __guc_action_register_context(guc, ce->guc_id.id, offset,
> + loop);
> if (likely(!ret)) {
> unsigned long flags;
>
> @@ -1418,6 +1488,7 @@ static int deregister_context(struct intel_context *ce, u32 guc_id)
> {
> struct intel_guc *guc = ce_to_guc(ce);
>
> + GEM_BUG_ON(intel_context_is_child(ce));
> trace_intel_context_deregister(ce);
>
> return __guc_action_deregister_context(guc, guc_id);
> @@ -1445,6 +1516,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)
> struct guc_lrc_desc *desc;
> bool context_registered;
> intel_wakeref_t wakeref;
> + struct intel_context *child;
> int ret = 0;
>
> GEM_BUG_ON(!engine->mask);
> @@ -1470,6 +1542,41 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)
> desc->context_flags = CONTEXT_REGISTRATION_FLAG_KMD;
> guc_context_policy_init(engine, desc);
>
> + /*
> + * If context is a parent, we need to register a process descriptor
> + * describing a work queue and register all child contexts.
> + */
> + if (intel_context_is_parent(ce)) {
> + struct guc_process_desc *pdesc;
> +
> + ce->parallel.guc.wqi_tail = 0;
> + ce->parallel.guc.wqi_head = 0;
> +
> + desc->process_desc = i915_ggtt_offset(ce->state) +
> + __get_process_desc_offset(ce);
> + desc->wq_addr = i915_ggtt_offset(ce->state) +
> + __get_wq_offset(ce);
> + desc->wq_size = WQ_SIZE;
> +
> + pdesc = __get_process_desc(ce);
> + memset(pdesc, 0, sizeof(*(pdesc)));
> + pdesc->stage_id = ce->guc_id.id;
> + pdesc->wq_base_addr = desc->wq_addr;
> + pdesc->wq_size_bytes = desc->wq_size;
> + pdesc->wq_status = WQ_STATUS_ACTIVE;
> +
> + for_each_child(ce, child) {
> + desc = __get_lrc_desc(guc, child->guc_id.id);
> +
> + desc->engine_class =
> + engine_class_to_guc_class(engine->class);
> + desc->hw_context_desc = child->lrc.lrca;
> + desc->priority = ce->guc_state.prio;
> + desc->context_flags = CONTEXT_REGISTRATION_FLAG_KMD;
> + guc_context_policy_init(engine, desc);
> + }
> + }
> +
> /*
> * The context_lookup xarray is used to determine if the hardware
> * context is currently registered. There are two cases in which it
> @@ -2804,6 +2911,12 @@ g2h_context_lookup(struct intel_guc *guc, u32 desc_idx)
> return NULL;
> }
>
> + if (unlikely(intel_context_is_child(ce))) {
> + drm_err(&guc_to_gt(guc)->i915->drm,
> + "Context is child, desc_idx %u", desc_idx);
> + return NULL;
> + }
> +
> return ce;
> }
>
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 08/25] drm/i915/guc: Add multi-lrc context registration
@ 2021-10-14 18:18 ` John Harrison
0 siblings, 0 replies; 69+ messages in thread
From: John Harrison @ 2021-10-14 18:18 UTC (permalink / raw)
To: Matthew Brost, intel-gfx, dri-devel
On 10/14/2021 10:19, Matthew Brost wrote:
> Add multi-lrc context registration H2G. In addition a workqueue and
> process descriptor are setup during multi-lrc context registration as
> these data structures are needed for multi-lrc submission.
>
> v2:
> (John Harrison)
> - Move GuC specific fields into sub-struct
> - Clean up WQ defines
> - Add comment explaining math to derive WQ / PD address
> v3:
> (John Harrison)
> - Add PARENT_SCRATCH_SIZE define
> - Update comment explaining multi-lrc register
> v4:
> (John Harrison)
> - Move PARENT_SCRATCH_SIZE to common file
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gt/intel_context.h | 2 +
> drivers/gpu/drm/i915/gt/intel_context_types.h | 12 ++
> drivers/gpu/drm/i915/gt/intel_lrc.c | 5 +
> .../gpu/drm/i915/gt/uc/abi/guc_actions_abi.h | 1 +
> drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 2 -
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 115 +++++++++++++++++-
> 6 files changed, 134 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
> index b63c10a144af..9f0995150a7a 100644
> --- a/drivers/gpu/drm/i915/gt/intel_context.h
> +++ b/drivers/gpu/drm/i915/gt/intel_context.h
> @@ -44,6 +44,8 @@ void intel_context_free(struct intel_context *ce);
> int intel_context_reconfigure_sseu(struct intel_context *ce,
> const struct intel_sseu sseu);
>
> +#define PARENT_SCRATCH_SIZE PAGE_SIZE
Would have been nice to have a comment. At least something like 'For
multi-LRC submission, see uc/intel_guc_submission.c for details'. But
the description is there in the other file for those who want to look.
So either way:
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
> +
> static inline bool intel_context_is_child(struct intel_context *ce)
> {
> return !!ce->parallel.parent;
> diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
> index 76dfca57cb45..48decb5ee954 100644
> --- a/drivers/gpu/drm/i915/gt/intel_context_types.h
> +++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
> @@ -239,6 +239,18 @@ struct intel_context {
> struct intel_context *parent;
> /** @number_children: number of children if parent */
> u8 number_children;
> + /** @guc: GuC specific members for parallel submission */
> + struct {
> + /** @wqi_head: head pointer in work queue */
> + u16 wqi_head;
> + /** @wqi_tail: tail pointer in work queue */
> + u16 wqi_tail;
> + /**
> + * @parent_page: page in context state (ce->state) used
> + * by parent for work queue, process descriptor
> + */
> + u8 parent_page;
> + } guc;
> } parallel;
>
> #ifdef CONFIG_DRM_I915_SELFTEST
> diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
> index 3ef9eaf8c50e..56156cf18c41 100644
> --- a/drivers/gpu/drm/i915/gt/intel_lrc.c
> +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
> @@ -942,6 +942,11 @@ __lrc_alloc_state(struct intel_context *ce, struct intel_engine_cs *engine)
> context_size += PAGE_SIZE;
> }
>
> + if (intel_context_is_parent(ce) && intel_engine_uses_guc(engine)) {
> + ce->parallel.guc.parent_page = context_size / PAGE_SIZE;
> + context_size += PARENT_SCRATCH_SIZE;
> + }
> +
> obj = i915_gem_object_create_lmem(engine->i915, context_size,
> I915_BO_ALLOC_PM_VOLATILE);
> if (IS_ERR(obj))
> diff --git a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
> index 8ff582222aff..ba10bd374cee 100644
> --- a/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
> +++ b/drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
> @@ -142,6 +142,7 @@ enum intel_guc_action {
> INTEL_GUC_ACTION_REGISTER_COMMAND_TRANSPORT_BUFFER = 0x4505,
> INTEL_GUC_ACTION_DEREGISTER_COMMAND_TRANSPORT_BUFFER = 0x4506,
> INTEL_GUC_ACTION_DEREGISTER_CONTEXT_DONE = 0x4600,
> + INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC = 0x4601,
> INTEL_GUC_ACTION_RESET_CLIENT = 0x5507,
> INTEL_GUC_ACTION_LIMIT
> };
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
> index fa4be13c8854..0eeb2a9feeed 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
> @@ -52,8 +52,6 @@
>
> #define GUC_DOORBELL_INVALID 256
>
> -#define GUC_WQ_SIZE (PAGE_SIZE * 2)
> -
> /* Work queue item header definitions */
> #define WQ_STATUS_ACTIVE 1
> #define WQ_STATUS_SUSPENDED 2
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index fd6594bc1b96..d9f5be00e586 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -344,6 +344,46 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
> return rb_entry(rb, struct i915_priolist, node);
> }
>
> +/*
> + * When using multi-lrc submission a scratch memory area is reserved in the
> + * parent's context state for the process descriptor and work queue. Currently
> + * the scratch area is sized to a page.
> + *
> + * The layout of this scratch area is below:
> + * 0 guc_process_desc
> + * ... unused
> + * PARENT_SCRATCH_SIZE / 2 work queue start
> + * ... work queue
> + * PARENT_SCRATCH_SIZE - 1 work queue end
> + */
> +#define WQ_SIZE (PARENT_SCRATCH_SIZE / 2)
> +#define WQ_OFFSET (PARENT_SCRATCH_SIZE - WQ_SIZE)
> +static u32 __get_process_desc_offset(struct intel_context *ce)
> +{
> + GEM_BUG_ON(!ce->parallel.guc.parent_page);
> +
> + return ce->parallel.guc.parent_page * PAGE_SIZE;
> +}
> +
> +static u32 __get_wq_offset(struct intel_context *ce)
> +{
> + return __get_process_desc_offset(ce) + WQ_OFFSET;
> +}
> +
> +static struct guc_process_desc *
> +__get_process_desc(struct intel_context *ce)
> +{
> + /*
> + * Need to subtract LRC_STATE_OFFSET here as the
> + * parallel.guc.parent_page is the offset into ce->state while
> + * ce->lrc_reg_reg is ce->state + LRC_STATE_OFFSET.
> + */
> + return (struct guc_process_desc *)
> + (ce->lrc_reg_state +
> + ((__get_process_desc_offset(ce) -
> + LRC_STATE_OFFSET) / sizeof(u32)));
> +}
> +
> static struct guc_lrc_desc *__get_lrc_desc(struct intel_guc *guc, u32 index)
> {
> struct guc_lrc_desc *base = guc->lrc_desc_pool_vaddr;
> @@ -1365,6 +1405,30 @@ static void unpin_guc_id(struct intel_guc *guc, struct intel_context *ce)
> spin_unlock_irqrestore(&guc->submission_state.lock, flags);
> }
>
> +static int __guc_action_register_multi_lrc(struct intel_guc *guc,
> + struct intel_context *ce,
> + u32 guc_id,
> + u32 offset,
> + bool loop)
> +{
> + struct intel_context *child;
> + u32 action[4 + MAX_ENGINE_INSTANCE];
> + int len = 0;
> +
> + GEM_BUG_ON(ce->parallel.number_children > MAX_ENGINE_INSTANCE);
> +
> + action[len++] = INTEL_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC;
> + action[len++] = guc_id;
> + action[len++] = ce->parallel.number_children + 1;
> + action[len++] = offset;
> + for_each_child(ce, child) {
> + offset += sizeof(struct guc_lrc_desc);
> + action[len++] = offset;
> + }
> +
> + return guc_submission_send_busy_loop(guc, action, len, 0, loop);
> +}
> +
> static int __guc_action_register_context(struct intel_guc *guc,
> u32 guc_id,
> u32 offset,
> @@ -1387,9 +1451,15 @@ static int register_context(struct intel_context *ce, bool loop)
> ce->guc_id.id * sizeof(struct guc_lrc_desc);
> int ret;
>
> + GEM_BUG_ON(intel_context_is_child(ce));
> trace_intel_context_register(ce);
>
> - ret = __guc_action_register_context(guc, ce->guc_id.id, offset, loop);
> + if (intel_context_is_parent(ce))
> + ret = __guc_action_register_multi_lrc(guc, ce, ce->guc_id.id,
> + offset, loop);
> + else
> + ret = __guc_action_register_context(guc, ce->guc_id.id, offset,
> + loop);
> if (likely(!ret)) {
> unsigned long flags;
>
> @@ -1418,6 +1488,7 @@ static int deregister_context(struct intel_context *ce, u32 guc_id)
> {
> struct intel_guc *guc = ce_to_guc(ce);
>
> + GEM_BUG_ON(intel_context_is_child(ce));
> trace_intel_context_deregister(ce);
>
> return __guc_action_deregister_context(guc, guc_id);
> @@ -1445,6 +1516,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)
> struct guc_lrc_desc *desc;
> bool context_registered;
> intel_wakeref_t wakeref;
> + struct intel_context *child;
> int ret = 0;
>
> GEM_BUG_ON(!engine->mask);
> @@ -1470,6 +1542,41 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)
> desc->context_flags = CONTEXT_REGISTRATION_FLAG_KMD;
> guc_context_policy_init(engine, desc);
>
> + /*
> + * If context is a parent, we need to register a process descriptor
> + * describing a work queue and register all child contexts.
> + */
> + if (intel_context_is_parent(ce)) {
> + struct guc_process_desc *pdesc;
> +
> + ce->parallel.guc.wqi_tail = 0;
> + ce->parallel.guc.wqi_head = 0;
> +
> + desc->process_desc = i915_ggtt_offset(ce->state) +
> + __get_process_desc_offset(ce);
> + desc->wq_addr = i915_ggtt_offset(ce->state) +
> + __get_wq_offset(ce);
> + desc->wq_size = WQ_SIZE;
> +
> + pdesc = __get_process_desc(ce);
> + memset(pdesc, 0, sizeof(*(pdesc)));
> + pdesc->stage_id = ce->guc_id.id;
> + pdesc->wq_base_addr = desc->wq_addr;
> + pdesc->wq_size_bytes = desc->wq_size;
> + pdesc->wq_status = WQ_STATUS_ACTIVE;
> +
> + for_each_child(ce, child) {
> + desc = __get_lrc_desc(guc, child->guc_id.id);
> +
> + desc->engine_class =
> + engine_class_to_guc_class(engine->class);
> + desc->hw_context_desc = child->lrc.lrca;
> + desc->priority = ce->guc_state.prio;
> + desc->context_flags = CONTEXT_REGISTRATION_FLAG_KMD;
> + guc_context_policy_init(engine, desc);
> + }
> + }
> +
> /*
> * The context_lookup xarray is used to determine if the hardware
> * context is currently registered. There are two cases in which it
> @@ -2804,6 +2911,12 @@ g2h_context_lookup(struct intel_guc *guc, u32 desc_idx)
> return NULL;
> }
>
> + if (unlikely(intel_context_is_child(ce))) {
> + drm_err(&guc_to_gt(guc)->i915->drm,
> + "Context is child, desc_idx %u", desc_idx);
> + return NULL;
> + }
> +
> return ce;
> }
>
^ permalink raw reply [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 09/25] drm/i915/guc: Ensure GuC schedule operations do not operate on child contexts
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
In GuC parent-child contexts the parent context controls the scheduling,
ensure only the parent does the scheduling operations.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index d9f5be00e586..fbcf2dc2b2de 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -324,6 +324,12 @@ static inline void decr_context_committed_requests(struct intel_context *ce)
GEM_BUG_ON(ce->guc_state.number_committed_requests < 0);
}
+static struct intel_context *
+request_to_scheduling_context(struct i915_request *rq)
+{
+ return intel_context_to_parent(rq->context);
+}
+
static inline bool context_guc_id_invalid(struct intel_context *ce)
{
return ce->guc_id.id == GUC_INVALID_LRC_ID;
@@ -1711,6 +1717,7 @@ static void __guc_context_sched_disable(struct intel_guc *guc,
GEM_BUG_ON(guc_id == GUC_INVALID_LRC_ID);
+ GEM_BUG_ON(intel_context_is_child(ce));
trace_intel_context_sched_disable(ce);
guc_submission_send_busy_loop(guc, action, ARRAY_SIZE(action),
@@ -1936,6 +1943,8 @@ static void guc_context_sched_disable(struct intel_context *ce)
intel_wakeref_t wakeref;
u16 guc_id;
+ GEM_BUG_ON(intel_context_is_child(ce));
+
spin_lock_irqsave(&ce->guc_state.lock, flags);
/*
@@ -2304,6 +2313,8 @@ static void guc_signal_context_fence(struct intel_context *ce)
{
unsigned long flags;
+ GEM_BUG_ON(intel_context_is_child(ce));
+
spin_lock_irqsave(&ce->guc_state.lock, flags);
clr_context_wait_for_deregister_to_register(ce);
__guc_signal_context_fence(ce);
@@ -2334,7 +2345,7 @@ static void guc_context_init(struct intel_context *ce)
static int guc_request_alloc(struct i915_request *rq)
{
- struct intel_context *ce = rq->context;
+ struct intel_context *ce = request_to_scheduling_context(rq);
struct intel_guc *guc = ce_to_guc(ce);
unsigned long flags;
int ret;
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 09/25] drm/i915/guc: Ensure GuC schedule operations do not operate on child contexts
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
In GuC parent-child contexts the parent context controls the scheduling,
ensure only the parent does the scheduling operations.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index d9f5be00e586..fbcf2dc2b2de 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -324,6 +324,12 @@ static inline void decr_context_committed_requests(struct intel_context *ce)
GEM_BUG_ON(ce->guc_state.number_committed_requests < 0);
}
+static struct intel_context *
+request_to_scheduling_context(struct i915_request *rq)
+{
+ return intel_context_to_parent(rq->context);
+}
+
static inline bool context_guc_id_invalid(struct intel_context *ce)
{
return ce->guc_id.id == GUC_INVALID_LRC_ID;
@@ -1711,6 +1717,7 @@ static void __guc_context_sched_disable(struct intel_guc *guc,
GEM_BUG_ON(guc_id == GUC_INVALID_LRC_ID);
+ GEM_BUG_ON(intel_context_is_child(ce));
trace_intel_context_sched_disable(ce);
guc_submission_send_busy_loop(guc, action, ARRAY_SIZE(action),
@@ -1936,6 +1943,8 @@ static void guc_context_sched_disable(struct intel_context *ce)
intel_wakeref_t wakeref;
u16 guc_id;
+ GEM_BUG_ON(intel_context_is_child(ce));
+
spin_lock_irqsave(&ce->guc_state.lock, flags);
/*
@@ -2304,6 +2313,8 @@ static void guc_signal_context_fence(struct intel_context *ce)
{
unsigned long flags;
+ GEM_BUG_ON(intel_context_is_child(ce));
+
spin_lock_irqsave(&ce->guc_state.lock, flags);
clr_context_wait_for_deregister_to_register(ce);
__guc_signal_context_fence(ce);
@@ -2334,7 +2345,7 @@ static void guc_context_init(struct intel_context *ce)
static int guc_request_alloc(struct i915_request *rq)
{
- struct intel_context *ce = rq->context;
+ struct intel_context *ce = request_to_scheduling_context(rq);
struct intel_guc *guc = ce_to_guc(ce);
unsigned long flags;
int ret;
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 10/25] drm/i915/guc: Assign contexts in parent-child relationship consecutive guc_ids
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Assign contexts in parent-child relationship consecutive guc_ids. This
is accomplished by partitioning guc_id space between ones that need to
be consecutive (1/16 available guc_ids) and ones that do not (15/16 of
available guc_ids). The consecutive search is implemented via the bitmap
API.
This is a precursor to the full GuC multi-lrc implementation but aligns
to how GuC mutli-lrc interface is defined - guc_ids must be consecutive
when using the GuC multi-lrc interface.
v2:
(Daniel Vetter)
- Explicitly state why we assign consecutive guc_ids
v3:
(John Harrison)
- Bring back in spin lock
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/uc/intel_guc.h | 6 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 104 ++++++++++++++----
2 files changed, 86 insertions(+), 24 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
index 74f071a0b6d5..4ca197f400ba 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
@@ -82,9 +82,13 @@ struct intel_guc {
*/
spinlock_t lock;
/**
- * @guc_ids: used to allocate new guc_ids
+ * @guc_ids: used to allocate new guc_ids, single-lrc
*/
struct ida guc_ids;
+ /**
+ * @guc_ids_bitmap: used to allocate new guc_ids, multi-lrc
+ */
+ unsigned long *guc_ids_bitmap;
/**
* @guc_id_list: list of intel_context with valid guc_ids but no
* refs
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index fbcf2dc2b2de..7ba4f1d6f69c 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -128,6 +128,16 @@ guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count);
#define GUC_REQUEST_SIZE 64 /* bytes */
+/*
+ * We reserve 1/16 of the guc_ids for multi-lrc as these need to be contiguous
+ * per the GuC submission interface. A different allocation algorithm is used
+ * (bitmap vs. ida) between multi-lrc and single-lrc hence the reason to
+ * partition the guc_id space. We believe the number of multi-lrc contexts in
+ * use should be low and 1/16 should be sufficient. Minimum of 32 guc_ids for
+ * multi-lrc.
+ */
+#define NUMBER_MULTI_LRC_GUC_ID (GUC_MAX_LRC_DESCRIPTORS / 16)
+
/*
* Below is a set of functions which control the GuC scheduling state which
* require a lock.
@@ -1207,6 +1217,11 @@ int intel_guc_submission_init(struct intel_guc *guc)
INIT_WORK(&guc->submission_state.destroyed_worker,
destroyed_worker_func);
+ guc->submission_state.guc_ids_bitmap =
+ bitmap_zalloc(NUMBER_MULTI_LRC_GUC_ID, GFP_KERNEL);
+ if (!guc->submission_state.guc_ids_bitmap)
+ return -ENOMEM;
+
return 0;
}
@@ -1218,6 +1233,7 @@ void intel_guc_submission_fini(struct intel_guc *guc)
guc_flush_destroyed_contexts(guc);
guc_lrc_desc_pool_destroy(guc);
i915_sched_engine_put(guc->sched_engine);
+ bitmap_free(guc->submission_state.guc_ids_bitmap);
}
static inline void queue_request(struct i915_sched_engine *sched_engine,
@@ -1269,18 +1285,43 @@ static void guc_submit_request(struct i915_request *rq)
spin_unlock_irqrestore(&sched_engine->lock, flags);
}
-static int new_guc_id(struct intel_guc *guc)
+static int new_guc_id(struct intel_guc *guc, struct intel_context *ce)
{
- return ida_simple_get(&guc->submission_state.guc_ids, 0,
- GUC_MAX_LRC_DESCRIPTORS, GFP_KERNEL |
- __GFP_RETRY_MAYFAIL | __GFP_NOWARN);
+ int ret;
+
+ GEM_BUG_ON(intel_context_is_child(ce));
+
+ if (intel_context_is_parent(ce))
+ ret = bitmap_find_free_region(guc->submission_state.guc_ids_bitmap,
+ NUMBER_MULTI_LRC_GUC_ID,
+ order_base_2(ce->parallel.number_children
+ + 1));
+ else
+ ret = ida_simple_get(&guc->submission_state.guc_ids,
+ NUMBER_MULTI_LRC_GUC_ID,
+ GUC_MAX_LRC_DESCRIPTORS,
+ GFP_KERNEL | __GFP_RETRY_MAYFAIL |
+ __GFP_NOWARN);
+ if (unlikely(ret < 0))
+ return ret;
+
+ ce->guc_id.id = ret;
+ return 0;
}
static void __release_guc_id(struct intel_guc *guc, struct intel_context *ce)
{
+ GEM_BUG_ON(intel_context_is_child(ce));
+
if (!context_guc_id_invalid(ce)) {
- ida_simple_remove(&guc->submission_state.guc_ids,
- ce->guc_id.id);
+ if (intel_context_is_parent(ce))
+ bitmap_release_region(guc->submission_state.guc_ids_bitmap,
+ ce->guc_id.id,
+ order_base_2(ce->parallel.number_children
+ + 1));
+ else
+ ida_simple_remove(&guc->submission_state.guc_ids,
+ ce->guc_id.id);
reset_lrc_desc(guc, ce->guc_id.id);
set_context_guc_id_invalid(ce);
}
@@ -1297,49 +1338,64 @@ static void release_guc_id(struct intel_guc *guc, struct intel_context *ce)
spin_unlock_irqrestore(&guc->submission_state.lock, flags);
}
-static int steal_guc_id(struct intel_guc *guc)
+static int steal_guc_id(struct intel_guc *guc, struct intel_context *ce)
{
- struct intel_context *ce;
- int guc_id;
+ struct intel_context *cn;
lockdep_assert_held(&guc->submission_state.lock);
+ GEM_BUG_ON(intel_context_is_child(ce));
+ GEM_BUG_ON(intel_context_is_parent(ce));
if (!list_empty(&guc->submission_state.guc_id_list)) {
- ce = list_first_entry(&guc->submission_state.guc_id_list,
+ cn = list_first_entry(&guc->submission_state.guc_id_list,
struct intel_context,
guc_id.link);
- GEM_BUG_ON(atomic_read(&ce->guc_id.ref));
- GEM_BUG_ON(context_guc_id_invalid(ce));
+ GEM_BUG_ON(atomic_read(&cn->guc_id.ref));
+ GEM_BUG_ON(context_guc_id_invalid(cn));
+ GEM_BUG_ON(intel_context_is_child(cn));
+ GEM_BUG_ON(intel_context_is_parent(cn));
- list_del_init(&ce->guc_id.link);
- guc_id = ce->guc_id.id;
+ list_del_init(&cn->guc_id.link);
+ ce->guc_id = cn->guc_id;
spin_lock(&ce->guc_state.lock);
- clr_context_registered(ce);
+ clr_context_registered(cn);
spin_unlock(&ce->guc_state.lock);
- set_context_guc_id_invalid(ce);
- return guc_id;
+ set_context_guc_id_invalid(cn);
+
+ return 0;
} else {
return -EAGAIN;
}
}
-static int assign_guc_id(struct intel_guc *guc, u16 *out)
+static int assign_guc_id(struct intel_guc *guc, struct intel_context *ce)
{
int ret;
lockdep_assert_held(&guc->submission_state.lock);
+ GEM_BUG_ON(intel_context_is_child(ce));
- ret = new_guc_id(guc);
+ ret = new_guc_id(guc, ce);
if (unlikely(ret < 0)) {
- ret = steal_guc_id(guc);
+ if (intel_context_is_parent(ce))
+ return -ENOSPC;
+
+ ret = steal_guc_id(guc, ce);
if (ret < 0)
return ret;
}
- *out = ret;
+ if (intel_context_is_parent(ce)) {
+ struct intel_context *child;
+ int i = 1;
+
+ for_each_child(ce, child)
+ child->guc_id.id = ce->guc_id.id + i++;
+ }
+
return 0;
}
@@ -1357,7 +1413,7 @@ static int pin_guc_id(struct intel_guc *guc, struct intel_context *ce)
might_lock(&ce->guc_state.lock);
if (context_guc_id_invalid(ce)) {
- ret = assign_guc_id(guc, &ce->guc_id.id);
+ ret = assign_guc_id(guc, ce);
if (ret)
goto out_unlock;
ret = 1; /* Indidcates newly assigned guc_id */
@@ -1399,8 +1455,10 @@ static void unpin_guc_id(struct intel_guc *guc, struct intel_context *ce)
unsigned long flags;
GEM_BUG_ON(atomic_read(&ce->guc_id.ref) < 0);
+ GEM_BUG_ON(intel_context_is_child(ce));
- if (unlikely(context_guc_id_invalid(ce)))
+ if (unlikely(context_guc_id_invalid(ce) ||
+ intel_context_is_parent(ce)))
return;
spin_lock_irqsave(&guc->submission_state.lock, flags);
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 10/25] drm/i915/guc: Assign contexts in parent-child relationship consecutive guc_ids
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Assign contexts in parent-child relationship consecutive guc_ids. This
is accomplished by partitioning guc_id space between ones that need to
be consecutive (1/16 available guc_ids) and ones that do not (15/16 of
available guc_ids). The consecutive search is implemented via the bitmap
API.
This is a precursor to the full GuC multi-lrc implementation but aligns
to how GuC mutli-lrc interface is defined - guc_ids must be consecutive
when using the GuC multi-lrc interface.
v2:
(Daniel Vetter)
- Explicitly state why we assign consecutive guc_ids
v3:
(John Harrison)
- Bring back in spin lock
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/uc/intel_guc.h | 6 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 104 ++++++++++++++----
2 files changed, 86 insertions(+), 24 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
index 74f071a0b6d5..4ca197f400ba 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
@@ -82,9 +82,13 @@ struct intel_guc {
*/
spinlock_t lock;
/**
- * @guc_ids: used to allocate new guc_ids
+ * @guc_ids: used to allocate new guc_ids, single-lrc
*/
struct ida guc_ids;
+ /**
+ * @guc_ids_bitmap: used to allocate new guc_ids, multi-lrc
+ */
+ unsigned long *guc_ids_bitmap;
/**
* @guc_id_list: list of intel_context with valid guc_ids but no
* refs
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index fbcf2dc2b2de..7ba4f1d6f69c 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -128,6 +128,16 @@ guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count);
#define GUC_REQUEST_SIZE 64 /* bytes */
+/*
+ * We reserve 1/16 of the guc_ids for multi-lrc as these need to be contiguous
+ * per the GuC submission interface. A different allocation algorithm is used
+ * (bitmap vs. ida) between multi-lrc and single-lrc hence the reason to
+ * partition the guc_id space. We believe the number of multi-lrc contexts in
+ * use should be low and 1/16 should be sufficient. Minimum of 32 guc_ids for
+ * multi-lrc.
+ */
+#define NUMBER_MULTI_LRC_GUC_ID (GUC_MAX_LRC_DESCRIPTORS / 16)
+
/*
* Below is a set of functions which control the GuC scheduling state which
* require a lock.
@@ -1207,6 +1217,11 @@ int intel_guc_submission_init(struct intel_guc *guc)
INIT_WORK(&guc->submission_state.destroyed_worker,
destroyed_worker_func);
+ guc->submission_state.guc_ids_bitmap =
+ bitmap_zalloc(NUMBER_MULTI_LRC_GUC_ID, GFP_KERNEL);
+ if (!guc->submission_state.guc_ids_bitmap)
+ return -ENOMEM;
+
return 0;
}
@@ -1218,6 +1233,7 @@ void intel_guc_submission_fini(struct intel_guc *guc)
guc_flush_destroyed_contexts(guc);
guc_lrc_desc_pool_destroy(guc);
i915_sched_engine_put(guc->sched_engine);
+ bitmap_free(guc->submission_state.guc_ids_bitmap);
}
static inline void queue_request(struct i915_sched_engine *sched_engine,
@@ -1269,18 +1285,43 @@ static void guc_submit_request(struct i915_request *rq)
spin_unlock_irqrestore(&sched_engine->lock, flags);
}
-static int new_guc_id(struct intel_guc *guc)
+static int new_guc_id(struct intel_guc *guc, struct intel_context *ce)
{
- return ida_simple_get(&guc->submission_state.guc_ids, 0,
- GUC_MAX_LRC_DESCRIPTORS, GFP_KERNEL |
- __GFP_RETRY_MAYFAIL | __GFP_NOWARN);
+ int ret;
+
+ GEM_BUG_ON(intel_context_is_child(ce));
+
+ if (intel_context_is_parent(ce))
+ ret = bitmap_find_free_region(guc->submission_state.guc_ids_bitmap,
+ NUMBER_MULTI_LRC_GUC_ID,
+ order_base_2(ce->parallel.number_children
+ + 1));
+ else
+ ret = ida_simple_get(&guc->submission_state.guc_ids,
+ NUMBER_MULTI_LRC_GUC_ID,
+ GUC_MAX_LRC_DESCRIPTORS,
+ GFP_KERNEL | __GFP_RETRY_MAYFAIL |
+ __GFP_NOWARN);
+ if (unlikely(ret < 0))
+ return ret;
+
+ ce->guc_id.id = ret;
+ return 0;
}
static void __release_guc_id(struct intel_guc *guc, struct intel_context *ce)
{
+ GEM_BUG_ON(intel_context_is_child(ce));
+
if (!context_guc_id_invalid(ce)) {
- ida_simple_remove(&guc->submission_state.guc_ids,
- ce->guc_id.id);
+ if (intel_context_is_parent(ce))
+ bitmap_release_region(guc->submission_state.guc_ids_bitmap,
+ ce->guc_id.id,
+ order_base_2(ce->parallel.number_children
+ + 1));
+ else
+ ida_simple_remove(&guc->submission_state.guc_ids,
+ ce->guc_id.id);
reset_lrc_desc(guc, ce->guc_id.id);
set_context_guc_id_invalid(ce);
}
@@ -1297,49 +1338,64 @@ static void release_guc_id(struct intel_guc *guc, struct intel_context *ce)
spin_unlock_irqrestore(&guc->submission_state.lock, flags);
}
-static int steal_guc_id(struct intel_guc *guc)
+static int steal_guc_id(struct intel_guc *guc, struct intel_context *ce)
{
- struct intel_context *ce;
- int guc_id;
+ struct intel_context *cn;
lockdep_assert_held(&guc->submission_state.lock);
+ GEM_BUG_ON(intel_context_is_child(ce));
+ GEM_BUG_ON(intel_context_is_parent(ce));
if (!list_empty(&guc->submission_state.guc_id_list)) {
- ce = list_first_entry(&guc->submission_state.guc_id_list,
+ cn = list_first_entry(&guc->submission_state.guc_id_list,
struct intel_context,
guc_id.link);
- GEM_BUG_ON(atomic_read(&ce->guc_id.ref));
- GEM_BUG_ON(context_guc_id_invalid(ce));
+ GEM_BUG_ON(atomic_read(&cn->guc_id.ref));
+ GEM_BUG_ON(context_guc_id_invalid(cn));
+ GEM_BUG_ON(intel_context_is_child(cn));
+ GEM_BUG_ON(intel_context_is_parent(cn));
- list_del_init(&ce->guc_id.link);
- guc_id = ce->guc_id.id;
+ list_del_init(&cn->guc_id.link);
+ ce->guc_id = cn->guc_id;
spin_lock(&ce->guc_state.lock);
- clr_context_registered(ce);
+ clr_context_registered(cn);
spin_unlock(&ce->guc_state.lock);
- set_context_guc_id_invalid(ce);
- return guc_id;
+ set_context_guc_id_invalid(cn);
+
+ return 0;
} else {
return -EAGAIN;
}
}
-static int assign_guc_id(struct intel_guc *guc, u16 *out)
+static int assign_guc_id(struct intel_guc *guc, struct intel_context *ce)
{
int ret;
lockdep_assert_held(&guc->submission_state.lock);
+ GEM_BUG_ON(intel_context_is_child(ce));
- ret = new_guc_id(guc);
+ ret = new_guc_id(guc, ce);
if (unlikely(ret < 0)) {
- ret = steal_guc_id(guc);
+ if (intel_context_is_parent(ce))
+ return -ENOSPC;
+
+ ret = steal_guc_id(guc, ce);
if (ret < 0)
return ret;
}
- *out = ret;
+ if (intel_context_is_parent(ce)) {
+ struct intel_context *child;
+ int i = 1;
+
+ for_each_child(ce, child)
+ child->guc_id.id = ce->guc_id.id + i++;
+ }
+
return 0;
}
@@ -1357,7 +1413,7 @@ static int pin_guc_id(struct intel_guc *guc, struct intel_context *ce)
might_lock(&ce->guc_state.lock);
if (context_guc_id_invalid(ce)) {
- ret = assign_guc_id(guc, &ce->guc_id.id);
+ ret = assign_guc_id(guc, ce);
if (ret)
goto out_unlock;
ret = 1; /* Indidcates newly assigned guc_id */
@@ -1399,8 +1455,10 @@ static void unpin_guc_id(struct intel_guc *guc, struct intel_context *ce)
unsigned long flags;
GEM_BUG_ON(atomic_read(&ce->guc_id.ref) < 0);
+ GEM_BUG_ON(intel_context_is_child(ce));
- if (unlikely(context_guc_id_invalid(ce)))
+ if (unlikely(context_guc_id_invalid(ce) ||
+ intel_context_is_parent(ce)))
return;
spin_lock_irqsave(&guc->submission_state.lock, flags);
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 11/25] drm/i915/guc: Implement parallel context pin / unpin functions
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Parallel contexts are perma-pinned by the upper layers which makes the
backend implementation rather simple. The parent pins the guc_id and
children increment the parent's pin count on pin to ensure all the
contexts are unpinned before we disable scheduling with the GuC / or
deregister the context.
v2:
(Daniel Vetter)
- Perma-pin parallel contexts
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 70 +++++++++++++++++++
1 file changed, 70 insertions(+)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 7ba4f1d6f69c..a412593342db 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -2584,6 +2584,76 @@ static const struct intel_context_ops virtual_guc_context_ops = {
.get_sibling = guc_virtual_get_sibling,
};
+/* Future patches will use this function */
+__maybe_unused
+static int guc_parent_context_pin(struct intel_context *ce, void *vaddr)
+{
+ struct intel_engine_cs *engine = guc_virtual_get_sibling(ce->engine, 0);
+ struct intel_guc *guc = ce_to_guc(ce);
+ int ret;
+
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+ GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
+
+ ret = pin_guc_id(guc, ce);
+ if (unlikely(ret < 0))
+ return ret;
+
+ return __guc_context_pin(ce, engine, vaddr);
+}
+
+/* Future patches will use this function */
+__maybe_unused
+static int guc_child_context_pin(struct intel_context *ce, void *vaddr)
+{
+ struct intel_engine_cs *engine = guc_virtual_get_sibling(ce->engine, 0);
+
+ GEM_BUG_ON(!intel_context_is_child(ce));
+ GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
+
+ __intel_context_pin(ce->parallel.parent);
+ return __guc_context_pin(ce, engine, vaddr);
+}
+
+/* Future patches will use this function */
+__maybe_unused
+static void guc_parent_context_unpin(struct intel_context *ce)
+{
+ struct intel_guc *guc = ce_to_guc(ce);
+
+ GEM_BUG_ON(context_enabled(ce));
+ GEM_BUG_ON(intel_context_is_barrier(ce));
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+ GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
+
+ unpin_guc_id(guc, ce);
+ lrc_unpin(ce);
+}
+
+/* Future patches will use this function */
+__maybe_unused
+static void guc_child_context_unpin(struct intel_context *ce)
+{
+ GEM_BUG_ON(context_enabled(ce));
+ GEM_BUG_ON(intel_context_is_barrier(ce));
+ GEM_BUG_ON(!intel_context_is_child(ce));
+ GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
+
+ lrc_unpin(ce);
+}
+
+/* Future patches will use this function */
+__maybe_unused
+static void guc_child_context_post_unpin(struct intel_context *ce)
+{
+ GEM_BUG_ON(!intel_context_is_child(ce));
+ GEM_BUG_ON(!intel_context_is_pinned(ce->parallel.parent));
+ GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
+
+ lrc_post_unpin(ce);
+ intel_context_unpin(ce->parallel.parent);
+}
+
static bool
guc_irq_enable_breadcrumbs(struct intel_breadcrumbs *b)
{
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 11/25] drm/i915/guc: Implement parallel context pin / unpin functions
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Parallel contexts are perma-pinned by the upper layers which makes the
backend implementation rather simple. The parent pins the guc_id and
children increment the parent's pin count on pin to ensure all the
contexts are unpinned before we disable scheduling with the GuC / or
deregister the context.
v2:
(Daniel Vetter)
- Perma-pin parallel contexts
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 70 +++++++++++++++++++
1 file changed, 70 insertions(+)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 7ba4f1d6f69c..a412593342db 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -2584,6 +2584,76 @@ static const struct intel_context_ops virtual_guc_context_ops = {
.get_sibling = guc_virtual_get_sibling,
};
+/* Future patches will use this function */
+__maybe_unused
+static int guc_parent_context_pin(struct intel_context *ce, void *vaddr)
+{
+ struct intel_engine_cs *engine = guc_virtual_get_sibling(ce->engine, 0);
+ struct intel_guc *guc = ce_to_guc(ce);
+ int ret;
+
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+ GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
+
+ ret = pin_guc_id(guc, ce);
+ if (unlikely(ret < 0))
+ return ret;
+
+ return __guc_context_pin(ce, engine, vaddr);
+}
+
+/* Future patches will use this function */
+__maybe_unused
+static int guc_child_context_pin(struct intel_context *ce, void *vaddr)
+{
+ struct intel_engine_cs *engine = guc_virtual_get_sibling(ce->engine, 0);
+
+ GEM_BUG_ON(!intel_context_is_child(ce));
+ GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
+
+ __intel_context_pin(ce->parallel.parent);
+ return __guc_context_pin(ce, engine, vaddr);
+}
+
+/* Future patches will use this function */
+__maybe_unused
+static void guc_parent_context_unpin(struct intel_context *ce)
+{
+ struct intel_guc *guc = ce_to_guc(ce);
+
+ GEM_BUG_ON(context_enabled(ce));
+ GEM_BUG_ON(intel_context_is_barrier(ce));
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+ GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
+
+ unpin_guc_id(guc, ce);
+ lrc_unpin(ce);
+}
+
+/* Future patches will use this function */
+__maybe_unused
+static void guc_child_context_unpin(struct intel_context *ce)
+{
+ GEM_BUG_ON(context_enabled(ce));
+ GEM_BUG_ON(intel_context_is_barrier(ce));
+ GEM_BUG_ON(!intel_context_is_child(ce));
+ GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
+
+ lrc_unpin(ce);
+}
+
+/* Future patches will use this function */
+__maybe_unused
+static void guc_child_context_post_unpin(struct intel_context *ce)
+{
+ GEM_BUG_ON(!intel_context_is_child(ce));
+ GEM_BUG_ON(!intel_context_is_pinned(ce->parallel.parent));
+ GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
+
+ lrc_post_unpin(ce);
+ intel_context_unpin(ce->parallel.parent);
+}
+
static bool
guc_irq_enable_breadcrumbs(struct intel_breadcrumbs *b)
{
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 12/25] drm/i915/guc: Implement multi-lrc submission
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Implement multi-lrc submission via a single workqueue entry and single
H2G. The workqueue entry contains an updated tail value for each
request, of all the contexts in the multi-lrc submission, and updates
these values simultaneously. As such, the tasklet and bypass path have
been updated to coalesce requests into a single submission.
v2:
(John Harrison)
- s/wqe/wqi
- Use FIELD_PREP macros
- Add GEM_BUG_ONs ensures length fits within field
- Add comment / white space to intel_guc_write_barrier
(Kernel test robot)
- Make need_tasklet a static function
v3:
(Docs)
- A comment for submission_stall_reason
v4:
(Kernel test robot)
- Initialize return value in bypass tasklt submit function
(John Harrison)
- Add comment near work queue defs
- Add BUILD_BUG_ON to ensure WQ_SIZE is a power of 2
- Update write_barrier comment to talk about work queue
v5:
(John Harrison)
- Fix typo in work queue comment
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/uc/intel_guc.c | 29 ++
drivers/gpu/drm/i915/gt/uc/intel_guc.h | 11 +
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 24 +-
drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 30 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 323 +++++++++++++++---
drivers/gpu/drm/i915/i915_request.h | 8 +
6 files changed, 350 insertions(+), 75 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.c b/drivers/gpu/drm/i915/gt/uc/intel_guc.c
index 8f8182bf7c11..6e228343e8cb 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.c
@@ -756,3 +756,32 @@ void intel_guc_load_status(struct intel_guc *guc, struct drm_printer *p)
}
}
}
+
+void intel_guc_write_barrier(struct intel_guc *guc)
+{
+ struct intel_gt *gt = guc_to_gt(guc);
+
+ if (i915_gem_object_is_lmem(guc->ct.vma->obj)) {
+ /*
+ * Ensure intel_uncore_write_fw can be used rather than
+ * intel_uncore_write.
+ */
+ GEM_BUG_ON(guc->send_regs.fw_domains);
+
+ /*
+ * This register is used by the i915 and GuC for MMIO based
+ * communication. Once we are in this code CTBs are the only
+ * method the i915 uses to communicate with the GuC so it is
+ * safe to write to this register (a value of 0 is NOP for MMIO
+ * communication). If we ever start mixing CTBs and MMIOs a new
+ * register will have to be chosen. This function is also used
+ * to enforce ordering of a work queue item write and an update
+ * to the process descriptor. When a work queue is being used,
+ * CTBs are also the only mechanism of communication.
+ */
+ intel_uncore_write_fw(gt->uncore, GEN11_SOFT_SCRATCH(0), 0);
+ } else {
+ /* wmb() sufficient for a barrier if in smem */
+ wmb();
+ }
+}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
index 4ca197f400ba..31cf9fb48c7e 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
@@ -46,6 +46,15 @@ struct intel_guc {
* submitted until the stalled request is processed.
*/
struct i915_request *stalled_request;
+ /**
+ * @submission_stall_reason: reason why submission is stalled
+ */
+ enum {
+ STALL_NONE,
+ STALL_REGISTER_CONTEXT,
+ STALL_MOVE_LRC_TAIL,
+ STALL_ADD_REQUEST,
+ } submission_stall_reason;
/* intel_guc_recv interrupt related state */
/** @irq_lock: protects GuC irq state */
@@ -367,4 +376,6 @@ void intel_guc_submission_cancel_requests(struct intel_guc *guc);
void intel_guc_load_status(struct intel_guc *guc, struct drm_printer *p);
+void intel_guc_write_barrier(struct intel_guc *guc);
+
#endif
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
index 0a3504bc0b61..a0cc34be7b56 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
@@ -383,28 +383,6 @@ static u32 ct_get_next_fence(struct intel_guc_ct *ct)
return ++ct->requests.last_fence;
}
-static void write_barrier(struct intel_guc_ct *ct)
-{
- struct intel_guc *guc = ct_to_guc(ct);
- struct intel_gt *gt = guc_to_gt(guc);
-
- if (i915_gem_object_is_lmem(guc->ct.vma->obj)) {
- GEM_BUG_ON(guc->send_regs.fw_domains);
- /*
- * This register is used by the i915 and GuC for MMIO based
- * communication. Once we are in this code CTBs are the only
- * method the i915 uses to communicate with the GuC so it is
- * safe to write to this register (a value of 0 is NOP for MMIO
- * communication). If we ever start mixing CTBs and MMIOs a new
- * register will have to be chosen.
- */
- intel_uncore_write_fw(gt->uncore, GEN11_SOFT_SCRATCH(0), 0);
- } else {
- /* wmb() sufficient for a barrier if in smem */
- wmb();
- }
-}
-
static int ct_write(struct intel_guc_ct *ct,
const u32 *action,
u32 len /* in dwords */,
@@ -474,7 +452,7 @@ static int ct_write(struct intel_guc_ct *ct,
* make sure H2G buffer update and LRC tail update (if this triggering a
* submission) are visible before updating the descriptor tail
*/
- write_barrier(ct);
+ intel_guc_write_barrier(ct_to_guc(ct));
/* update local copies */
ctb->tail = tail;
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
index 0eeb2a9feeed..18da67cfcd92 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
@@ -52,25 +52,27 @@
#define GUC_DOORBELL_INVALID 256
-/* Work queue item header definitions */
+/*
+ * Work queue item header definitions
+ *
+ * Work queue is circular buffer used to submit complex (multi-lrc) submissions
+ * to the GuC. A work queue item is an entry in the circular buffer.
+ */
#define WQ_STATUS_ACTIVE 1
#define WQ_STATUS_SUSPENDED 2
#define WQ_STATUS_CMD_ERROR 3
#define WQ_STATUS_ENGINE_ID_NOT_USED 4
#define WQ_STATUS_SUSPENDED_FROM_RESET 5
-#define WQ_TYPE_SHIFT 0
-#define WQ_TYPE_BATCH_BUF (0x1 << WQ_TYPE_SHIFT)
-#define WQ_TYPE_PSEUDO (0x2 << WQ_TYPE_SHIFT)
-#define WQ_TYPE_INORDER (0x3 << WQ_TYPE_SHIFT)
-#define WQ_TYPE_NOOP (0x4 << WQ_TYPE_SHIFT)
-#define WQ_TARGET_SHIFT 10
-#define WQ_LEN_SHIFT 16
-#define WQ_NO_WCFLUSH_WAIT (1 << 27)
-#define WQ_PRESENT_WORKLOAD (1 << 28)
-
-#define WQ_RING_TAIL_SHIFT 20
-#define WQ_RING_TAIL_MAX 0x7FF /* 2^11 QWords */
-#define WQ_RING_TAIL_MASK (WQ_RING_TAIL_MAX << WQ_RING_TAIL_SHIFT)
+#define WQ_TYPE_BATCH_BUF 0x1
+#define WQ_TYPE_PSEUDO 0x2
+#define WQ_TYPE_INORDER 0x3
+#define WQ_TYPE_NOOP 0x4
+#define WQ_TYPE_MULTI_LRC 0x5
+#define WQ_TYPE_MASK GENMASK(7, 0)
+#define WQ_LEN_MASK GENMASK(26, 16)
+
+#define WQ_GUC_ID_MASK GENMASK(15, 0)
+#define WQ_RING_TAIL_MASK GENMASK(28, 18)
#define GUC_STAGE_DESC_ATTR_ACTIVE BIT(0)
#define GUC_STAGE_DESC_ATTR_PENDING_DB BIT(1)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index a412593342db..71ae5eb69849 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -400,6 +400,29 @@ __get_process_desc(struct intel_context *ce)
LRC_STATE_OFFSET) / sizeof(u32)));
}
+static u32 *get_wq_pointer(struct guc_process_desc *desc,
+ struct intel_context *ce,
+ u32 wqi_size)
+{
+ /*
+ * Check for space in work queue. Caching a value of head pointer in
+ * intel_context structure in order reduce the number accesses to shared
+ * GPU memory which may be across a PCIe bus.
+ */
+#define AVAILABLE_SPACE \
+ CIRC_SPACE(ce->parallel.guc.wqi_tail, ce->parallel.guc.wqi_head, WQ_SIZE)
+ if (wqi_size > AVAILABLE_SPACE) {
+ ce->parallel.guc.wqi_head = READ_ONCE(desc->head);
+
+ if (wqi_size > AVAILABLE_SPACE)
+ return NULL;
+ }
+#undef AVAILABLE_SPACE
+
+ return ((u32 *)__get_process_desc(ce)) +
+ ((WQ_OFFSET + ce->parallel.guc.wqi_tail) / sizeof(u32));
+}
+
static struct guc_lrc_desc *__get_lrc_desc(struct intel_guc *guc, u32 index)
{
struct guc_lrc_desc *base = guc->lrc_desc_pool_vaddr;
@@ -559,10 +582,10 @@ int intel_guc_wait_for_idle(struct intel_guc *guc, long timeout)
static int guc_lrc_desc_pin(struct intel_context *ce, bool loop);
-static int guc_add_request(struct intel_guc *guc, struct i915_request *rq)
+static int __guc_add_request(struct intel_guc *guc, struct i915_request *rq)
{
int err = 0;
- struct intel_context *ce = rq->context;
+ struct intel_context *ce = request_to_scheduling_context(rq);
u32 action[3];
int len = 0;
u32 g2h_len_dw = 0;
@@ -583,26 +606,17 @@ static int guc_add_request(struct intel_guc *guc, struct i915_request *rq)
GEM_BUG_ON(!atomic_read(&ce->guc_id.ref));
GEM_BUG_ON(context_guc_id_invalid(ce));
- /*
- * Corner case where the GuC firmware was blown away and reloaded while
- * this context was pinned.
- */
- if (unlikely(!lrc_desc_registered(guc, ce->guc_id.id))) {
- err = guc_lrc_desc_pin(ce, false);
- if (unlikely(err))
- return err;
- }
-
spin_lock(&ce->guc_state.lock);
/*
* The request / context will be run on the hardware when scheduling
- * gets enabled in the unblock.
+ * gets enabled in the unblock. For multi-lrc we still submit the
+ * context to move the LRC tails.
*/
- if (unlikely(context_blocked(ce)))
+ if (unlikely(context_blocked(ce) && !intel_context_is_parent(ce)))
goto out;
- enabled = context_enabled(ce);
+ enabled = context_enabled(ce) || context_blocked(ce);
if (!enabled) {
action[len++] = INTEL_GUC_ACTION_SCHED_CONTEXT_MODE_SET;
@@ -621,6 +635,18 @@ static int guc_add_request(struct intel_guc *guc, struct i915_request *rq)
trace_intel_context_sched_enable(ce);
atomic_inc(&guc->outstanding_submission_g2h);
set_context_enabled(ce);
+
+ /*
+ * Without multi-lrc KMD does the submission step (moving the
+ * lrc tail) so enabling scheduling is sufficient to submit the
+ * context. This isn't the case in multi-lrc submission as the
+ * GuC needs to move the tails, hence the need for another H2G
+ * to submit a multi-lrc context after enabling scheduling.
+ */
+ if (intel_context_is_parent(ce)) {
+ action[0] = INTEL_GUC_ACTION_SCHED_CONTEXT;
+ err = intel_guc_send_nb(guc, action, len - 1, 0);
+ }
} else if (!enabled) {
clr_context_pending_enable(ce);
intel_context_put(ce);
@@ -633,6 +659,18 @@ static int guc_add_request(struct intel_guc *guc, struct i915_request *rq)
return err;
}
+static int guc_add_request(struct intel_guc *guc, struct i915_request *rq)
+{
+ int ret = __guc_add_request(guc, rq);
+
+ if (unlikely(ret == -EBUSY)) {
+ guc->stalled_request = rq;
+ guc->submission_stall_reason = STALL_ADD_REQUEST;
+ }
+
+ return ret;
+}
+
static inline void guc_set_lrc_tail(struct i915_request *rq)
{
rq->context->lrc_reg_state[CTX_RING_TAIL] =
@@ -644,6 +682,136 @@ static inline int rq_prio(const struct i915_request *rq)
return rq->sched.attr.priority;
}
+static bool is_multi_lrc_rq(struct i915_request *rq)
+{
+ return intel_context_is_child(rq->context) ||
+ intel_context_is_parent(rq->context);
+}
+
+static bool can_merge_rq(struct i915_request *rq,
+ struct i915_request *last)
+{
+ return request_to_scheduling_context(rq) ==
+ request_to_scheduling_context(last);
+}
+
+static u32 wq_space_until_wrap(struct intel_context *ce)
+{
+ return (WQ_SIZE - ce->parallel.guc.wqi_tail);
+}
+
+static void write_wqi(struct guc_process_desc *desc,
+ struct intel_context *ce,
+ u32 wqi_size)
+{
+ BUILD_BUG_ON(!is_power_of_2(WQ_SIZE));
+
+ /*
+ * Ensure WQI are visible before updating tail
+ */
+ intel_guc_write_barrier(ce_to_guc(ce));
+
+ ce->parallel.guc.wqi_tail = (ce->parallel.guc.wqi_tail + wqi_size) &
+ (WQ_SIZE - 1);
+ WRITE_ONCE(desc->tail, ce->parallel.guc.wqi_tail);
+}
+
+static int guc_wq_noop_append(struct intel_context *ce)
+{
+ struct guc_process_desc *desc = __get_process_desc(ce);
+ u32 *wqi = get_wq_pointer(desc, ce, wq_space_until_wrap(ce));
+ u32 len_dw = wq_space_until_wrap(ce) / sizeof(u32) - 1;
+
+ if (!wqi)
+ return -EBUSY;
+
+ GEM_BUG_ON(!FIELD_FIT(WQ_LEN_MASK, len_dw));
+
+ *wqi = FIELD_PREP(WQ_TYPE_MASK, WQ_TYPE_NOOP) |
+ FIELD_PREP(WQ_LEN_MASK, len_dw);
+ ce->parallel.guc.wqi_tail = 0;
+
+ return 0;
+}
+
+static int __guc_wq_item_append(struct i915_request *rq)
+{
+ struct intel_context *ce = request_to_scheduling_context(rq);
+ struct intel_context *child;
+ struct guc_process_desc *desc = __get_process_desc(ce);
+ unsigned int wqi_size = (ce->parallel.number_children + 4) *
+ sizeof(u32);
+ u32 *wqi;
+ u32 len_dw = (wqi_size / sizeof(u32)) - 1;
+ int ret;
+
+ /* Ensure context is in correct state updating work queue */
+ GEM_BUG_ON(!atomic_read(&ce->guc_id.ref));
+ GEM_BUG_ON(context_guc_id_invalid(ce));
+ GEM_BUG_ON(context_wait_for_deregister_to_register(ce));
+ GEM_BUG_ON(!lrc_desc_registered(ce_to_guc(ce), ce->guc_id.id));
+
+ /* Insert NOOP if this work queue item will wrap the tail pointer. */
+ if (wqi_size > wq_space_until_wrap(ce)) {
+ ret = guc_wq_noop_append(ce);
+ if (ret)
+ return ret;
+ }
+
+ wqi = get_wq_pointer(desc, ce, wqi_size);
+ if (!wqi)
+ return -EBUSY;
+
+ GEM_BUG_ON(!FIELD_FIT(WQ_LEN_MASK, len_dw));
+
+ *wqi++ = FIELD_PREP(WQ_TYPE_MASK, WQ_TYPE_MULTI_LRC) |
+ FIELD_PREP(WQ_LEN_MASK, len_dw);
+ *wqi++ = ce->lrc.lrca;
+ *wqi++ = FIELD_PREP(WQ_GUC_ID_MASK, ce->guc_id.id) |
+ FIELD_PREP(WQ_RING_TAIL_MASK, ce->ring->tail / sizeof(u64));
+ *wqi++ = 0; /* fence_id */
+ for_each_child(ce, child)
+ *wqi++ = child->ring->tail / sizeof(u64);
+
+ write_wqi(desc, ce, wqi_size);
+
+ return 0;
+}
+
+static int guc_wq_item_append(struct intel_guc *guc,
+ struct i915_request *rq)
+{
+ struct intel_context *ce = request_to_scheduling_context(rq);
+ int ret = 0;
+
+ if (likely(!intel_context_is_banned(ce))) {
+ ret = __guc_wq_item_append(rq);
+
+ if (unlikely(ret == -EBUSY)) {
+ guc->stalled_request = rq;
+ guc->submission_stall_reason = STALL_MOVE_LRC_TAIL;
+ }
+ }
+
+ return ret;
+}
+
+static bool multi_lrc_submit(struct i915_request *rq)
+{
+ struct intel_context *ce = request_to_scheduling_context(rq);
+
+ intel_ring_set_tail(rq->ring, rq->tail);
+
+ /*
+ * We expect the front end (execbuf IOCTL) to set this flag on the last
+ * request generated from a multi-BB submission. This indicates to the
+ * backend (GuC interface) that we should submit this context thus
+ * submitting all the requests generated in parallel.
+ */
+ return test_bit(I915_FENCE_FLAG_SUBMIT_PARALLEL, &rq->fence.flags) ||
+ intel_context_is_banned(ce);
+}
+
static int guc_dequeue_one_context(struct intel_guc *guc)
{
struct i915_sched_engine * const sched_engine = guc->sched_engine;
@@ -657,7 +825,17 @@ static int guc_dequeue_one_context(struct intel_guc *guc)
if (guc->stalled_request) {
submit = true;
last = guc->stalled_request;
- goto resubmit;
+
+ switch (guc->submission_stall_reason) {
+ case STALL_REGISTER_CONTEXT:
+ goto register_context;
+ case STALL_MOVE_LRC_TAIL:
+ goto move_lrc_tail;
+ case STALL_ADD_REQUEST:
+ goto add_request;
+ default:
+ MISSING_CASE(guc->submission_stall_reason);
+ }
}
while ((rb = rb_first_cached(&sched_engine->queue))) {
@@ -665,8 +843,8 @@ static int guc_dequeue_one_context(struct intel_guc *guc)
struct i915_request *rq, *rn;
priolist_for_each_request_consume(rq, rn, p) {
- if (last && rq->context != last->context)
- goto done;
+ if (last && !can_merge_rq(rq, last))
+ goto register_context;
list_del_init(&rq->sched.link);
@@ -674,33 +852,84 @@ static int guc_dequeue_one_context(struct intel_guc *guc)
trace_i915_request_in(rq, 0);
last = rq;
- submit = true;
+
+ if (is_multi_lrc_rq(rq)) {
+ /*
+ * We need to coalesce all multi-lrc requests in
+ * a relationship into a single H2G. We are
+ * guaranteed that all of these requests will be
+ * submitted sequentially.
+ */
+ if (multi_lrc_submit(rq)) {
+ submit = true;
+ goto register_context;
+ }
+ } else {
+ submit = true;
+ }
}
rb_erase_cached(&p->node, &sched_engine->queue);
i915_priolist_free(p);
}
-done:
+
+register_context:
if (submit) {
- guc_set_lrc_tail(last);
-resubmit:
+ struct intel_context *ce = request_to_scheduling_context(last);
+
+ if (unlikely(!lrc_desc_registered(guc, ce->guc_id.id) &&
+ !intel_context_is_banned(ce))) {
+ ret = guc_lrc_desc_pin(ce, false);
+ if (unlikely(ret == -EPIPE)) {
+ goto deadlk;
+ } else if (ret == -EBUSY) {
+ guc->stalled_request = last;
+ guc->submission_stall_reason =
+ STALL_REGISTER_CONTEXT;
+ goto schedule_tasklet;
+ } else if (ret != 0) {
+ GEM_WARN_ON(ret); /* Unexpected */
+ goto deadlk;
+ }
+ }
+
+move_lrc_tail:
+ if (is_multi_lrc_rq(last)) {
+ ret = guc_wq_item_append(guc, last);
+ if (ret == -EBUSY) {
+ goto schedule_tasklet;
+ } else if (ret != 0) {
+ GEM_WARN_ON(ret); /* Unexpected */
+ goto deadlk;
+ }
+ } else {
+ guc_set_lrc_tail(last);
+ }
+
+add_request:
ret = guc_add_request(guc, last);
- if (unlikely(ret == -EPIPE))
+ if (unlikely(ret == -EPIPE)) {
+ goto deadlk;
+ } else if (ret == -EBUSY) {
+ goto schedule_tasklet;
+ } else if (ret != 0) {
+ GEM_WARN_ON(ret); /* Unexpected */
goto deadlk;
- else if (ret == -EBUSY) {
- tasklet_schedule(&sched_engine->tasklet);
- guc->stalled_request = last;
- return false;
}
}
guc->stalled_request = NULL;
+ guc->submission_stall_reason = STALL_NONE;
return submit;
deadlk:
sched_engine->tasklet.callback = NULL;
tasklet_disable_nosync(&sched_engine->tasklet);
return false;
+
+schedule_tasklet:
+ tasklet_schedule(&sched_engine->tasklet);
+ return false;
}
static void guc_submission_tasklet(struct tasklet_struct *t)
@@ -1250,16 +1479,22 @@ static inline void queue_request(struct i915_sched_engine *sched_engine,
static int guc_bypass_tasklet_submit(struct intel_guc *guc,
struct i915_request *rq)
{
- int ret;
+ int ret = 0;
__i915_request_submit(rq);
trace_i915_request_in(rq, 0);
- guc_set_lrc_tail(rq);
- ret = guc_add_request(guc, rq);
- if (ret == -EBUSY)
- guc->stalled_request = rq;
+ if (is_multi_lrc_rq(rq)) {
+ if (multi_lrc_submit(rq)) {
+ ret = guc_wq_item_append(guc, rq);
+ if (!ret)
+ ret = guc_add_request(guc, rq);
+ }
+ } else {
+ guc_set_lrc_tail(rq);
+ ret = guc_add_request(guc, rq);
+ }
if (unlikely(ret == -EPIPE))
disable_submission(guc);
@@ -1267,6 +1502,16 @@ static int guc_bypass_tasklet_submit(struct intel_guc *guc,
return ret;
}
+static bool need_tasklet(struct intel_guc *guc, struct i915_request *rq)
+{
+ struct i915_sched_engine *sched_engine = rq->engine->sched_engine;
+ struct intel_context *ce = request_to_scheduling_context(rq);
+
+ return submission_disabled(guc) || guc->stalled_request ||
+ !i915_sched_engine_is_empty(sched_engine) ||
+ !lrc_desc_registered(guc, ce->guc_id.id);
+}
+
static void guc_submit_request(struct i915_request *rq)
{
struct i915_sched_engine *sched_engine = rq->engine->sched_engine;
@@ -1276,8 +1521,7 @@ static void guc_submit_request(struct i915_request *rq)
/* Will be called from irq-context when using foreign fences. */
spin_lock_irqsave(&sched_engine->lock, flags);
- if (submission_disabled(guc) || guc->stalled_request ||
- !i915_sched_engine_is_empty(sched_engine))
+ if (need_tasklet(guc, rq))
queue_request(sched_engine, rq, rq_prio(rq));
else if (guc_bypass_tasklet_submit(guc, rq) == -EBUSY)
tasklet_hi_schedule(&sched_engine->tasklet);
@@ -2259,9 +2503,10 @@ static inline bool new_guc_prio_higher(u8 old_guc_prio, u8 new_guc_prio)
static void add_to_context(struct i915_request *rq)
{
- struct intel_context *ce = rq->context;
+ struct intel_context *ce = request_to_scheduling_context(rq);
u8 new_guc_prio = map_i915_prio_to_guc_prio(rq_prio(rq));
+ GEM_BUG_ON(intel_context_is_child(ce));
GEM_BUG_ON(rq->guc_prio == GUC_PRIO_FINI);
spin_lock(&ce->guc_state.lock);
@@ -2294,7 +2539,9 @@ static void guc_prio_fini(struct i915_request *rq, struct intel_context *ce)
static void remove_from_context(struct i915_request *rq)
{
- struct intel_context *ce = rq->context;
+ struct intel_context *ce = request_to_scheduling_context(rq);
+
+ GEM_BUG_ON(intel_context_is_child(ce));
spin_lock_irq(&ce->guc_state.lock);
@@ -2713,7 +2960,7 @@ static void guc_init_breadcrumbs(struct intel_engine_cs *engine)
static void guc_bump_inflight_request_prio(struct i915_request *rq,
int prio)
{
- struct intel_context *ce = rq->context;
+ struct intel_context *ce = request_to_scheduling_context(rq);
u8 new_guc_prio = map_i915_prio_to_guc_prio(prio);
/* Short circuit function */
@@ -2736,7 +2983,7 @@ static void guc_bump_inflight_request_prio(struct i915_request *rq,
static void guc_retire_inflight_request_prio(struct i915_request *rq)
{
- struct intel_context *ce = rq->context;
+ struct intel_context *ce = request_to_scheduling_context(rq);
spin_lock(&ce->guc_state.lock);
guc_prio_fini(rq, ce);
diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
index 7bd9ed20623e..8950785e55d6 100644
--- a/drivers/gpu/drm/i915/i915_request.h
+++ b/drivers/gpu/drm/i915/i915_request.h
@@ -139,6 +139,14 @@ enum {
* the GPU. Here we track such boost requests on a per-request basis.
*/
I915_FENCE_FLAG_BOOST,
+
+ /*
+ * I915_FENCE_FLAG_SUBMIT_PARALLEL - request with a context in a
+ * parent-child relationship (parallel submission, multi-lrc) should
+ * trigger a submission to the GuC rather than just moving the context
+ * tail.
+ */
+ I915_FENCE_FLAG_SUBMIT_PARALLEL,
};
/**
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 12/25] drm/i915/guc: Implement multi-lrc submission
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Implement multi-lrc submission via a single workqueue entry and single
H2G. The workqueue entry contains an updated tail value for each
request, of all the contexts in the multi-lrc submission, and updates
these values simultaneously. As such, the tasklet and bypass path have
been updated to coalesce requests into a single submission.
v2:
(John Harrison)
- s/wqe/wqi
- Use FIELD_PREP macros
- Add GEM_BUG_ONs ensures length fits within field
- Add comment / white space to intel_guc_write_barrier
(Kernel test robot)
- Make need_tasklet a static function
v3:
(Docs)
- A comment for submission_stall_reason
v4:
(Kernel test robot)
- Initialize return value in bypass tasklt submit function
(John Harrison)
- Add comment near work queue defs
- Add BUILD_BUG_ON to ensure WQ_SIZE is a power of 2
- Update write_barrier comment to talk about work queue
v5:
(John Harrison)
- Fix typo in work queue comment
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/uc/intel_guc.c | 29 ++
drivers/gpu/drm/i915/gt/uc/intel_guc.h | 11 +
drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c | 24 +-
drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 30 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 323 +++++++++++++++---
drivers/gpu/drm/i915/i915_request.h | 8 +
6 files changed, 350 insertions(+), 75 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.c b/drivers/gpu/drm/i915/gt/uc/intel_guc.c
index 8f8182bf7c11..6e228343e8cb 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.c
@@ -756,3 +756,32 @@ void intel_guc_load_status(struct intel_guc *guc, struct drm_printer *p)
}
}
}
+
+void intel_guc_write_barrier(struct intel_guc *guc)
+{
+ struct intel_gt *gt = guc_to_gt(guc);
+
+ if (i915_gem_object_is_lmem(guc->ct.vma->obj)) {
+ /*
+ * Ensure intel_uncore_write_fw can be used rather than
+ * intel_uncore_write.
+ */
+ GEM_BUG_ON(guc->send_regs.fw_domains);
+
+ /*
+ * This register is used by the i915 and GuC for MMIO based
+ * communication. Once we are in this code CTBs are the only
+ * method the i915 uses to communicate with the GuC so it is
+ * safe to write to this register (a value of 0 is NOP for MMIO
+ * communication). If we ever start mixing CTBs and MMIOs a new
+ * register will have to be chosen. This function is also used
+ * to enforce ordering of a work queue item write and an update
+ * to the process descriptor. When a work queue is being used,
+ * CTBs are also the only mechanism of communication.
+ */
+ intel_uncore_write_fw(gt->uncore, GEN11_SOFT_SCRATCH(0), 0);
+ } else {
+ /* wmb() sufficient for a barrier if in smem */
+ wmb();
+ }
+}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc.h b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
index 4ca197f400ba..31cf9fb48c7e 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc.h
@@ -46,6 +46,15 @@ struct intel_guc {
* submitted until the stalled request is processed.
*/
struct i915_request *stalled_request;
+ /**
+ * @submission_stall_reason: reason why submission is stalled
+ */
+ enum {
+ STALL_NONE,
+ STALL_REGISTER_CONTEXT,
+ STALL_MOVE_LRC_TAIL,
+ STALL_ADD_REQUEST,
+ } submission_stall_reason;
/* intel_guc_recv interrupt related state */
/** @irq_lock: protects GuC irq state */
@@ -367,4 +376,6 @@ void intel_guc_submission_cancel_requests(struct intel_guc *guc);
void intel_guc_load_status(struct intel_guc *guc, struct drm_printer *p);
+void intel_guc_write_barrier(struct intel_guc *guc);
+
#endif
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
index 0a3504bc0b61..a0cc34be7b56 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_ct.c
@@ -383,28 +383,6 @@ static u32 ct_get_next_fence(struct intel_guc_ct *ct)
return ++ct->requests.last_fence;
}
-static void write_barrier(struct intel_guc_ct *ct)
-{
- struct intel_guc *guc = ct_to_guc(ct);
- struct intel_gt *gt = guc_to_gt(guc);
-
- if (i915_gem_object_is_lmem(guc->ct.vma->obj)) {
- GEM_BUG_ON(guc->send_regs.fw_domains);
- /*
- * This register is used by the i915 and GuC for MMIO based
- * communication. Once we are in this code CTBs are the only
- * method the i915 uses to communicate with the GuC so it is
- * safe to write to this register (a value of 0 is NOP for MMIO
- * communication). If we ever start mixing CTBs and MMIOs a new
- * register will have to be chosen.
- */
- intel_uncore_write_fw(gt->uncore, GEN11_SOFT_SCRATCH(0), 0);
- } else {
- /* wmb() sufficient for a barrier if in smem */
- wmb();
- }
-}
-
static int ct_write(struct intel_guc_ct *ct,
const u32 *action,
u32 len /* in dwords */,
@@ -474,7 +452,7 @@ static int ct_write(struct intel_guc_ct *ct,
* make sure H2G buffer update and LRC tail update (if this triggering a
* submission) are visible before updating the descriptor tail
*/
- write_barrier(ct);
+ intel_guc_write_barrier(ct_to_guc(ct));
/* update local copies */
ctb->tail = tail;
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
index 0eeb2a9feeed..18da67cfcd92 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
@@ -52,25 +52,27 @@
#define GUC_DOORBELL_INVALID 256
-/* Work queue item header definitions */
+/*
+ * Work queue item header definitions
+ *
+ * Work queue is circular buffer used to submit complex (multi-lrc) submissions
+ * to the GuC. A work queue item is an entry in the circular buffer.
+ */
#define WQ_STATUS_ACTIVE 1
#define WQ_STATUS_SUSPENDED 2
#define WQ_STATUS_CMD_ERROR 3
#define WQ_STATUS_ENGINE_ID_NOT_USED 4
#define WQ_STATUS_SUSPENDED_FROM_RESET 5
-#define WQ_TYPE_SHIFT 0
-#define WQ_TYPE_BATCH_BUF (0x1 << WQ_TYPE_SHIFT)
-#define WQ_TYPE_PSEUDO (0x2 << WQ_TYPE_SHIFT)
-#define WQ_TYPE_INORDER (0x3 << WQ_TYPE_SHIFT)
-#define WQ_TYPE_NOOP (0x4 << WQ_TYPE_SHIFT)
-#define WQ_TARGET_SHIFT 10
-#define WQ_LEN_SHIFT 16
-#define WQ_NO_WCFLUSH_WAIT (1 << 27)
-#define WQ_PRESENT_WORKLOAD (1 << 28)
-
-#define WQ_RING_TAIL_SHIFT 20
-#define WQ_RING_TAIL_MAX 0x7FF /* 2^11 QWords */
-#define WQ_RING_TAIL_MASK (WQ_RING_TAIL_MAX << WQ_RING_TAIL_SHIFT)
+#define WQ_TYPE_BATCH_BUF 0x1
+#define WQ_TYPE_PSEUDO 0x2
+#define WQ_TYPE_INORDER 0x3
+#define WQ_TYPE_NOOP 0x4
+#define WQ_TYPE_MULTI_LRC 0x5
+#define WQ_TYPE_MASK GENMASK(7, 0)
+#define WQ_LEN_MASK GENMASK(26, 16)
+
+#define WQ_GUC_ID_MASK GENMASK(15, 0)
+#define WQ_RING_TAIL_MASK GENMASK(28, 18)
#define GUC_STAGE_DESC_ATTR_ACTIVE BIT(0)
#define GUC_STAGE_DESC_ATTR_PENDING_DB BIT(1)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index a412593342db..71ae5eb69849 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -400,6 +400,29 @@ __get_process_desc(struct intel_context *ce)
LRC_STATE_OFFSET) / sizeof(u32)));
}
+static u32 *get_wq_pointer(struct guc_process_desc *desc,
+ struct intel_context *ce,
+ u32 wqi_size)
+{
+ /*
+ * Check for space in work queue. Caching a value of head pointer in
+ * intel_context structure in order reduce the number accesses to shared
+ * GPU memory which may be across a PCIe bus.
+ */
+#define AVAILABLE_SPACE \
+ CIRC_SPACE(ce->parallel.guc.wqi_tail, ce->parallel.guc.wqi_head, WQ_SIZE)
+ if (wqi_size > AVAILABLE_SPACE) {
+ ce->parallel.guc.wqi_head = READ_ONCE(desc->head);
+
+ if (wqi_size > AVAILABLE_SPACE)
+ return NULL;
+ }
+#undef AVAILABLE_SPACE
+
+ return ((u32 *)__get_process_desc(ce)) +
+ ((WQ_OFFSET + ce->parallel.guc.wqi_tail) / sizeof(u32));
+}
+
static struct guc_lrc_desc *__get_lrc_desc(struct intel_guc *guc, u32 index)
{
struct guc_lrc_desc *base = guc->lrc_desc_pool_vaddr;
@@ -559,10 +582,10 @@ int intel_guc_wait_for_idle(struct intel_guc *guc, long timeout)
static int guc_lrc_desc_pin(struct intel_context *ce, bool loop);
-static int guc_add_request(struct intel_guc *guc, struct i915_request *rq)
+static int __guc_add_request(struct intel_guc *guc, struct i915_request *rq)
{
int err = 0;
- struct intel_context *ce = rq->context;
+ struct intel_context *ce = request_to_scheduling_context(rq);
u32 action[3];
int len = 0;
u32 g2h_len_dw = 0;
@@ -583,26 +606,17 @@ static int guc_add_request(struct intel_guc *guc, struct i915_request *rq)
GEM_BUG_ON(!atomic_read(&ce->guc_id.ref));
GEM_BUG_ON(context_guc_id_invalid(ce));
- /*
- * Corner case where the GuC firmware was blown away and reloaded while
- * this context was pinned.
- */
- if (unlikely(!lrc_desc_registered(guc, ce->guc_id.id))) {
- err = guc_lrc_desc_pin(ce, false);
- if (unlikely(err))
- return err;
- }
-
spin_lock(&ce->guc_state.lock);
/*
* The request / context will be run on the hardware when scheduling
- * gets enabled in the unblock.
+ * gets enabled in the unblock. For multi-lrc we still submit the
+ * context to move the LRC tails.
*/
- if (unlikely(context_blocked(ce)))
+ if (unlikely(context_blocked(ce) && !intel_context_is_parent(ce)))
goto out;
- enabled = context_enabled(ce);
+ enabled = context_enabled(ce) || context_blocked(ce);
if (!enabled) {
action[len++] = INTEL_GUC_ACTION_SCHED_CONTEXT_MODE_SET;
@@ -621,6 +635,18 @@ static int guc_add_request(struct intel_guc *guc, struct i915_request *rq)
trace_intel_context_sched_enable(ce);
atomic_inc(&guc->outstanding_submission_g2h);
set_context_enabled(ce);
+
+ /*
+ * Without multi-lrc KMD does the submission step (moving the
+ * lrc tail) so enabling scheduling is sufficient to submit the
+ * context. This isn't the case in multi-lrc submission as the
+ * GuC needs to move the tails, hence the need for another H2G
+ * to submit a multi-lrc context after enabling scheduling.
+ */
+ if (intel_context_is_parent(ce)) {
+ action[0] = INTEL_GUC_ACTION_SCHED_CONTEXT;
+ err = intel_guc_send_nb(guc, action, len - 1, 0);
+ }
} else if (!enabled) {
clr_context_pending_enable(ce);
intel_context_put(ce);
@@ -633,6 +659,18 @@ static int guc_add_request(struct intel_guc *guc, struct i915_request *rq)
return err;
}
+static int guc_add_request(struct intel_guc *guc, struct i915_request *rq)
+{
+ int ret = __guc_add_request(guc, rq);
+
+ if (unlikely(ret == -EBUSY)) {
+ guc->stalled_request = rq;
+ guc->submission_stall_reason = STALL_ADD_REQUEST;
+ }
+
+ return ret;
+}
+
static inline void guc_set_lrc_tail(struct i915_request *rq)
{
rq->context->lrc_reg_state[CTX_RING_TAIL] =
@@ -644,6 +682,136 @@ static inline int rq_prio(const struct i915_request *rq)
return rq->sched.attr.priority;
}
+static bool is_multi_lrc_rq(struct i915_request *rq)
+{
+ return intel_context_is_child(rq->context) ||
+ intel_context_is_parent(rq->context);
+}
+
+static bool can_merge_rq(struct i915_request *rq,
+ struct i915_request *last)
+{
+ return request_to_scheduling_context(rq) ==
+ request_to_scheduling_context(last);
+}
+
+static u32 wq_space_until_wrap(struct intel_context *ce)
+{
+ return (WQ_SIZE - ce->parallel.guc.wqi_tail);
+}
+
+static void write_wqi(struct guc_process_desc *desc,
+ struct intel_context *ce,
+ u32 wqi_size)
+{
+ BUILD_BUG_ON(!is_power_of_2(WQ_SIZE));
+
+ /*
+ * Ensure WQI are visible before updating tail
+ */
+ intel_guc_write_barrier(ce_to_guc(ce));
+
+ ce->parallel.guc.wqi_tail = (ce->parallel.guc.wqi_tail + wqi_size) &
+ (WQ_SIZE - 1);
+ WRITE_ONCE(desc->tail, ce->parallel.guc.wqi_tail);
+}
+
+static int guc_wq_noop_append(struct intel_context *ce)
+{
+ struct guc_process_desc *desc = __get_process_desc(ce);
+ u32 *wqi = get_wq_pointer(desc, ce, wq_space_until_wrap(ce));
+ u32 len_dw = wq_space_until_wrap(ce) / sizeof(u32) - 1;
+
+ if (!wqi)
+ return -EBUSY;
+
+ GEM_BUG_ON(!FIELD_FIT(WQ_LEN_MASK, len_dw));
+
+ *wqi = FIELD_PREP(WQ_TYPE_MASK, WQ_TYPE_NOOP) |
+ FIELD_PREP(WQ_LEN_MASK, len_dw);
+ ce->parallel.guc.wqi_tail = 0;
+
+ return 0;
+}
+
+static int __guc_wq_item_append(struct i915_request *rq)
+{
+ struct intel_context *ce = request_to_scheduling_context(rq);
+ struct intel_context *child;
+ struct guc_process_desc *desc = __get_process_desc(ce);
+ unsigned int wqi_size = (ce->parallel.number_children + 4) *
+ sizeof(u32);
+ u32 *wqi;
+ u32 len_dw = (wqi_size / sizeof(u32)) - 1;
+ int ret;
+
+ /* Ensure context is in correct state updating work queue */
+ GEM_BUG_ON(!atomic_read(&ce->guc_id.ref));
+ GEM_BUG_ON(context_guc_id_invalid(ce));
+ GEM_BUG_ON(context_wait_for_deregister_to_register(ce));
+ GEM_BUG_ON(!lrc_desc_registered(ce_to_guc(ce), ce->guc_id.id));
+
+ /* Insert NOOP if this work queue item will wrap the tail pointer. */
+ if (wqi_size > wq_space_until_wrap(ce)) {
+ ret = guc_wq_noop_append(ce);
+ if (ret)
+ return ret;
+ }
+
+ wqi = get_wq_pointer(desc, ce, wqi_size);
+ if (!wqi)
+ return -EBUSY;
+
+ GEM_BUG_ON(!FIELD_FIT(WQ_LEN_MASK, len_dw));
+
+ *wqi++ = FIELD_PREP(WQ_TYPE_MASK, WQ_TYPE_MULTI_LRC) |
+ FIELD_PREP(WQ_LEN_MASK, len_dw);
+ *wqi++ = ce->lrc.lrca;
+ *wqi++ = FIELD_PREP(WQ_GUC_ID_MASK, ce->guc_id.id) |
+ FIELD_PREP(WQ_RING_TAIL_MASK, ce->ring->tail / sizeof(u64));
+ *wqi++ = 0; /* fence_id */
+ for_each_child(ce, child)
+ *wqi++ = child->ring->tail / sizeof(u64);
+
+ write_wqi(desc, ce, wqi_size);
+
+ return 0;
+}
+
+static int guc_wq_item_append(struct intel_guc *guc,
+ struct i915_request *rq)
+{
+ struct intel_context *ce = request_to_scheduling_context(rq);
+ int ret = 0;
+
+ if (likely(!intel_context_is_banned(ce))) {
+ ret = __guc_wq_item_append(rq);
+
+ if (unlikely(ret == -EBUSY)) {
+ guc->stalled_request = rq;
+ guc->submission_stall_reason = STALL_MOVE_LRC_TAIL;
+ }
+ }
+
+ return ret;
+}
+
+static bool multi_lrc_submit(struct i915_request *rq)
+{
+ struct intel_context *ce = request_to_scheduling_context(rq);
+
+ intel_ring_set_tail(rq->ring, rq->tail);
+
+ /*
+ * We expect the front end (execbuf IOCTL) to set this flag on the last
+ * request generated from a multi-BB submission. This indicates to the
+ * backend (GuC interface) that we should submit this context thus
+ * submitting all the requests generated in parallel.
+ */
+ return test_bit(I915_FENCE_FLAG_SUBMIT_PARALLEL, &rq->fence.flags) ||
+ intel_context_is_banned(ce);
+}
+
static int guc_dequeue_one_context(struct intel_guc *guc)
{
struct i915_sched_engine * const sched_engine = guc->sched_engine;
@@ -657,7 +825,17 @@ static int guc_dequeue_one_context(struct intel_guc *guc)
if (guc->stalled_request) {
submit = true;
last = guc->stalled_request;
- goto resubmit;
+
+ switch (guc->submission_stall_reason) {
+ case STALL_REGISTER_CONTEXT:
+ goto register_context;
+ case STALL_MOVE_LRC_TAIL:
+ goto move_lrc_tail;
+ case STALL_ADD_REQUEST:
+ goto add_request;
+ default:
+ MISSING_CASE(guc->submission_stall_reason);
+ }
}
while ((rb = rb_first_cached(&sched_engine->queue))) {
@@ -665,8 +843,8 @@ static int guc_dequeue_one_context(struct intel_guc *guc)
struct i915_request *rq, *rn;
priolist_for_each_request_consume(rq, rn, p) {
- if (last && rq->context != last->context)
- goto done;
+ if (last && !can_merge_rq(rq, last))
+ goto register_context;
list_del_init(&rq->sched.link);
@@ -674,33 +852,84 @@ static int guc_dequeue_one_context(struct intel_guc *guc)
trace_i915_request_in(rq, 0);
last = rq;
- submit = true;
+
+ if (is_multi_lrc_rq(rq)) {
+ /*
+ * We need to coalesce all multi-lrc requests in
+ * a relationship into a single H2G. We are
+ * guaranteed that all of these requests will be
+ * submitted sequentially.
+ */
+ if (multi_lrc_submit(rq)) {
+ submit = true;
+ goto register_context;
+ }
+ } else {
+ submit = true;
+ }
}
rb_erase_cached(&p->node, &sched_engine->queue);
i915_priolist_free(p);
}
-done:
+
+register_context:
if (submit) {
- guc_set_lrc_tail(last);
-resubmit:
+ struct intel_context *ce = request_to_scheduling_context(last);
+
+ if (unlikely(!lrc_desc_registered(guc, ce->guc_id.id) &&
+ !intel_context_is_banned(ce))) {
+ ret = guc_lrc_desc_pin(ce, false);
+ if (unlikely(ret == -EPIPE)) {
+ goto deadlk;
+ } else if (ret == -EBUSY) {
+ guc->stalled_request = last;
+ guc->submission_stall_reason =
+ STALL_REGISTER_CONTEXT;
+ goto schedule_tasklet;
+ } else if (ret != 0) {
+ GEM_WARN_ON(ret); /* Unexpected */
+ goto deadlk;
+ }
+ }
+
+move_lrc_tail:
+ if (is_multi_lrc_rq(last)) {
+ ret = guc_wq_item_append(guc, last);
+ if (ret == -EBUSY) {
+ goto schedule_tasklet;
+ } else if (ret != 0) {
+ GEM_WARN_ON(ret); /* Unexpected */
+ goto deadlk;
+ }
+ } else {
+ guc_set_lrc_tail(last);
+ }
+
+add_request:
ret = guc_add_request(guc, last);
- if (unlikely(ret == -EPIPE))
+ if (unlikely(ret == -EPIPE)) {
+ goto deadlk;
+ } else if (ret == -EBUSY) {
+ goto schedule_tasklet;
+ } else if (ret != 0) {
+ GEM_WARN_ON(ret); /* Unexpected */
goto deadlk;
- else if (ret == -EBUSY) {
- tasklet_schedule(&sched_engine->tasklet);
- guc->stalled_request = last;
- return false;
}
}
guc->stalled_request = NULL;
+ guc->submission_stall_reason = STALL_NONE;
return submit;
deadlk:
sched_engine->tasklet.callback = NULL;
tasklet_disable_nosync(&sched_engine->tasklet);
return false;
+
+schedule_tasklet:
+ tasklet_schedule(&sched_engine->tasklet);
+ return false;
}
static void guc_submission_tasklet(struct tasklet_struct *t)
@@ -1250,16 +1479,22 @@ static inline void queue_request(struct i915_sched_engine *sched_engine,
static int guc_bypass_tasklet_submit(struct intel_guc *guc,
struct i915_request *rq)
{
- int ret;
+ int ret = 0;
__i915_request_submit(rq);
trace_i915_request_in(rq, 0);
- guc_set_lrc_tail(rq);
- ret = guc_add_request(guc, rq);
- if (ret == -EBUSY)
- guc->stalled_request = rq;
+ if (is_multi_lrc_rq(rq)) {
+ if (multi_lrc_submit(rq)) {
+ ret = guc_wq_item_append(guc, rq);
+ if (!ret)
+ ret = guc_add_request(guc, rq);
+ }
+ } else {
+ guc_set_lrc_tail(rq);
+ ret = guc_add_request(guc, rq);
+ }
if (unlikely(ret == -EPIPE))
disable_submission(guc);
@@ -1267,6 +1502,16 @@ static int guc_bypass_tasklet_submit(struct intel_guc *guc,
return ret;
}
+static bool need_tasklet(struct intel_guc *guc, struct i915_request *rq)
+{
+ struct i915_sched_engine *sched_engine = rq->engine->sched_engine;
+ struct intel_context *ce = request_to_scheduling_context(rq);
+
+ return submission_disabled(guc) || guc->stalled_request ||
+ !i915_sched_engine_is_empty(sched_engine) ||
+ !lrc_desc_registered(guc, ce->guc_id.id);
+}
+
static void guc_submit_request(struct i915_request *rq)
{
struct i915_sched_engine *sched_engine = rq->engine->sched_engine;
@@ -1276,8 +1521,7 @@ static void guc_submit_request(struct i915_request *rq)
/* Will be called from irq-context when using foreign fences. */
spin_lock_irqsave(&sched_engine->lock, flags);
- if (submission_disabled(guc) || guc->stalled_request ||
- !i915_sched_engine_is_empty(sched_engine))
+ if (need_tasklet(guc, rq))
queue_request(sched_engine, rq, rq_prio(rq));
else if (guc_bypass_tasklet_submit(guc, rq) == -EBUSY)
tasklet_hi_schedule(&sched_engine->tasklet);
@@ -2259,9 +2503,10 @@ static inline bool new_guc_prio_higher(u8 old_guc_prio, u8 new_guc_prio)
static void add_to_context(struct i915_request *rq)
{
- struct intel_context *ce = rq->context;
+ struct intel_context *ce = request_to_scheduling_context(rq);
u8 new_guc_prio = map_i915_prio_to_guc_prio(rq_prio(rq));
+ GEM_BUG_ON(intel_context_is_child(ce));
GEM_BUG_ON(rq->guc_prio == GUC_PRIO_FINI);
spin_lock(&ce->guc_state.lock);
@@ -2294,7 +2539,9 @@ static void guc_prio_fini(struct i915_request *rq, struct intel_context *ce)
static void remove_from_context(struct i915_request *rq)
{
- struct intel_context *ce = rq->context;
+ struct intel_context *ce = request_to_scheduling_context(rq);
+
+ GEM_BUG_ON(intel_context_is_child(ce));
spin_lock_irq(&ce->guc_state.lock);
@@ -2713,7 +2960,7 @@ static void guc_init_breadcrumbs(struct intel_engine_cs *engine)
static void guc_bump_inflight_request_prio(struct i915_request *rq,
int prio)
{
- struct intel_context *ce = rq->context;
+ struct intel_context *ce = request_to_scheduling_context(rq);
u8 new_guc_prio = map_i915_prio_to_guc_prio(prio);
/* Short circuit function */
@@ -2736,7 +2983,7 @@ static void guc_bump_inflight_request_prio(struct i915_request *rq,
static void guc_retire_inflight_request_prio(struct i915_request *rq)
{
- struct intel_context *ce = rq->context;
+ struct intel_context *ce = request_to_scheduling_context(rq);
spin_lock(&ce->guc_state.lock);
guc_prio_fini(rq, ce);
diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
index 7bd9ed20623e..8950785e55d6 100644
--- a/drivers/gpu/drm/i915/i915_request.h
+++ b/drivers/gpu/drm/i915/i915_request.h
@@ -139,6 +139,14 @@ enum {
* the GPU. Here we track such boost requests on a per-request basis.
*/
I915_FENCE_FLAG_BOOST,
+
+ /*
+ * I915_FENCE_FLAG_SUBMIT_PARALLEL - request with a context in a
+ * parent-child relationship (parallel submission, multi-lrc) should
+ * trigger a submission to the GuC rather than just moving the context
+ * tail.
+ */
+ I915_FENCE_FLAG_SUBMIT_PARALLEL,
};
/**
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 13/25] drm/i915/guc: Insert submit fences between requests in parent-child relationship
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
The GuC must receive requests in the order submitted for contexts in a
parent-child relationship to function correctly. To ensure this, insert
a submit fence between the current request and last request submitted
for requests / contexts in a parent child relationship. This is
conceptually similar to a single timeline.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Cc: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.h | 5 +
drivers/gpu/drm/i915/gt/intel_context_types.h | 6 +
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 5 +-
drivers/gpu/drm/i915/i915_request.c | 120 ++++++++++++++----
4 files changed, 108 insertions(+), 28 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
index 9f0995150a7a..edf12caaade3 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.h
+++ b/drivers/gpu/drm/i915/gt/intel_context.h
@@ -77,6 +77,11 @@ intel_context_to_parent(struct intel_context *ce)
}
}
+static inline bool intel_context_is_parallel(struct intel_context *ce)
+{
+ return intel_context_is_child(ce) || intel_context_is_parent(ce);
+}
+
void intel_context_bind_parent_child(struct intel_context *parent,
struct intel_context *child);
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 48decb5ee954..8309d1141d0a 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -237,6 +237,12 @@ struct intel_context {
};
/** @parent: pointer to parent if child */
struct intel_context *parent;
+ /**
+ * @last_rq: last request submitted on a parallel context, used
+ * to insert submit fences between requests in the parallel
+ * context
+ */
+ struct i915_request *last_rq;
/** @number_children: number of children if parent */
u8 number_children;
/** @guc: GuC specific members for parallel submission */
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 71ae5eb69849..ebb64fb50396 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -684,8 +684,7 @@ static inline int rq_prio(const struct i915_request *rq)
static bool is_multi_lrc_rq(struct i915_request *rq)
{
- return intel_context_is_child(rq->context) ||
- intel_context_is_parent(rq->context);
+ return intel_context_is_parallel(rq->context);
}
static bool can_merge_rq(struct i915_request *rq,
@@ -2873,6 +2872,8 @@ static void guc_parent_context_unpin(struct intel_context *ce)
GEM_BUG_ON(!intel_context_is_parent(ce));
GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
+ if (ce->parallel.last_rq)
+ i915_request_put(ce->parallel.last_rq);
unpin_guc_id(guc, ce);
lrc_unpin(ce);
}
diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
index c0d27072c28d..8bdf9f2f9b90 100644
--- a/drivers/gpu/drm/i915/i915_request.c
+++ b/drivers/gpu/drm/i915/i915_request.c
@@ -1525,36 +1525,62 @@ i915_request_await_object(struct i915_request *to,
return ret;
}
+static inline bool is_parallel_rq(struct i915_request *rq)
+{
+ return intel_context_is_parallel(rq->context);
+}
+
+static inline struct intel_context *request_to_parent(struct i915_request *rq)
+{
+ return intel_context_to_parent(rq->context);
+}
+
static struct i915_request *
-__i915_request_add_to_timeline(struct i915_request *rq)
+__i915_request_ensure_parallel_ordering(struct i915_request *rq,
+ struct intel_timeline *timeline)
{
- struct intel_timeline *timeline = i915_request_timeline(rq);
struct i915_request *prev;
- /*
- * Dependency tracking and request ordering along the timeline
- * is special cased so that we can eliminate redundant ordering
- * operations while building the request (we know that the timeline
- * itself is ordered, and here we guarantee it).
- *
- * As we know we will need to emit tracking along the timeline,
- * we embed the hooks into our request struct -- at the cost of
- * having to have specialised no-allocation interfaces (which will
- * be beneficial elsewhere).
- *
- * A second benefit to open-coding i915_request_await_request is
- * that we can apply a slight variant of the rules specialised
- * for timelines that jump between engines (such as virtual engines).
- * If we consider the case of virtual engine, we must emit a dma-fence
- * to prevent scheduling of the second request until the first is
- * complete (to maximise our greedy late load balancing) and this
- * precludes optimising to use semaphores serialisation of a single
- * timeline across engines.
- */
+ GEM_BUG_ON(!is_parallel_rq(rq));
+
+ prev = request_to_parent(rq)->parallel.last_rq;
+ if (prev) {
+ if (!__i915_request_is_complete(prev)) {
+ i915_sw_fence_await_sw_fence(&rq->submit,
+ &prev->submit,
+ &rq->submitq);
+
+ if (rq->engine->sched_engine->schedule)
+ __i915_sched_node_add_dependency(&rq->sched,
+ &prev->sched,
+ &rq->dep,
+ 0);
+ }
+ i915_request_put(prev);
+ }
+
+ request_to_parent(rq)->parallel.last_rq = i915_request_get(rq);
+
+ return to_request(__i915_active_fence_set(&timeline->last_request,
+ &rq->fence));
+}
+
+static struct i915_request *
+__i915_request_ensure_ordering(struct i915_request *rq,
+ struct intel_timeline *timeline)
+{
+ struct i915_request *prev;
+
+ GEM_BUG_ON(is_parallel_rq(rq));
+
prev = to_request(__i915_active_fence_set(&timeline->last_request,
&rq->fence));
+
if (prev && !__i915_request_is_complete(prev)) {
bool uses_guc = intel_engine_uses_guc(rq->engine);
+ bool pow2 = is_power_of_2(READ_ONCE(prev->engine)->mask |
+ rq->engine->mask);
+ bool same_context = prev->context == rq->context;
/*
* The requests are supposed to be kept in order. However,
@@ -1562,13 +1588,11 @@ __i915_request_add_to_timeline(struct i915_request *rq)
* is used as a barrier for external modification to this
* context.
*/
- GEM_BUG_ON(prev->context == rq->context &&
+ GEM_BUG_ON(same_context &&
i915_seqno_passed(prev->fence.seqno,
rq->fence.seqno));
- if ((!uses_guc &&
- is_power_of_2(READ_ONCE(prev->engine)->mask | rq->engine->mask)) ||
- (uses_guc && prev->context == rq->context))
+ if ((same_context && uses_guc) || (!uses_guc && pow2))
i915_sw_fence_await_sw_fence(&rq->submit,
&prev->submit,
&rq->submitq);
@@ -1583,6 +1607,50 @@ __i915_request_add_to_timeline(struct i915_request *rq)
0);
}
+ return prev;
+}
+
+static struct i915_request *
+__i915_request_add_to_timeline(struct i915_request *rq)
+{
+ struct intel_timeline *timeline = i915_request_timeline(rq);
+ struct i915_request *prev;
+
+ /*
+ * Dependency tracking and request ordering along the timeline
+ * is special cased so that we can eliminate redundant ordering
+ * operations while building the request (we know that the timeline
+ * itself is ordered, and here we guarantee it).
+ *
+ * As we know we will need to emit tracking along the timeline,
+ * we embed the hooks into our request struct -- at the cost of
+ * having to have specialised no-allocation interfaces (which will
+ * be beneficial elsewhere).
+ *
+ * A second benefit to open-coding i915_request_await_request is
+ * that we can apply a slight variant of the rules specialised
+ * for timelines that jump between engines (such as virtual engines).
+ * If we consider the case of virtual engine, we must emit a dma-fence
+ * to prevent scheduling of the second request until the first is
+ * complete (to maximise our greedy late load balancing) and this
+ * precludes optimising to use semaphores serialisation of a single
+ * timeline across engines.
+ *
+ * We do not order parallel submission requests on the timeline as each
+ * parallel submission context has its own timeline and the ordering
+ * rules for parallel requests are that they must be submitted in the
+ * order received from the execbuf IOCTL. So rather than using the
+ * timeline we store a pointer to last request submitted in the
+ * relationship in the gem context and insert a submission fence
+ * between that request and request passed into this function or
+ * alternatively we use completion fence if gem context has a single
+ * timeline and this is the first submission of an execbuf IOCTL.
+ */
+ if (likely(!is_parallel_rq(rq)))
+ prev = __i915_request_ensure_ordering(rq, timeline);
+ else
+ prev = __i915_request_ensure_parallel_ordering(rq, timeline);
+
/*
* Make sure that no request gazumped us - if it was allocated after
* our i915_request_alloc() and called __i915_request_add() before
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 13/25] drm/i915/guc: Insert submit fences between requests in parent-child relationship
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
The GuC must receive requests in the order submitted for contexts in a
parent-child relationship to function correctly. To ensure this, insert
a submit fence between the current request and last request submitted
for requests / contexts in a parent child relationship. This is
conceptually similar to a single timeline.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Cc: John Harrison <John.C.Harrison@Intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.h | 5 +
drivers/gpu/drm/i915/gt/intel_context_types.h | 6 +
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 5 +-
drivers/gpu/drm/i915/i915_request.c | 120 ++++++++++++++----
4 files changed, 108 insertions(+), 28 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
index 9f0995150a7a..edf12caaade3 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.h
+++ b/drivers/gpu/drm/i915/gt/intel_context.h
@@ -77,6 +77,11 @@ intel_context_to_parent(struct intel_context *ce)
}
}
+static inline bool intel_context_is_parallel(struct intel_context *ce)
+{
+ return intel_context_is_child(ce) || intel_context_is_parent(ce);
+}
+
void intel_context_bind_parent_child(struct intel_context *parent,
struct intel_context *child);
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 48decb5ee954..8309d1141d0a 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -237,6 +237,12 @@ struct intel_context {
};
/** @parent: pointer to parent if child */
struct intel_context *parent;
+ /**
+ * @last_rq: last request submitted on a parallel context, used
+ * to insert submit fences between requests in the parallel
+ * context
+ */
+ struct i915_request *last_rq;
/** @number_children: number of children if parent */
u8 number_children;
/** @guc: GuC specific members for parallel submission */
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 71ae5eb69849..ebb64fb50396 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -684,8 +684,7 @@ static inline int rq_prio(const struct i915_request *rq)
static bool is_multi_lrc_rq(struct i915_request *rq)
{
- return intel_context_is_child(rq->context) ||
- intel_context_is_parent(rq->context);
+ return intel_context_is_parallel(rq->context);
}
static bool can_merge_rq(struct i915_request *rq,
@@ -2873,6 +2872,8 @@ static void guc_parent_context_unpin(struct intel_context *ce)
GEM_BUG_ON(!intel_context_is_parent(ce));
GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
+ if (ce->parallel.last_rq)
+ i915_request_put(ce->parallel.last_rq);
unpin_guc_id(guc, ce);
lrc_unpin(ce);
}
diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
index c0d27072c28d..8bdf9f2f9b90 100644
--- a/drivers/gpu/drm/i915/i915_request.c
+++ b/drivers/gpu/drm/i915/i915_request.c
@@ -1525,36 +1525,62 @@ i915_request_await_object(struct i915_request *to,
return ret;
}
+static inline bool is_parallel_rq(struct i915_request *rq)
+{
+ return intel_context_is_parallel(rq->context);
+}
+
+static inline struct intel_context *request_to_parent(struct i915_request *rq)
+{
+ return intel_context_to_parent(rq->context);
+}
+
static struct i915_request *
-__i915_request_add_to_timeline(struct i915_request *rq)
+__i915_request_ensure_parallel_ordering(struct i915_request *rq,
+ struct intel_timeline *timeline)
{
- struct intel_timeline *timeline = i915_request_timeline(rq);
struct i915_request *prev;
- /*
- * Dependency tracking and request ordering along the timeline
- * is special cased so that we can eliminate redundant ordering
- * operations while building the request (we know that the timeline
- * itself is ordered, and here we guarantee it).
- *
- * As we know we will need to emit tracking along the timeline,
- * we embed the hooks into our request struct -- at the cost of
- * having to have specialised no-allocation interfaces (which will
- * be beneficial elsewhere).
- *
- * A second benefit to open-coding i915_request_await_request is
- * that we can apply a slight variant of the rules specialised
- * for timelines that jump between engines (such as virtual engines).
- * If we consider the case of virtual engine, we must emit a dma-fence
- * to prevent scheduling of the second request until the first is
- * complete (to maximise our greedy late load balancing) and this
- * precludes optimising to use semaphores serialisation of a single
- * timeline across engines.
- */
+ GEM_BUG_ON(!is_parallel_rq(rq));
+
+ prev = request_to_parent(rq)->parallel.last_rq;
+ if (prev) {
+ if (!__i915_request_is_complete(prev)) {
+ i915_sw_fence_await_sw_fence(&rq->submit,
+ &prev->submit,
+ &rq->submitq);
+
+ if (rq->engine->sched_engine->schedule)
+ __i915_sched_node_add_dependency(&rq->sched,
+ &prev->sched,
+ &rq->dep,
+ 0);
+ }
+ i915_request_put(prev);
+ }
+
+ request_to_parent(rq)->parallel.last_rq = i915_request_get(rq);
+
+ return to_request(__i915_active_fence_set(&timeline->last_request,
+ &rq->fence));
+}
+
+static struct i915_request *
+__i915_request_ensure_ordering(struct i915_request *rq,
+ struct intel_timeline *timeline)
+{
+ struct i915_request *prev;
+
+ GEM_BUG_ON(is_parallel_rq(rq));
+
prev = to_request(__i915_active_fence_set(&timeline->last_request,
&rq->fence));
+
if (prev && !__i915_request_is_complete(prev)) {
bool uses_guc = intel_engine_uses_guc(rq->engine);
+ bool pow2 = is_power_of_2(READ_ONCE(prev->engine)->mask |
+ rq->engine->mask);
+ bool same_context = prev->context == rq->context;
/*
* The requests are supposed to be kept in order. However,
@@ -1562,13 +1588,11 @@ __i915_request_add_to_timeline(struct i915_request *rq)
* is used as a barrier for external modification to this
* context.
*/
- GEM_BUG_ON(prev->context == rq->context &&
+ GEM_BUG_ON(same_context &&
i915_seqno_passed(prev->fence.seqno,
rq->fence.seqno));
- if ((!uses_guc &&
- is_power_of_2(READ_ONCE(prev->engine)->mask | rq->engine->mask)) ||
- (uses_guc && prev->context == rq->context))
+ if ((same_context && uses_guc) || (!uses_guc && pow2))
i915_sw_fence_await_sw_fence(&rq->submit,
&prev->submit,
&rq->submitq);
@@ -1583,6 +1607,50 @@ __i915_request_add_to_timeline(struct i915_request *rq)
0);
}
+ return prev;
+}
+
+static struct i915_request *
+__i915_request_add_to_timeline(struct i915_request *rq)
+{
+ struct intel_timeline *timeline = i915_request_timeline(rq);
+ struct i915_request *prev;
+
+ /*
+ * Dependency tracking and request ordering along the timeline
+ * is special cased so that we can eliminate redundant ordering
+ * operations while building the request (we know that the timeline
+ * itself is ordered, and here we guarantee it).
+ *
+ * As we know we will need to emit tracking along the timeline,
+ * we embed the hooks into our request struct -- at the cost of
+ * having to have specialised no-allocation interfaces (which will
+ * be beneficial elsewhere).
+ *
+ * A second benefit to open-coding i915_request_await_request is
+ * that we can apply a slight variant of the rules specialised
+ * for timelines that jump between engines (such as virtual engines).
+ * If we consider the case of virtual engine, we must emit a dma-fence
+ * to prevent scheduling of the second request until the first is
+ * complete (to maximise our greedy late load balancing) and this
+ * precludes optimising to use semaphores serialisation of a single
+ * timeline across engines.
+ *
+ * We do not order parallel submission requests on the timeline as each
+ * parallel submission context has its own timeline and the ordering
+ * rules for parallel requests are that they must be submitted in the
+ * order received from the execbuf IOCTL. So rather than using the
+ * timeline we store a pointer to last request submitted in the
+ * relationship in the gem context and insert a submission fence
+ * between that request and request passed into this function or
+ * alternatively we use completion fence if gem context has a single
+ * timeline and this is the first submission of an execbuf IOCTL.
+ */
+ if (likely(!is_parallel_rq(rq)))
+ prev = __i915_request_ensure_ordering(rq, timeline);
+ else
+ prev = __i915_request_ensure_parallel_ordering(rq, timeline);
+
/*
* Make sure that no request gazumped us - if it was allocated after
* our i915_request_alloc() and called __i915_request_add() before
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 14/25] drm/i915/guc: Implement multi-lrc reset
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Update context and full GPU reset to work with multi-lrc. The idea is
parent context tracks all the active requests inflight for itself and
its children. The parent context owns the reset replaying / canceling
requests as needed.
v2:
(John Harrison)
- Simply loop in find active request
- Add comments to find ative request / reset loop
v3:
(John Harrison)
- s/its'/its/g
- Fix comment when searching for active request
- Reorder if state in __guc_reset_context
v4:
(Kernel test robot)
- Delete ununsed is_multi_lrc function
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.c | 15 ++++-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 64 +++++++++++++------
2 files changed, 58 insertions(+), 21 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index 79f321c6c008..6aab60584ee5 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -529,20 +529,29 @@ struct i915_request *intel_context_create_request(struct intel_context *ce)
struct i915_request *intel_context_find_active_request(struct intel_context *ce)
{
+ struct intel_context *parent = intel_context_to_parent(ce);
struct i915_request *rq, *active = NULL;
unsigned long flags;
GEM_BUG_ON(!intel_engine_uses_guc(ce->engine));
- spin_lock_irqsave(&ce->guc_state.lock, flags);
- list_for_each_entry_reverse(rq, &ce->guc_state.requests,
+ /*
+ * We search the parent list to find an active request on the submitted
+ * context. The parent list contains the requests for all the contexts
+ * in the relationship so we have to do a compare of each request's
+ * context.
+ */
+ spin_lock_irqsave(&parent->guc_state.lock, flags);
+ list_for_each_entry_reverse(rq, &parent->guc_state.requests,
sched.link) {
+ if (rq->context != ce)
+ continue;
if (i915_request_completed(rq))
break;
active = rq;
}
- spin_unlock_irqrestore(&ce->guc_state.lock, flags);
+ spin_unlock_irqrestore(&parent->guc_state.lock, flags);
return active;
}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index ebb64fb50396..112b5e6fe39d 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -1217,10 +1217,15 @@ __unwind_incomplete_requests(struct intel_context *ce)
static void __guc_reset_context(struct intel_context *ce, bool stalled)
{
+ bool local_stalled;
struct i915_request *rq;
unsigned long flags;
u32 head;
+ int i, number_children = ce->parallel.number_children;
bool skip = false;
+ struct intel_context *parent = ce;
+
+ GEM_BUG_ON(intel_context_is_child(ce));
intel_context_get(ce);
@@ -1246,25 +1251,38 @@ static void __guc_reset_context(struct intel_context *ce, bool stalled)
if (unlikely(skip))
goto out_put;
- rq = intel_context_find_active_request(ce);
- if (!rq) {
- head = ce->ring->tail;
- stalled = false;
- goto out_replay;
- }
+ /*
+ * For each context in the relationship find the hanging request
+ * resetting each context / request as needed
+ */
+ for (i = 0; i < number_children + 1; ++i) {
+ if (!intel_context_is_pinned(ce))
+ goto next_context;
+
+ local_stalled = false;
+ rq = intel_context_find_active_request(ce);
+ if (!rq) {
+ head = ce->ring->tail;
+ goto out_replay;
+ }
- if (!i915_request_started(rq))
- stalled = false;
+ if (i915_request_started(rq))
+ local_stalled = true;
- GEM_BUG_ON(i915_active_is_idle(&ce->active));
- head = intel_ring_wrap(ce->ring, rq->head);
- __i915_request_reset(rq, stalled);
+ GEM_BUG_ON(i915_active_is_idle(&ce->active));
+ head = intel_ring_wrap(ce->ring, rq->head);
+ __i915_request_reset(rq, local_stalled && stalled);
out_replay:
- guc_reset_state(ce, head, stalled);
- __unwind_incomplete_requests(ce);
+ guc_reset_state(ce, head, local_stalled && stalled);
+next_context:
+ if (i != number_children)
+ ce = list_next_entry(ce, parallel.child_link);
+ }
+
+ __unwind_incomplete_requests(parent);
out_put:
- intel_context_put(ce);
+ intel_context_put(parent);
}
void intel_guc_submission_reset(struct intel_guc *guc, bool stalled)
@@ -1285,7 +1303,8 @@ void intel_guc_submission_reset(struct intel_guc *guc, bool stalled)
xa_unlock(&guc->context_lookup);
- if (intel_context_is_pinned(ce))
+ if (intel_context_is_pinned(ce) &&
+ !intel_context_is_child(ce))
__guc_reset_context(ce, stalled);
intel_context_put(ce);
@@ -1377,7 +1396,8 @@ void intel_guc_submission_cancel_requests(struct intel_guc *guc)
xa_unlock(&guc->context_lookup);
- if (intel_context_is_pinned(ce))
+ if (intel_context_is_pinned(ce) &&
+ !intel_context_is_child(ce))
guc_cancel_context_requests(ce);
intel_context_put(ce);
@@ -2070,6 +2090,8 @@ static struct i915_sw_fence *guc_context_block(struct intel_context *ce)
u16 guc_id;
bool enabled;
+ GEM_BUG_ON(intel_context_is_child(ce));
+
spin_lock_irqsave(&ce->guc_state.lock, flags);
incr_context_blocked(ce);
@@ -2124,6 +2146,7 @@ static void guc_context_unblock(struct intel_context *ce)
bool enable;
GEM_BUG_ON(context_enabled(ce));
+ GEM_BUG_ON(intel_context_is_child(ce));
spin_lock_irqsave(&ce->guc_state.lock, flags);
@@ -2150,11 +2173,14 @@ static void guc_context_unblock(struct intel_context *ce)
static void guc_context_cancel_request(struct intel_context *ce,
struct i915_request *rq)
{
+ struct intel_context *block_context =
+ request_to_scheduling_context(rq);
+
if (i915_sw_fence_signaled(&rq->submit)) {
struct i915_sw_fence *fence;
intel_context_get(ce);
- fence = guc_context_block(ce);
+ fence = guc_context_block(block_context);
i915_sw_fence_wait(fence);
if (!i915_request_completed(rq)) {
__i915_request_skip(rq);
@@ -2168,7 +2194,7 @@ static void guc_context_cancel_request(struct intel_context *ce,
*/
flush_work(&ce_to_guc(ce)->ct.requests.worker);
- guc_context_unblock(ce);
+ guc_context_unblock(block_context);
intel_context_put(ce);
}
}
@@ -2194,6 +2220,8 @@ static void guc_context_ban(struct intel_context *ce, struct i915_request *rq)
intel_wakeref_t wakeref;
unsigned long flags;
+ GEM_BUG_ON(intel_context_is_child(ce));
+
guc_flush_submissions(guc);
spin_lock_irqsave(&ce->guc_state.lock, flags);
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 14/25] drm/i915/guc: Implement multi-lrc reset
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Update context and full GPU reset to work with multi-lrc. The idea is
parent context tracks all the active requests inflight for itself and
its children. The parent context owns the reset replaying / canceling
requests as needed.
v2:
(John Harrison)
- Simply loop in find active request
- Add comments to find ative request / reset loop
v3:
(John Harrison)
- s/its'/its/g
- Fix comment when searching for active request
- Reorder if state in __guc_reset_context
v4:
(Kernel test robot)
- Delete ununsed is_multi_lrc function
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.c | 15 ++++-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 64 +++++++++++++------
2 files changed, 58 insertions(+), 21 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index 79f321c6c008..6aab60584ee5 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -529,20 +529,29 @@ struct i915_request *intel_context_create_request(struct intel_context *ce)
struct i915_request *intel_context_find_active_request(struct intel_context *ce)
{
+ struct intel_context *parent = intel_context_to_parent(ce);
struct i915_request *rq, *active = NULL;
unsigned long flags;
GEM_BUG_ON(!intel_engine_uses_guc(ce->engine));
- spin_lock_irqsave(&ce->guc_state.lock, flags);
- list_for_each_entry_reverse(rq, &ce->guc_state.requests,
+ /*
+ * We search the parent list to find an active request on the submitted
+ * context. The parent list contains the requests for all the contexts
+ * in the relationship so we have to do a compare of each request's
+ * context.
+ */
+ spin_lock_irqsave(&parent->guc_state.lock, flags);
+ list_for_each_entry_reverse(rq, &parent->guc_state.requests,
sched.link) {
+ if (rq->context != ce)
+ continue;
if (i915_request_completed(rq))
break;
active = rq;
}
- spin_unlock_irqrestore(&ce->guc_state.lock, flags);
+ spin_unlock_irqrestore(&parent->guc_state.lock, flags);
return active;
}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index ebb64fb50396..112b5e6fe39d 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -1217,10 +1217,15 @@ __unwind_incomplete_requests(struct intel_context *ce)
static void __guc_reset_context(struct intel_context *ce, bool stalled)
{
+ bool local_stalled;
struct i915_request *rq;
unsigned long flags;
u32 head;
+ int i, number_children = ce->parallel.number_children;
bool skip = false;
+ struct intel_context *parent = ce;
+
+ GEM_BUG_ON(intel_context_is_child(ce));
intel_context_get(ce);
@@ -1246,25 +1251,38 @@ static void __guc_reset_context(struct intel_context *ce, bool stalled)
if (unlikely(skip))
goto out_put;
- rq = intel_context_find_active_request(ce);
- if (!rq) {
- head = ce->ring->tail;
- stalled = false;
- goto out_replay;
- }
+ /*
+ * For each context in the relationship find the hanging request
+ * resetting each context / request as needed
+ */
+ for (i = 0; i < number_children + 1; ++i) {
+ if (!intel_context_is_pinned(ce))
+ goto next_context;
+
+ local_stalled = false;
+ rq = intel_context_find_active_request(ce);
+ if (!rq) {
+ head = ce->ring->tail;
+ goto out_replay;
+ }
- if (!i915_request_started(rq))
- stalled = false;
+ if (i915_request_started(rq))
+ local_stalled = true;
- GEM_BUG_ON(i915_active_is_idle(&ce->active));
- head = intel_ring_wrap(ce->ring, rq->head);
- __i915_request_reset(rq, stalled);
+ GEM_BUG_ON(i915_active_is_idle(&ce->active));
+ head = intel_ring_wrap(ce->ring, rq->head);
+ __i915_request_reset(rq, local_stalled && stalled);
out_replay:
- guc_reset_state(ce, head, stalled);
- __unwind_incomplete_requests(ce);
+ guc_reset_state(ce, head, local_stalled && stalled);
+next_context:
+ if (i != number_children)
+ ce = list_next_entry(ce, parallel.child_link);
+ }
+
+ __unwind_incomplete_requests(parent);
out_put:
- intel_context_put(ce);
+ intel_context_put(parent);
}
void intel_guc_submission_reset(struct intel_guc *guc, bool stalled)
@@ -1285,7 +1303,8 @@ void intel_guc_submission_reset(struct intel_guc *guc, bool stalled)
xa_unlock(&guc->context_lookup);
- if (intel_context_is_pinned(ce))
+ if (intel_context_is_pinned(ce) &&
+ !intel_context_is_child(ce))
__guc_reset_context(ce, stalled);
intel_context_put(ce);
@@ -1377,7 +1396,8 @@ void intel_guc_submission_cancel_requests(struct intel_guc *guc)
xa_unlock(&guc->context_lookup);
- if (intel_context_is_pinned(ce))
+ if (intel_context_is_pinned(ce) &&
+ !intel_context_is_child(ce))
guc_cancel_context_requests(ce);
intel_context_put(ce);
@@ -2070,6 +2090,8 @@ static struct i915_sw_fence *guc_context_block(struct intel_context *ce)
u16 guc_id;
bool enabled;
+ GEM_BUG_ON(intel_context_is_child(ce));
+
spin_lock_irqsave(&ce->guc_state.lock, flags);
incr_context_blocked(ce);
@@ -2124,6 +2146,7 @@ static void guc_context_unblock(struct intel_context *ce)
bool enable;
GEM_BUG_ON(context_enabled(ce));
+ GEM_BUG_ON(intel_context_is_child(ce));
spin_lock_irqsave(&ce->guc_state.lock, flags);
@@ -2150,11 +2173,14 @@ static void guc_context_unblock(struct intel_context *ce)
static void guc_context_cancel_request(struct intel_context *ce,
struct i915_request *rq)
{
+ struct intel_context *block_context =
+ request_to_scheduling_context(rq);
+
if (i915_sw_fence_signaled(&rq->submit)) {
struct i915_sw_fence *fence;
intel_context_get(ce);
- fence = guc_context_block(ce);
+ fence = guc_context_block(block_context);
i915_sw_fence_wait(fence);
if (!i915_request_completed(rq)) {
__i915_request_skip(rq);
@@ -2168,7 +2194,7 @@ static void guc_context_cancel_request(struct intel_context *ce,
*/
flush_work(&ce_to_guc(ce)->ct.requests.worker);
- guc_context_unblock(ce);
+ guc_context_unblock(block_context);
intel_context_put(ce);
}
}
@@ -2194,6 +2220,8 @@ static void guc_context_ban(struct intel_context *ce, struct i915_request *rq)
intel_wakeref_t wakeref;
unsigned long flags;
+ GEM_BUG_ON(intel_context_is_child(ce));
+
guc_flush_submissions(guc);
spin_lock_irqsave(&ce->guc_state.lock, flags);
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 15/25] drm/i915/guc: Update debugfs for GuC multi-lrc
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Display the workqueue status in debugfs for GuC contexts that are in
parent-child relationship.
v2:
(John Harrison)
- Output number children in debugfs
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 52 ++++++++++++++-----
1 file changed, 38 insertions(+), 14 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 112b5e6fe39d..938dc34e8d3a 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -3702,6 +3702,25 @@ static inline void guc_log_context_priority(struct drm_printer *p,
drm_printf(p, "\n");
}
+static inline void guc_log_context(struct drm_printer *p,
+ struct intel_context *ce)
+{
+ drm_printf(p, "GuC lrc descriptor %u:\n", ce->guc_id.id);
+ drm_printf(p, "\tHW Context Desc: 0x%08x\n", ce->lrc.lrca);
+ drm_printf(p, "\t\tLRC Head: Internal %u, Memory %u\n",
+ ce->ring->head,
+ ce->lrc_reg_state[CTX_RING_HEAD]);
+ drm_printf(p, "\t\tLRC Tail: Internal %u, Memory %u\n",
+ ce->ring->tail,
+ ce->lrc_reg_state[CTX_RING_TAIL]);
+ drm_printf(p, "\t\tContext Pin Count: %u\n",
+ atomic_read(&ce->pin_count));
+ drm_printf(p, "\t\tGuC ID Ref Count: %u\n",
+ atomic_read(&ce->guc_id.ref));
+ drm_printf(p, "\t\tSchedule State: 0x%x\n\n",
+ ce->guc_state.sched_state);
+}
+
void intel_guc_submission_print_context_info(struct intel_guc *guc,
struct drm_printer *p)
{
@@ -3711,22 +3730,27 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc,
xa_lock_irqsave(&guc->context_lookup, flags);
xa_for_each(&guc->context_lookup, index, ce) {
- drm_printf(p, "GuC lrc descriptor %u:\n", ce->guc_id.id);
- drm_printf(p, "\tHW Context Desc: 0x%08x\n", ce->lrc.lrca);
- drm_printf(p, "\t\tLRC Head: Internal %u, Memory %u\n",
- ce->ring->head,
- ce->lrc_reg_state[CTX_RING_HEAD]);
- drm_printf(p, "\t\tLRC Tail: Internal %u, Memory %u\n",
- ce->ring->tail,
- ce->lrc_reg_state[CTX_RING_TAIL]);
- drm_printf(p, "\t\tContext Pin Count: %u\n",
- atomic_read(&ce->pin_count));
- drm_printf(p, "\t\tGuC ID Ref Count: %u\n",
- atomic_read(&ce->guc_id.ref));
- drm_printf(p, "\t\tSchedule State: 0x%x\n\n",
- ce->guc_state.sched_state);
+ GEM_BUG_ON(intel_context_is_child(ce));
+ guc_log_context(p, ce);
guc_log_context_priority(p, ce);
+
+ if (intel_context_is_parent(ce)) {
+ struct guc_process_desc *desc = __get_process_desc(ce);
+ struct intel_context *child;
+
+ drm_printf(p, "\t\tNumber children: %u\n",
+ ce->parallel.number_children);
+ drm_printf(p, "\t\tWQI Head: %u\n",
+ READ_ONCE(desc->head));
+ drm_printf(p, "\t\tWQI Tail: %u\n",
+ READ_ONCE(desc->tail));
+ drm_printf(p, "\t\tWQI Status: %u\n\n",
+ READ_ONCE(desc->wq_status));
+
+ for_each_child(ce, child)
+ guc_log_context(p, child);
+ }
}
xa_unlock_irqrestore(&guc->context_lookup, flags);
}
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 15/25] drm/i915/guc: Update debugfs for GuC multi-lrc
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Display the workqueue status in debugfs for GuC contexts that are in
parent-child relationship.
v2:
(John Harrison)
- Output number children in debugfs
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 52 ++++++++++++++-----
1 file changed, 38 insertions(+), 14 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 112b5e6fe39d..938dc34e8d3a 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -3702,6 +3702,25 @@ static inline void guc_log_context_priority(struct drm_printer *p,
drm_printf(p, "\n");
}
+static inline void guc_log_context(struct drm_printer *p,
+ struct intel_context *ce)
+{
+ drm_printf(p, "GuC lrc descriptor %u:\n", ce->guc_id.id);
+ drm_printf(p, "\tHW Context Desc: 0x%08x\n", ce->lrc.lrca);
+ drm_printf(p, "\t\tLRC Head: Internal %u, Memory %u\n",
+ ce->ring->head,
+ ce->lrc_reg_state[CTX_RING_HEAD]);
+ drm_printf(p, "\t\tLRC Tail: Internal %u, Memory %u\n",
+ ce->ring->tail,
+ ce->lrc_reg_state[CTX_RING_TAIL]);
+ drm_printf(p, "\t\tContext Pin Count: %u\n",
+ atomic_read(&ce->pin_count));
+ drm_printf(p, "\t\tGuC ID Ref Count: %u\n",
+ atomic_read(&ce->guc_id.ref));
+ drm_printf(p, "\t\tSchedule State: 0x%x\n\n",
+ ce->guc_state.sched_state);
+}
+
void intel_guc_submission_print_context_info(struct intel_guc *guc,
struct drm_printer *p)
{
@@ -3711,22 +3730,27 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc,
xa_lock_irqsave(&guc->context_lookup, flags);
xa_for_each(&guc->context_lookup, index, ce) {
- drm_printf(p, "GuC lrc descriptor %u:\n", ce->guc_id.id);
- drm_printf(p, "\tHW Context Desc: 0x%08x\n", ce->lrc.lrca);
- drm_printf(p, "\t\tLRC Head: Internal %u, Memory %u\n",
- ce->ring->head,
- ce->lrc_reg_state[CTX_RING_HEAD]);
- drm_printf(p, "\t\tLRC Tail: Internal %u, Memory %u\n",
- ce->ring->tail,
- ce->lrc_reg_state[CTX_RING_TAIL]);
- drm_printf(p, "\t\tContext Pin Count: %u\n",
- atomic_read(&ce->pin_count));
- drm_printf(p, "\t\tGuC ID Ref Count: %u\n",
- atomic_read(&ce->guc_id.ref));
- drm_printf(p, "\t\tSchedule State: 0x%x\n\n",
- ce->guc_state.sched_state);
+ GEM_BUG_ON(intel_context_is_child(ce));
+ guc_log_context(p, ce);
guc_log_context_priority(p, ce);
+
+ if (intel_context_is_parent(ce)) {
+ struct guc_process_desc *desc = __get_process_desc(ce);
+ struct intel_context *child;
+
+ drm_printf(p, "\t\tNumber children: %u\n",
+ ce->parallel.number_children);
+ drm_printf(p, "\t\tWQI Head: %u\n",
+ READ_ONCE(desc->head));
+ drm_printf(p, "\t\tWQI Tail: %u\n",
+ READ_ONCE(desc->tail));
+ drm_printf(p, "\t\tWQI Status: %u\n\n",
+ READ_ONCE(desc->wq_status));
+
+ for_each_child(ce, child)
+ guc_log_context(p, child);
+ }
}
xa_unlock_irqrestore(&guc->context_lookup, flags);
}
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 16/25] drm/i915/guc: Connect UAPI to GuC multi-lrc interface
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Introduce 'set parallel submit' extension to connect UAPI to GuC
multi-lrc interface. Kernel doc in new uAPI should explain it all.
IGT: https://patchwork.freedesktop.org/patch/447008/?series=93071&rev=1
media UMD: https://github.com/intel/media-driver/pull/1252
v2:
(Daniel Vetter)
- Add IGT link and placeholder for media UMD link
v3:
(Kernel test robot)
- Fix warning in unpin engines call
(John Harrison)
- Reword a bunch of the kernel doc
v4:
(John Harrison)
- Add comment why perma-pin is done after setting gem context
- Update some comments / docs for proto contexts
v5:
(John Harrison)
- Rework perma-pin comment
- Add BUG_IN if context is pinned when setting gem context
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gem/i915_gem_context.c | 230 +++++++++++++++++-
.../gpu/drm/i915/gem/i915_gem_context_types.h | 16 +-
drivers/gpu/drm/i915/gt/intel_context_types.h | 9 +-
drivers/gpu/drm/i915/gt/intel_engine.h | 12 +-
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 6 +-
.../drm/i915/gt/intel_execlists_submission.c | 6 +-
drivers/gpu/drm/i915/gt/selftest_execlists.c | 12 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 114 ++++++++-
include/uapi/drm/i915_drm.h | 131 ++++++++++
9 files changed, 505 insertions(+), 31 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
index d225d3dd0b40..9a00f11fef46 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
@@ -556,9 +556,150 @@ set_proto_ctx_engines_bond(struct i915_user_extension __user *base, void *data)
return 0;
}
+static int
+set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
+ void *data)
+{
+ struct i915_context_engines_parallel_submit __user *ext =
+ container_of_user(base, typeof(*ext), base);
+ const struct set_proto_ctx_engines *set = data;
+ struct drm_i915_private *i915 = set->i915;
+ u64 flags;
+ int err = 0, n, i, j;
+ u16 slot, width, num_siblings;
+ struct intel_engine_cs **siblings = NULL;
+ intel_engine_mask_t prev_mask;
+
+ /* Disabling for now */
+ return -ENODEV;
+
+ /* FIXME: This is NIY for execlists */
+ if (!(intel_uc_uses_guc_submission(&i915->gt.uc)))
+ return -ENODEV;
+
+ if (get_user(slot, &ext->engine_index))
+ return -EFAULT;
+
+ if (get_user(width, &ext->width))
+ return -EFAULT;
+
+ if (get_user(num_siblings, &ext->num_siblings))
+ return -EFAULT;
+
+ if (slot >= set->num_engines) {
+ drm_dbg(&i915->drm, "Invalid placement value, %d >= %d\n",
+ slot, set->num_engines);
+ return -EINVAL;
+ }
+
+ if (set->engines[slot].type != I915_GEM_ENGINE_TYPE_INVALID) {
+ drm_dbg(&i915->drm,
+ "Invalid placement[%d], already occupied\n", slot);
+ return -EINVAL;
+ }
+
+ if (get_user(flags, &ext->flags))
+ return -EFAULT;
+
+ if (flags) {
+ drm_dbg(&i915->drm, "Unknown flags 0x%02llx", flags);
+ return -EINVAL;
+ }
+
+ for (n = 0; n < ARRAY_SIZE(ext->mbz64); n++) {
+ err = check_user_mbz(&ext->mbz64[n]);
+ if (err)
+ return err;
+ }
+
+ if (width < 2) {
+ drm_dbg(&i915->drm, "Width (%d) < 2\n", width);
+ return -EINVAL;
+ }
+
+ if (num_siblings < 1) {
+ drm_dbg(&i915->drm, "Number siblings (%d) < 1\n",
+ num_siblings);
+ return -EINVAL;
+ }
+
+ siblings = kmalloc_array(num_siblings * width,
+ sizeof(*siblings),
+ GFP_KERNEL);
+ if (!siblings)
+ return -ENOMEM;
+
+ /* Create contexts / engines */
+ for (i = 0; i < width; ++i) {
+ intel_engine_mask_t current_mask = 0;
+ struct i915_engine_class_instance prev_engine;
+
+ for (j = 0; j < num_siblings; ++j) {
+ struct i915_engine_class_instance ci;
+
+ n = i * num_siblings + j;
+ if (copy_from_user(&ci, &ext->engines[n], sizeof(ci))) {
+ err = -EFAULT;
+ goto out_err;
+ }
+
+ siblings[n] =
+ intel_engine_lookup_user(i915, ci.engine_class,
+ ci.engine_instance);
+ if (!siblings[n]) {
+ drm_dbg(&i915->drm,
+ "Invalid sibling[%d]: { class:%d, inst:%d }\n",
+ n, ci.engine_class, ci.engine_instance);
+ err = -EINVAL;
+ goto out_err;
+ }
+
+ if (n) {
+ if (prev_engine.engine_class !=
+ ci.engine_class) {
+ drm_dbg(&i915->drm,
+ "Mismatched class %d, %d\n",
+ prev_engine.engine_class,
+ ci.engine_class);
+ err = -EINVAL;
+ goto out_err;
+ }
+ }
+
+ prev_engine = ci;
+ current_mask |= siblings[n]->logical_mask;
+ }
+
+ if (i > 0) {
+ if (current_mask != prev_mask << 1) {
+ drm_dbg(&i915->drm,
+ "Non contiguous logical mask 0x%x, 0x%x\n",
+ prev_mask, current_mask);
+ err = -EINVAL;
+ goto out_err;
+ }
+ }
+ prev_mask = current_mask;
+ }
+
+ set->engines[slot].type = I915_GEM_ENGINE_TYPE_PARALLEL;
+ set->engines[slot].num_siblings = num_siblings;
+ set->engines[slot].width = width;
+ set->engines[slot].siblings = siblings;
+
+ return 0;
+
+out_err:
+ kfree(siblings);
+
+ return err;
+}
+
static const i915_user_extension_fn set_proto_ctx_engines_extensions[] = {
[I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE] = set_proto_ctx_engines_balance,
[I915_CONTEXT_ENGINES_EXT_BOND] = set_proto_ctx_engines_bond,
+ [I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT] =
+ set_proto_ctx_engines_parallel_submit,
};
static int set_proto_ctx_engines(struct drm_i915_file_private *fpriv,
@@ -794,6 +935,7 @@ static int intel_context_set_gem(struct intel_context *ce,
GEM_BUG_ON(rcu_access_pointer(ce->gem_context));
RCU_INIT_POINTER(ce->gem_context, ctx);
+ GEM_BUG_ON(intel_context_is_pinned(ce));
ce->ring_size = SZ_16K;
i915_vm_put(ce->vm);
@@ -818,6 +960,25 @@ static int intel_context_set_gem(struct intel_context *ce,
return ret;
}
+static void __unpin_engines(struct i915_gem_engines *e, unsigned int count)
+{
+ while (count--) {
+ struct intel_context *ce = e->engines[count], *child;
+
+ if (!ce || !test_bit(CONTEXT_PERMA_PIN, &ce->flags))
+ continue;
+
+ for_each_child(ce, child)
+ intel_context_unpin(child);
+ intel_context_unpin(ce);
+ }
+}
+
+static void unpin_engines(struct i915_gem_engines *e)
+{
+ __unpin_engines(e, e->num_engines);
+}
+
static void __free_engines(struct i915_gem_engines *e, unsigned int count)
{
while (count--) {
@@ -933,6 +1094,40 @@ static struct i915_gem_engines *default_engines(struct i915_gem_context *ctx,
return err;
}
+static int perma_pin_contexts(struct intel_context *ce)
+{
+ struct intel_context *child;
+ int i = 0, j = 0, ret;
+
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ ret = intel_context_pin(ce);
+ if (unlikely(ret))
+ return ret;
+
+ for_each_child(ce, child) {
+ ret = intel_context_pin(child);
+ if (unlikely(ret))
+ goto unwind;
+ ++i;
+ }
+
+ set_bit(CONTEXT_PERMA_PIN, &ce->flags);
+
+ return 0;
+
+unwind:
+ intel_context_unpin(ce);
+ for_each_child(ce, child) {
+ if (j++ < i)
+ intel_context_unpin(child);
+ else
+ break;
+ }
+
+ return ret;
+}
+
static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
unsigned int num_engines,
struct i915_gem_proto_engine *pe)
@@ -946,7 +1141,7 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
e->num_engines = num_engines;
for (n = 0; n < num_engines; n++) {
- struct intel_context *ce;
+ struct intel_context *ce, *child;
int ret;
switch (pe[n].type) {
@@ -956,7 +1151,13 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
case I915_GEM_ENGINE_TYPE_BALANCED:
ce = intel_engine_create_virtual(pe[n].siblings,
- pe[n].num_siblings);
+ pe[n].num_siblings, 0);
+ break;
+
+ case I915_GEM_ENGINE_TYPE_PARALLEL:
+ ce = intel_engine_create_parallel(pe[n].siblings,
+ pe[n].num_siblings,
+ pe[n].width);
break;
case I915_GEM_ENGINE_TYPE_INVALID:
@@ -977,6 +1178,30 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
err = ERR_PTR(ret);
goto free_engines;
}
+ for_each_child(ce, child) {
+ ret = intel_context_set_gem(child, ctx, pe->sseu);
+ if (ret) {
+ err = ERR_PTR(ret);
+ goto free_engines;
+ }
+ }
+
+ /*
+ * XXX: Must be done after calling intel_context_set_gem as that
+ * function changes the ring size. The ring is allocated when
+ * the context is pinned. If the ring size is changed after
+ * allocation we have a mismatch of the ring size and will cause
+ * the context to hang. Presumably with a bit of reordering we
+ * could move the perma-pin step to the backend function
+ * intel_engine_create_parallel.
+ */
+ if (pe[n].type == I915_GEM_ENGINE_TYPE_PARALLEL) {
+ ret = perma_pin_contexts(ce);
+ if (ret) {
+ err = ERR_PTR(ret);
+ goto free_engines;
+ }
+ }
}
return e;
@@ -1219,6 +1444,7 @@ static void context_close(struct i915_gem_context *ctx)
/* Flush any concurrent set_engines() */
mutex_lock(&ctx->engines_mutex);
+ unpin_engines(__context_engines_static(ctx));
engines_idle_release(ctx, rcu_replace_pointer(ctx->engines, NULL, 1));
i915_gem_context_set_closed(ctx);
mutex_unlock(&ctx->engines_mutex);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h
index a627b09c4680..282cdb8a5c5a 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h
+++ b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h
@@ -78,13 +78,16 @@ enum i915_gem_engine_type {
/** @I915_GEM_ENGINE_TYPE_BALANCED: A load-balanced engine set */
I915_GEM_ENGINE_TYPE_BALANCED,
+
+ /** @I915_GEM_ENGINE_TYPE_PARALLEL: A parallel engine set */
+ I915_GEM_ENGINE_TYPE_PARALLEL,
};
/**
* struct i915_gem_proto_engine - prototype engine
*
* This struct describes an engine that a context may contain. Engines
- * have three types:
+ * have four types:
*
* - I915_GEM_ENGINE_TYPE_INVALID: Invalid engines can be created but they
* show up as a NULL in i915_gem_engines::engines[i] and any attempt to
@@ -97,6 +100,10 @@ enum i915_gem_engine_type {
*
* - I915_GEM_ENGINE_TYPE_BALANCED: A load-balanced engine set, described
* i915_gem_proto_engine::num_siblings and i915_gem_proto_engine::siblings.
+ *
+ * - I915_GEM_ENGINE_TYPE_PARALLEL: A parallel submission engine set, described
+ * i915_gem_proto_engine::width, i915_gem_proto_engine::num_siblings, and
+ * i915_gem_proto_engine::siblings.
*/
struct i915_gem_proto_engine {
/** @type: Type of this engine */
@@ -105,10 +112,13 @@ struct i915_gem_proto_engine {
/** @engine: Engine, for physical */
struct intel_engine_cs *engine;
- /** @num_siblings: Number of balanced siblings */
+ /** @num_siblings: Number of balanced or parallel siblings */
unsigned int num_siblings;
- /** @siblings: Balanced siblings */
+ /** @width: Width of each sibling */
+ unsigned int width;
+
+ /** @siblings: Balanced siblings or num_siblings * width for parallel */
struct intel_engine_cs **siblings;
/** @sseu: Client-set SSEU parameters */
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 8309d1141d0a..1d880303a7e4 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -55,9 +55,13 @@ struct intel_context_ops {
void (*reset)(struct intel_context *ce);
void (*destroy)(struct kref *kref);
- /* virtual engine/context interface */
+ /* virtual/parallel engine/context interface */
struct intel_context *(*create_virtual)(struct intel_engine_cs **engine,
- unsigned int count);
+ unsigned int count,
+ unsigned long flags);
+ struct intel_context *(*create_parallel)(struct intel_engine_cs **engines,
+ unsigned int num_siblings,
+ unsigned int width);
struct intel_engine_cs *(*get_sibling)(struct intel_engine_cs *engine,
unsigned int sibling);
};
@@ -113,6 +117,7 @@ struct intel_context {
#define CONTEXT_NOPREEMPT 8
#define CONTEXT_LRCA_DIRTY 9
#define CONTEXT_GUC_INIT 10
+#define CONTEXT_PERMA_PIN 11
struct {
u64 timeout_us;
diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
index d5ac49c0691e..08559ace0ada 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine.h
@@ -282,9 +282,19 @@ intel_engine_has_preempt_reset(const struct intel_engine_cs *engine)
return intel_engine_has_preemption(engine);
}
+#define FORCE_VIRTUAL BIT(0)
struct intel_context *
intel_engine_create_virtual(struct intel_engine_cs **siblings,
- unsigned int count);
+ unsigned int count, unsigned long flags);
+
+static inline struct intel_context *
+intel_engine_create_parallel(struct intel_engine_cs **engines,
+ unsigned int num_engines,
+ unsigned int width)
+{
+ GEM_BUG_ON(!engines[0]->cops->create_parallel);
+ return engines[0]->cops->create_parallel(engines, num_engines, width);
+}
static inline bool
intel_virtual_engine_has_heartbeat(const struct intel_engine_cs *engine)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index 2eb798ad068b..ff6753ccb129 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -1953,16 +1953,16 @@ ktime_t intel_engine_get_busy_time(struct intel_engine_cs *engine, ktime_t *now)
struct intel_context *
intel_engine_create_virtual(struct intel_engine_cs **siblings,
- unsigned int count)
+ unsigned int count, unsigned long flags)
{
if (count == 0)
return ERR_PTR(-EINVAL);
- if (count == 1)
+ if (count == 1 && !(flags & FORCE_VIRTUAL))
return intel_context_create(siblings[0]);
GEM_BUG_ON(!siblings[0]->cops->create_virtual);
- return siblings[0]->cops->create_virtual(siblings, count);
+ return siblings[0]->cops->create_virtual(siblings, count, flags);
}
struct i915_request *
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 43a74b216efb..bedb80057046 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -201,7 +201,8 @@ static struct virtual_engine *to_virtual_engine(struct intel_engine_cs *engine)
}
static struct intel_context *
-execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count);
+execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
+ unsigned long flags);
static struct i915_request *
__active_request(const struct intel_timeline * const tl,
@@ -3784,7 +3785,8 @@ static void virtual_submit_request(struct i915_request *rq)
}
static struct intel_context *
-execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count)
+execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
+ unsigned long flags)
{
struct virtual_engine *ve;
unsigned int n;
diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
index 25a8c4f62b0d..b367ecfa42de 100644
--- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
+++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
@@ -3733,7 +3733,7 @@ static int nop_virtual_engine(struct intel_gt *gt,
GEM_BUG_ON(!nctx || nctx > ARRAY_SIZE(ve));
for (n = 0; n < nctx; n++) {
- ve[n] = intel_engine_create_virtual(siblings, nsibling);
+ ve[n] = intel_engine_create_virtual(siblings, nsibling, 0);
if (IS_ERR(ve[n])) {
err = PTR_ERR(ve[n]);
nctx = n;
@@ -3929,7 +3929,7 @@ static int mask_virtual_engine(struct intel_gt *gt,
* restrict it to our desired engine within the virtual engine.
*/
- ve = intel_engine_create_virtual(siblings, nsibling);
+ ve = intel_engine_create_virtual(siblings, nsibling, 0);
if (IS_ERR(ve)) {
err = PTR_ERR(ve);
goto out_close;
@@ -4060,7 +4060,7 @@ static int slicein_virtual_engine(struct intel_gt *gt,
i915_request_add(rq);
}
- ce = intel_engine_create_virtual(siblings, nsibling);
+ ce = intel_engine_create_virtual(siblings, nsibling, 0);
if (IS_ERR(ce)) {
err = PTR_ERR(ce);
goto out;
@@ -4112,7 +4112,7 @@ static int sliceout_virtual_engine(struct intel_gt *gt,
/* XXX We do not handle oversubscription and fairness with normal rq */
for (n = 0; n < nsibling; n++) {
- ce = intel_engine_create_virtual(siblings, nsibling);
+ ce = intel_engine_create_virtual(siblings, nsibling, 0);
if (IS_ERR(ce)) {
err = PTR_ERR(ce);
goto out;
@@ -4214,7 +4214,7 @@ static int preserved_virtual_engine(struct intel_gt *gt,
if (err)
goto out_scratch;
- ve = intel_engine_create_virtual(siblings, nsibling);
+ ve = intel_engine_create_virtual(siblings, nsibling, 0);
if (IS_ERR(ve)) {
err = PTR_ERR(ve);
goto out_scratch;
@@ -4354,7 +4354,7 @@ static int reset_virtual_engine(struct intel_gt *gt,
if (igt_spinner_init(&spin, gt))
return -ENOMEM;
- ve = intel_engine_create_virtual(siblings, nsibling);
+ ve = intel_engine_create_virtual(siblings, nsibling, 0);
if (IS_ERR(ve)) {
err = PTR_ERR(ve);
goto out_spin;
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 938dc34e8d3a..7c12364a017a 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -124,7 +124,13 @@ struct guc_virtual_engine {
};
static struct intel_context *
-guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count);
+guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
+ unsigned long flags);
+
+static struct intel_context *
+guc_create_parallel(struct intel_engine_cs **engines,
+ unsigned int num_siblings,
+ unsigned int width);
#define GUC_REQUEST_SIZE 64 /* bytes */
@@ -2609,6 +2615,7 @@ static const struct intel_context_ops guc_context_ops = {
.destroy = guc_context_destroy,
.create_virtual = guc_create_virtual,
+ .create_parallel = guc_create_parallel,
};
static void submit_work_cb(struct irq_work *wrk)
@@ -2858,8 +2865,6 @@ static const struct intel_context_ops virtual_guc_context_ops = {
.get_sibling = guc_virtual_get_sibling,
};
-/* Future patches will use this function */
-__maybe_unused
static int guc_parent_context_pin(struct intel_context *ce, void *vaddr)
{
struct intel_engine_cs *engine = guc_virtual_get_sibling(ce->engine, 0);
@@ -2876,8 +2881,6 @@ static int guc_parent_context_pin(struct intel_context *ce, void *vaddr)
return __guc_context_pin(ce, engine, vaddr);
}
-/* Future patches will use this function */
-__maybe_unused
static int guc_child_context_pin(struct intel_context *ce, void *vaddr)
{
struct intel_engine_cs *engine = guc_virtual_get_sibling(ce->engine, 0);
@@ -2889,8 +2892,6 @@ static int guc_child_context_pin(struct intel_context *ce, void *vaddr)
return __guc_context_pin(ce, engine, vaddr);
}
-/* Future patches will use this function */
-__maybe_unused
static void guc_parent_context_unpin(struct intel_context *ce)
{
struct intel_guc *guc = ce_to_guc(ce);
@@ -2906,8 +2907,6 @@ static void guc_parent_context_unpin(struct intel_context *ce)
lrc_unpin(ce);
}
-/* Future patches will use this function */
-__maybe_unused
static void guc_child_context_unpin(struct intel_context *ce)
{
GEM_BUG_ON(context_enabled(ce));
@@ -2918,8 +2917,6 @@ static void guc_child_context_unpin(struct intel_context *ce)
lrc_unpin(ce);
}
-/* Future patches will use this function */
-__maybe_unused
static void guc_child_context_post_unpin(struct intel_context *ce)
{
GEM_BUG_ON(!intel_context_is_child(ce));
@@ -2930,6 +2927,98 @@ static void guc_child_context_post_unpin(struct intel_context *ce)
intel_context_unpin(ce->parallel.parent);
}
+static void guc_child_context_destroy(struct kref *kref)
+{
+ struct intel_context *ce = container_of(kref, typeof(*ce), ref);
+
+ __guc_context_destroy(ce);
+}
+
+static const struct intel_context_ops virtual_parent_context_ops = {
+ .alloc = guc_virtual_context_alloc,
+
+ .pre_pin = guc_context_pre_pin,
+ .pin = guc_parent_context_pin,
+ .unpin = guc_parent_context_unpin,
+ .post_unpin = guc_context_post_unpin,
+
+ .ban = guc_context_ban,
+
+ .cancel_request = guc_context_cancel_request,
+
+ .enter = guc_virtual_context_enter,
+ .exit = guc_virtual_context_exit,
+
+ .sched_disable = guc_context_sched_disable,
+
+ .destroy = guc_context_destroy,
+
+ .get_sibling = guc_virtual_get_sibling,
+};
+
+static const struct intel_context_ops virtual_child_context_ops = {
+ .alloc = guc_virtual_context_alloc,
+
+ .pre_pin = guc_context_pre_pin,
+ .pin = guc_child_context_pin,
+ .unpin = guc_child_context_unpin,
+ .post_unpin = guc_child_context_post_unpin,
+
+ .cancel_request = guc_context_cancel_request,
+
+ .enter = guc_virtual_context_enter,
+ .exit = guc_virtual_context_exit,
+
+ .destroy = guc_child_context_destroy,
+
+ .get_sibling = guc_virtual_get_sibling,
+};
+
+static struct intel_context *
+guc_create_parallel(struct intel_engine_cs **engines,
+ unsigned int num_siblings,
+ unsigned int width)
+{
+ struct intel_engine_cs **siblings = NULL;
+ struct intel_context *parent = NULL, *ce, *err;
+ int i, j;
+
+ siblings = kmalloc_array(num_siblings,
+ sizeof(*siblings),
+ GFP_KERNEL);
+ if (!siblings)
+ return ERR_PTR(-ENOMEM);
+
+ for (i = 0; i < width; ++i) {
+ for (j = 0; j < num_siblings; ++j)
+ siblings[j] = engines[i * num_siblings + j];
+
+ ce = intel_engine_create_virtual(siblings, num_siblings,
+ FORCE_VIRTUAL);
+ if (!ce) {
+ err = ERR_PTR(-ENOMEM);
+ goto unwind;
+ }
+
+ if (i == 0) {
+ parent = ce;
+ parent->ops = &virtual_parent_context_ops;
+ } else {
+ ce->ops = &virtual_child_context_ops;
+ intel_context_bind_parent_child(parent, ce);
+ }
+ }
+
+ kfree(siblings);
+ return parent;
+
+unwind:
+ if (parent)
+ intel_context_put(parent);
+ kfree(siblings);
+ return err;
+}
+
static bool
guc_irq_enable_breadcrumbs(struct intel_breadcrumbs *b)
{
@@ -3756,7 +3845,8 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc,
}
static struct intel_context *
-guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count)
+guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
+ unsigned long flags)
{
struct guc_virtual_engine *ve;
struct intel_guc *guc;
diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
index 0179f92e0916..c2a63e1584cb 100644
--- a/include/uapi/drm/i915_drm.h
+++ b/include/uapi/drm/i915_drm.h
@@ -1824,6 +1824,7 @@ struct drm_i915_gem_context_param {
* Extensions:
* i915_context_engines_load_balance (I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE)
* i915_context_engines_bond (I915_CONTEXT_ENGINES_EXT_BOND)
+ * i915_context_engines_parallel_submit (I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT)
*/
#define I915_CONTEXT_PARAM_ENGINES 0xa
@@ -2098,6 +2099,135 @@ struct i915_context_engines_bond {
struct i915_engine_class_instance engines[N__]; \
} __attribute__((packed)) name__
+/**
+ * struct i915_context_engines_parallel_submit - Configure engine for
+ * parallel submission.
+ *
+ * Setup a slot in the context engine map to allow multiple BBs to be submitted
+ * in a single execbuf IOCTL. Those BBs will then be scheduled to run on the GPU
+ * in parallel. Multiple hardware contexts are created internally in the i915 to
+ * run these BBs. Once a slot is configured for N BBs only N BBs can be
+ * submitted in each execbuf IOCTL and this is implicit behavior e.g. The user
+ * doesn't tell the execbuf IOCTL there are N BBs, the execbuf IOCTL knows how
+ * many BBs there are based on the slot's configuration. The N BBs are the last
+ * N buffer objects or first N if I915_EXEC_BATCH_FIRST is set.
+ *
+ * The default placement behavior is to create implicit bonds between each
+ * context if each context maps to more than 1 physical engine (e.g. context is
+ * a virtual engine). Also we only allow contexts of same engine class and these
+ * contexts must be in logically contiguous order. Examples of the placement
+ * behavior are described below. Lastly, the default is to not allow BBs to be
+ * preempted mid-batch. Rather insert coordinated preemption points on all
+ * hardware contexts between each set of BBs. Flags could be added in the future
+ * to change both of these default behaviors.
+ *
+ * Returns -EINVAL if hardware context placement configuration is invalid or if
+ * the placement configuration isn't supported on the platform / submission
+ * interface.
+ * Returns -ENODEV if extension isn't supported on the platform / submission
+ * interface.
+ *
+ * .. code-block:: none
+ *
+ * Examples syntax:
+ * CS[X] = generic engine of same class, logical instance X
+ * INVALID = I915_ENGINE_CLASS_INVALID, I915_ENGINE_CLASS_INVALID_NONE
+ *
+ * Example 1 pseudo code:
+ * set_engines(INVALID)
+ * set_parallel(engine_index=0, width=2, num_siblings=1,
+ * engines=CS[0],CS[1])
+ *
+ * Results in the following valid placement:
+ * CS[0], CS[1]
+ *
+ * Example 2 pseudo code:
+ * set_engines(INVALID)
+ * set_parallel(engine_index=0, width=2, num_siblings=2,
+ * engines=CS[0],CS[2],CS[1],CS[3])
+ *
+ * Results in the following valid placements:
+ * CS[0], CS[1]
+ * CS[2], CS[3]
+ *
+ * This can be thought of as two virtual engines, each containing two
+ * engines thereby making a 2D array. However, there are bonds tying the
+ * entries together and placing restrictions on how they can be scheduled.
+ * Specifically, the scheduler can choose only vertical columns from the 2D
+ * array. That is, CS[0] is bonded to CS[1] and CS[2] to CS[3]. So if the
+ * scheduler wants to submit to CS[0], it must also choose CS[1] and vice
+ * versa. Same for CS[2] requires also using CS[3].
+ * VE[0] = CS[0], CS[2]
+ * VE[1] = CS[1], CS[3]
+ *
+ * Example 3 pseudo code:
+ * set_engines(INVALID)
+ * set_parallel(engine_index=0, width=2, num_siblings=2,
+ * engines=CS[0],CS[1],CS[1],CS[3])
+ *
+ * Results in the following valid and invalid placements:
+ * CS[0], CS[1]
+ * CS[1], CS[3] - Not logically contiguous, return -EINVAL
+ */
+struct i915_context_engines_parallel_submit {
+ /**
+ * @base: base user extension.
+ */
+ struct i915_user_extension base;
+
+ /**
+ * @engine_index: slot for parallel engine
+ */
+ __u16 engine_index;
+
+ /**
+ * @width: number of contexts per parallel engine or in other words the
+ * number of batches in each submission
+ */
+ __u16 width;
+
+ /**
+ * @num_siblings: number of siblings per context or in other words the
+ * number of possible placements for each submission
+ */
+ __u16 num_siblings;
+
+ /**
+ * @mbz16: reserved for future use; must be zero
+ */
+ __u16 mbz16;
+
+ /**
+ * @flags: all undefined flags must be zero, currently not defined flags
+ */
+ __u64 flags;
+
+ /**
+ * @mbz64: reserved for future use; must be zero
+ */
+ __u64 mbz64[3];
+
+ /**
+ * @engines: 2-d array of engine instances to configure parallel engine
+ *
+ * length = width (i) * num_siblings (j)
+ * index = j + i * num_siblings
+ */
+ struct i915_engine_class_instance engines[0];
+
+} __packed;
+
+#define I915_DEFINE_CONTEXT_ENGINES_PARALLEL_SUBMIT(name__, N__) struct { \
+ struct i915_user_extension base; \
+ __u16 engine_index; \
+ __u16 width; \
+ __u16 num_siblings; \
+ __u16 mbz16; \
+ __u64 flags; \
+ __u64 mbz64[3]; \
+ struct i915_engine_class_instance engines[N__]; \
+} __attribute__((packed)) name__
+
/**
* DOC: Context Engine Map uAPI
*
@@ -2157,6 +2287,7 @@ struct i915_context_param_engines {
__u64 extensions; /* linked chain of extension blocks, 0 terminates */
#define I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE 0 /* see i915_context_engines_load_balance */
#define I915_CONTEXT_ENGINES_EXT_BOND 1 /* see i915_context_engines_bond */
+#define I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT 2 /* see i915_context_engines_parallel_submit */
struct i915_engine_class_instance engines[0];
} __attribute__((packed));
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 16/25] drm/i915/guc: Connect UAPI to GuC multi-lrc interface
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Introduce 'set parallel submit' extension to connect UAPI to GuC
multi-lrc interface. Kernel doc in new uAPI should explain it all.
IGT: https://patchwork.freedesktop.org/patch/447008/?series=93071&rev=1
media UMD: https://github.com/intel/media-driver/pull/1252
v2:
(Daniel Vetter)
- Add IGT link and placeholder for media UMD link
v3:
(Kernel test robot)
- Fix warning in unpin engines call
(John Harrison)
- Reword a bunch of the kernel doc
v4:
(John Harrison)
- Add comment why perma-pin is done after setting gem context
- Update some comments / docs for proto contexts
v5:
(John Harrison)
- Rework perma-pin comment
- Add BUG_IN if context is pinned when setting gem context
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gem/i915_gem_context.c | 230 +++++++++++++++++-
.../gpu/drm/i915/gem/i915_gem_context_types.h | 16 +-
drivers/gpu/drm/i915/gt/intel_context_types.h | 9 +-
drivers/gpu/drm/i915/gt/intel_engine.h | 12 +-
drivers/gpu/drm/i915/gt/intel_engine_cs.c | 6 +-
.../drm/i915/gt/intel_execlists_submission.c | 6 +-
drivers/gpu/drm/i915/gt/selftest_execlists.c | 12 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 114 ++++++++-
include/uapi/drm/i915_drm.h | 131 ++++++++++
9 files changed, 505 insertions(+), 31 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
index d225d3dd0b40..9a00f11fef46 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
@@ -556,9 +556,150 @@ set_proto_ctx_engines_bond(struct i915_user_extension __user *base, void *data)
return 0;
}
+static int
+set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
+ void *data)
+{
+ struct i915_context_engines_parallel_submit __user *ext =
+ container_of_user(base, typeof(*ext), base);
+ const struct set_proto_ctx_engines *set = data;
+ struct drm_i915_private *i915 = set->i915;
+ u64 flags;
+ int err = 0, n, i, j;
+ u16 slot, width, num_siblings;
+ struct intel_engine_cs **siblings = NULL;
+ intel_engine_mask_t prev_mask;
+
+ /* Disabling for now */
+ return -ENODEV;
+
+ /* FIXME: This is NIY for execlists */
+ if (!(intel_uc_uses_guc_submission(&i915->gt.uc)))
+ return -ENODEV;
+
+ if (get_user(slot, &ext->engine_index))
+ return -EFAULT;
+
+ if (get_user(width, &ext->width))
+ return -EFAULT;
+
+ if (get_user(num_siblings, &ext->num_siblings))
+ return -EFAULT;
+
+ if (slot >= set->num_engines) {
+ drm_dbg(&i915->drm, "Invalid placement value, %d >= %d\n",
+ slot, set->num_engines);
+ return -EINVAL;
+ }
+
+ if (set->engines[slot].type != I915_GEM_ENGINE_TYPE_INVALID) {
+ drm_dbg(&i915->drm,
+ "Invalid placement[%d], already occupied\n", slot);
+ return -EINVAL;
+ }
+
+ if (get_user(flags, &ext->flags))
+ return -EFAULT;
+
+ if (flags) {
+ drm_dbg(&i915->drm, "Unknown flags 0x%02llx", flags);
+ return -EINVAL;
+ }
+
+ for (n = 0; n < ARRAY_SIZE(ext->mbz64); n++) {
+ err = check_user_mbz(&ext->mbz64[n]);
+ if (err)
+ return err;
+ }
+
+ if (width < 2) {
+ drm_dbg(&i915->drm, "Width (%d) < 2\n", width);
+ return -EINVAL;
+ }
+
+ if (num_siblings < 1) {
+ drm_dbg(&i915->drm, "Number siblings (%d) < 1\n",
+ num_siblings);
+ return -EINVAL;
+ }
+
+ siblings = kmalloc_array(num_siblings * width,
+ sizeof(*siblings),
+ GFP_KERNEL);
+ if (!siblings)
+ return -ENOMEM;
+
+ /* Create contexts / engines */
+ for (i = 0; i < width; ++i) {
+ intel_engine_mask_t current_mask = 0;
+ struct i915_engine_class_instance prev_engine;
+
+ for (j = 0; j < num_siblings; ++j) {
+ struct i915_engine_class_instance ci;
+
+ n = i * num_siblings + j;
+ if (copy_from_user(&ci, &ext->engines[n], sizeof(ci))) {
+ err = -EFAULT;
+ goto out_err;
+ }
+
+ siblings[n] =
+ intel_engine_lookup_user(i915, ci.engine_class,
+ ci.engine_instance);
+ if (!siblings[n]) {
+ drm_dbg(&i915->drm,
+ "Invalid sibling[%d]: { class:%d, inst:%d }\n",
+ n, ci.engine_class, ci.engine_instance);
+ err = -EINVAL;
+ goto out_err;
+ }
+
+ if (n) {
+ if (prev_engine.engine_class !=
+ ci.engine_class) {
+ drm_dbg(&i915->drm,
+ "Mismatched class %d, %d\n",
+ prev_engine.engine_class,
+ ci.engine_class);
+ err = -EINVAL;
+ goto out_err;
+ }
+ }
+
+ prev_engine = ci;
+ current_mask |= siblings[n]->logical_mask;
+ }
+
+ if (i > 0) {
+ if (current_mask != prev_mask << 1) {
+ drm_dbg(&i915->drm,
+ "Non contiguous logical mask 0x%x, 0x%x\n",
+ prev_mask, current_mask);
+ err = -EINVAL;
+ goto out_err;
+ }
+ }
+ prev_mask = current_mask;
+ }
+
+ set->engines[slot].type = I915_GEM_ENGINE_TYPE_PARALLEL;
+ set->engines[slot].num_siblings = num_siblings;
+ set->engines[slot].width = width;
+ set->engines[slot].siblings = siblings;
+
+ return 0;
+
+out_err:
+ kfree(siblings);
+
+ return err;
+}
+
static const i915_user_extension_fn set_proto_ctx_engines_extensions[] = {
[I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE] = set_proto_ctx_engines_balance,
[I915_CONTEXT_ENGINES_EXT_BOND] = set_proto_ctx_engines_bond,
+ [I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT] =
+ set_proto_ctx_engines_parallel_submit,
};
static int set_proto_ctx_engines(struct drm_i915_file_private *fpriv,
@@ -794,6 +935,7 @@ static int intel_context_set_gem(struct intel_context *ce,
GEM_BUG_ON(rcu_access_pointer(ce->gem_context));
RCU_INIT_POINTER(ce->gem_context, ctx);
+ GEM_BUG_ON(intel_context_is_pinned(ce));
ce->ring_size = SZ_16K;
i915_vm_put(ce->vm);
@@ -818,6 +960,25 @@ static int intel_context_set_gem(struct intel_context *ce,
return ret;
}
+static void __unpin_engines(struct i915_gem_engines *e, unsigned int count)
+{
+ while (count--) {
+ struct intel_context *ce = e->engines[count], *child;
+
+ if (!ce || !test_bit(CONTEXT_PERMA_PIN, &ce->flags))
+ continue;
+
+ for_each_child(ce, child)
+ intel_context_unpin(child);
+ intel_context_unpin(ce);
+ }
+}
+
+static void unpin_engines(struct i915_gem_engines *e)
+{
+ __unpin_engines(e, e->num_engines);
+}
+
static void __free_engines(struct i915_gem_engines *e, unsigned int count)
{
while (count--) {
@@ -933,6 +1094,40 @@ static struct i915_gem_engines *default_engines(struct i915_gem_context *ctx,
return err;
}
+static int perma_pin_contexts(struct intel_context *ce)
+{
+ struct intel_context *child;
+ int i = 0, j = 0, ret;
+
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ ret = intel_context_pin(ce);
+ if (unlikely(ret))
+ return ret;
+
+ for_each_child(ce, child) {
+ ret = intel_context_pin(child);
+ if (unlikely(ret))
+ goto unwind;
+ ++i;
+ }
+
+ set_bit(CONTEXT_PERMA_PIN, &ce->flags);
+
+ return 0;
+
+unwind:
+ intel_context_unpin(ce);
+ for_each_child(ce, child) {
+ if (j++ < i)
+ intel_context_unpin(child);
+ else
+ break;
+ }
+
+ return ret;
+}
+
static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
unsigned int num_engines,
struct i915_gem_proto_engine *pe)
@@ -946,7 +1141,7 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
e->num_engines = num_engines;
for (n = 0; n < num_engines; n++) {
- struct intel_context *ce;
+ struct intel_context *ce, *child;
int ret;
switch (pe[n].type) {
@@ -956,7 +1151,13 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
case I915_GEM_ENGINE_TYPE_BALANCED:
ce = intel_engine_create_virtual(pe[n].siblings,
- pe[n].num_siblings);
+ pe[n].num_siblings, 0);
+ break;
+
+ case I915_GEM_ENGINE_TYPE_PARALLEL:
+ ce = intel_engine_create_parallel(pe[n].siblings,
+ pe[n].num_siblings,
+ pe[n].width);
break;
case I915_GEM_ENGINE_TYPE_INVALID:
@@ -977,6 +1178,30 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
err = ERR_PTR(ret);
goto free_engines;
}
+ for_each_child(ce, child) {
+ ret = intel_context_set_gem(child, ctx, pe->sseu);
+ if (ret) {
+ err = ERR_PTR(ret);
+ goto free_engines;
+ }
+ }
+
+ /*
+ * XXX: Must be done after calling intel_context_set_gem as that
+ * function changes the ring size. The ring is allocated when
+ * the context is pinned. If the ring size is changed after
+ * allocation we have a mismatch of the ring size and will cause
+ * the context to hang. Presumably with a bit of reordering we
+ * could move the perma-pin step to the backend function
+ * intel_engine_create_parallel.
+ */
+ if (pe[n].type == I915_GEM_ENGINE_TYPE_PARALLEL) {
+ ret = perma_pin_contexts(ce);
+ if (ret) {
+ err = ERR_PTR(ret);
+ goto free_engines;
+ }
+ }
}
return e;
@@ -1219,6 +1444,7 @@ static void context_close(struct i915_gem_context *ctx)
/* Flush any concurrent set_engines() */
mutex_lock(&ctx->engines_mutex);
+ unpin_engines(__context_engines_static(ctx));
engines_idle_release(ctx, rcu_replace_pointer(ctx->engines, NULL, 1));
i915_gem_context_set_closed(ctx);
mutex_unlock(&ctx->engines_mutex);
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h
index a627b09c4680..282cdb8a5c5a 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h
+++ b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h
@@ -78,13 +78,16 @@ enum i915_gem_engine_type {
/** @I915_GEM_ENGINE_TYPE_BALANCED: A load-balanced engine set */
I915_GEM_ENGINE_TYPE_BALANCED,
+
+ /** @I915_GEM_ENGINE_TYPE_PARALLEL: A parallel engine set */
+ I915_GEM_ENGINE_TYPE_PARALLEL,
};
/**
* struct i915_gem_proto_engine - prototype engine
*
* This struct describes an engine that a context may contain. Engines
- * have three types:
+ * have four types:
*
* - I915_GEM_ENGINE_TYPE_INVALID: Invalid engines can be created but they
* show up as a NULL in i915_gem_engines::engines[i] and any attempt to
@@ -97,6 +100,10 @@ enum i915_gem_engine_type {
*
* - I915_GEM_ENGINE_TYPE_BALANCED: A load-balanced engine set, described
* i915_gem_proto_engine::num_siblings and i915_gem_proto_engine::siblings.
+ *
+ * - I915_GEM_ENGINE_TYPE_PARALLEL: A parallel submission engine set, described
+ * i915_gem_proto_engine::width, i915_gem_proto_engine::num_siblings, and
+ * i915_gem_proto_engine::siblings.
*/
struct i915_gem_proto_engine {
/** @type: Type of this engine */
@@ -105,10 +112,13 @@ struct i915_gem_proto_engine {
/** @engine: Engine, for physical */
struct intel_engine_cs *engine;
- /** @num_siblings: Number of balanced siblings */
+ /** @num_siblings: Number of balanced or parallel siblings */
unsigned int num_siblings;
- /** @siblings: Balanced siblings */
+ /** @width: Width of each sibling */
+ unsigned int width;
+
+ /** @siblings: Balanced siblings or num_siblings * width for parallel */
struct intel_engine_cs **siblings;
/** @sseu: Client-set SSEU parameters */
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 8309d1141d0a..1d880303a7e4 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -55,9 +55,13 @@ struct intel_context_ops {
void (*reset)(struct intel_context *ce);
void (*destroy)(struct kref *kref);
- /* virtual engine/context interface */
+ /* virtual/parallel engine/context interface */
struct intel_context *(*create_virtual)(struct intel_engine_cs **engine,
- unsigned int count);
+ unsigned int count,
+ unsigned long flags);
+ struct intel_context *(*create_parallel)(struct intel_engine_cs **engines,
+ unsigned int num_siblings,
+ unsigned int width);
struct intel_engine_cs *(*get_sibling)(struct intel_engine_cs *engine,
unsigned int sibling);
};
@@ -113,6 +117,7 @@ struct intel_context {
#define CONTEXT_NOPREEMPT 8
#define CONTEXT_LRCA_DIRTY 9
#define CONTEXT_GUC_INIT 10
+#define CONTEXT_PERMA_PIN 11
struct {
u64 timeout_us;
diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
index d5ac49c0691e..08559ace0ada 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine.h
+++ b/drivers/gpu/drm/i915/gt/intel_engine.h
@@ -282,9 +282,19 @@ intel_engine_has_preempt_reset(const struct intel_engine_cs *engine)
return intel_engine_has_preemption(engine);
}
+#define FORCE_VIRTUAL BIT(0)
struct intel_context *
intel_engine_create_virtual(struct intel_engine_cs **siblings,
- unsigned int count);
+ unsigned int count, unsigned long flags);
+
+static inline struct intel_context *
+intel_engine_create_parallel(struct intel_engine_cs **engines,
+ unsigned int num_engines,
+ unsigned int width)
+{
+ GEM_BUG_ON(!engines[0]->cops->create_parallel);
+ return engines[0]->cops->create_parallel(engines, num_engines, width);
+}
static inline bool
intel_virtual_engine_has_heartbeat(const struct intel_engine_cs *engine)
diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
index 2eb798ad068b..ff6753ccb129 100644
--- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
+++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
@@ -1953,16 +1953,16 @@ ktime_t intel_engine_get_busy_time(struct intel_engine_cs *engine, ktime_t *now)
struct intel_context *
intel_engine_create_virtual(struct intel_engine_cs **siblings,
- unsigned int count)
+ unsigned int count, unsigned long flags)
{
if (count == 0)
return ERR_PTR(-EINVAL);
- if (count == 1)
+ if (count == 1 && !(flags & FORCE_VIRTUAL))
return intel_context_create(siblings[0]);
GEM_BUG_ON(!siblings[0]->cops->create_virtual);
- return siblings[0]->cops->create_virtual(siblings, count);
+ return siblings[0]->cops->create_virtual(siblings, count, flags);
}
struct i915_request *
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 43a74b216efb..bedb80057046 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -201,7 +201,8 @@ static struct virtual_engine *to_virtual_engine(struct intel_engine_cs *engine)
}
static struct intel_context *
-execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count);
+execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
+ unsigned long flags);
static struct i915_request *
__active_request(const struct intel_timeline * const tl,
@@ -3784,7 +3785,8 @@ static void virtual_submit_request(struct i915_request *rq)
}
static struct intel_context *
-execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count)
+execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
+ unsigned long flags)
{
struct virtual_engine *ve;
unsigned int n;
diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
index 25a8c4f62b0d..b367ecfa42de 100644
--- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
+++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
@@ -3733,7 +3733,7 @@ static int nop_virtual_engine(struct intel_gt *gt,
GEM_BUG_ON(!nctx || nctx > ARRAY_SIZE(ve));
for (n = 0; n < nctx; n++) {
- ve[n] = intel_engine_create_virtual(siblings, nsibling);
+ ve[n] = intel_engine_create_virtual(siblings, nsibling, 0);
if (IS_ERR(ve[n])) {
err = PTR_ERR(ve[n]);
nctx = n;
@@ -3929,7 +3929,7 @@ static int mask_virtual_engine(struct intel_gt *gt,
* restrict it to our desired engine within the virtual engine.
*/
- ve = intel_engine_create_virtual(siblings, nsibling);
+ ve = intel_engine_create_virtual(siblings, nsibling, 0);
if (IS_ERR(ve)) {
err = PTR_ERR(ve);
goto out_close;
@@ -4060,7 +4060,7 @@ static int slicein_virtual_engine(struct intel_gt *gt,
i915_request_add(rq);
}
- ce = intel_engine_create_virtual(siblings, nsibling);
+ ce = intel_engine_create_virtual(siblings, nsibling, 0);
if (IS_ERR(ce)) {
err = PTR_ERR(ce);
goto out;
@@ -4112,7 +4112,7 @@ static int sliceout_virtual_engine(struct intel_gt *gt,
/* XXX We do not handle oversubscription and fairness with normal rq */
for (n = 0; n < nsibling; n++) {
- ce = intel_engine_create_virtual(siblings, nsibling);
+ ce = intel_engine_create_virtual(siblings, nsibling, 0);
if (IS_ERR(ce)) {
err = PTR_ERR(ce);
goto out;
@@ -4214,7 +4214,7 @@ static int preserved_virtual_engine(struct intel_gt *gt,
if (err)
goto out_scratch;
- ve = intel_engine_create_virtual(siblings, nsibling);
+ ve = intel_engine_create_virtual(siblings, nsibling, 0);
if (IS_ERR(ve)) {
err = PTR_ERR(ve);
goto out_scratch;
@@ -4354,7 +4354,7 @@ static int reset_virtual_engine(struct intel_gt *gt,
if (igt_spinner_init(&spin, gt))
return -ENOMEM;
- ve = intel_engine_create_virtual(siblings, nsibling);
+ ve = intel_engine_create_virtual(siblings, nsibling, 0);
if (IS_ERR(ve)) {
err = PTR_ERR(ve);
goto out_spin;
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 938dc34e8d3a..7c12364a017a 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -124,7 +124,13 @@ struct guc_virtual_engine {
};
static struct intel_context *
-guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count);
+guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
+ unsigned long flags);
+
+static struct intel_context *
+guc_create_parallel(struct intel_engine_cs **engines,
+ unsigned int num_siblings,
+ unsigned int width);
#define GUC_REQUEST_SIZE 64 /* bytes */
@@ -2609,6 +2615,7 @@ static const struct intel_context_ops guc_context_ops = {
.destroy = guc_context_destroy,
.create_virtual = guc_create_virtual,
+ .create_parallel = guc_create_parallel,
};
static void submit_work_cb(struct irq_work *wrk)
@@ -2858,8 +2865,6 @@ static const struct intel_context_ops virtual_guc_context_ops = {
.get_sibling = guc_virtual_get_sibling,
};
-/* Future patches will use this function */
-__maybe_unused
static int guc_parent_context_pin(struct intel_context *ce, void *vaddr)
{
struct intel_engine_cs *engine = guc_virtual_get_sibling(ce->engine, 0);
@@ -2876,8 +2881,6 @@ static int guc_parent_context_pin(struct intel_context *ce, void *vaddr)
return __guc_context_pin(ce, engine, vaddr);
}
-/* Future patches will use this function */
-__maybe_unused
static int guc_child_context_pin(struct intel_context *ce, void *vaddr)
{
struct intel_engine_cs *engine = guc_virtual_get_sibling(ce->engine, 0);
@@ -2889,8 +2892,6 @@ static int guc_child_context_pin(struct intel_context *ce, void *vaddr)
return __guc_context_pin(ce, engine, vaddr);
}
-/* Future patches will use this function */
-__maybe_unused
static void guc_parent_context_unpin(struct intel_context *ce)
{
struct intel_guc *guc = ce_to_guc(ce);
@@ -2906,8 +2907,6 @@ static void guc_parent_context_unpin(struct intel_context *ce)
lrc_unpin(ce);
}
-/* Future patches will use this function */
-__maybe_unused
static void guc_child_context_unpin(struct intel_context *ce)
{
GEM_BUG_ON(context_enabled(ce));
@@ -2918,8 +2917,6 @@ static void guc_child_context_unpin(struct intel_context *ce)
lrc_unpin(ce);
}
-/* Future patches will use this function */
-__maybe_unused
static void guc_child_context_post_unpin(struct intel_context *ce)
{
GEM_BUG_ON(!intel_context_is_child(ce));
@@ -2930,6 +2927,98 @@ static void guc_child_context_post_unpin(struct intel_context *ce)
intel_context_unpin(ce->parallel.parent);
}
+static void guc_child_context_destroy(struct kref *kref)
+{
+ struct intel_context *ce = container_of(kref, typeof(*ce), ref);
+
+ __guc_context_destroy(ce);
+}
+
+static const struct intel_context_ops virtual_parent_context_ops = {
+ .alloc = guc_virtual_context_alloc,
+
+ .pre_pin = guc_context_pre_pin,
+ .pin = guc_parent_context_pin,
+ .unpin = guc_parent_context_unpin,
+ .post_unpin = guc_context_post_unpin,
+
+ .ban = guc_context_ban,
+
+ .cancel_request = guc_context_cancel_request,
+
+ .enter = guc_virtual_context_enter,
+ .exit = guc_virtual_context_exit,
+
+ .sched_disable = guc_context_sched_disable,
+
+ .destroy = guc_context_destroy,
+
+ .get_sibling = guc_virtual_get_sibling,
+};
+
+static const struct intel_context_ops virtual_child_context_ops = {
+ .alloc = guc_virtual_context_alloc,
+
+ .pre_pin = guc_context_pre_pin,
+ .pin = guc_child_context_pin,
+ .unpin = guc_child_context_unpin,
+ .post_unpin = guc_child_context_post_unpin,
+
+ .cancel_request = guc_context_cancel_request,
+
+ .enter = guc_virtual_context_enter,
+ .exit = guc_virtual_context_exit,
+
+ .destroy = guc_child_context_destroy,
+
+ .get_sibling = guc_virtual_get_sibling,
+};
+
+static struct intel_context *
+guc_create_parallel(struct intel_engine_cs **engines,
+ unsigned int num_siblings,
+ unsigned int width)
+{
+ struct intel_engine_cs **siblings = NULL;
+ struct intel_context *parent = NULL, *ce, *err;
+ int i, j;
+
+ siblings = kmalloc_array(num_siblings,
+ sizeof(*siblings),
+ GFP_KERNEL);
+ if (!siblings)
+ return ERR_PTR(-ENOMEM);
+
+ for (i = 0; i < width; ++i) {
+ for (j = 0; j < num_siblings; ++j)
+ siblings[j] = engines[i * num_siblings + j];
+
+ ce = intel_engine_create_virtual(siblings, num_siblings,
+ FORCE_VIRTUAL);
+ if (!ce) {
+ err = ERR_PTR(-ENOMEM);
+ goto unwind;
+ }
+
+ if (i == 0) {
+ parent = ce;
+ parent->ops = &virtual_parent_context_ops;
+ } else {
+ ce->ops = &virtual_child_context_ops;
+ intel_context_bind_parent_child(parent, ce);
+ }
+ }
+
+ kfree(siblings);
+ return parent;
+
+unwind:
+ if (parent)
+ intel_context_put(parent);
+ kfree(siblings);
+ return err;
+}
+
static bool
guc_irq_enable_breadcrumbs(struct intel_breadcrumbs *b)
{
@@ -3756,7 +3845,8 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc,
}
static struct intel_context *
-guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count)
+guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
+ unsigned long flags)
{
struct guc_virtual_engine *ve;
struct intel_guc *guc;
diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
index 0179f92e0916..c2a63e1584cb 100644
--- a/include/uapi/drm/i915_drm.h
+++ b/include/uapi/drm/i915_drm.h
@@ -1824,6 +1824,7 @@ struct drm_i915_gem_context_param {
* Extensions:
* i915_context_engines_load_balance (I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE)
* i915_context_engines_bond (I915_CONTEXT_ENGINES_EXT_BOND)
+ * i915_context_engines_parallel_submit (I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT)
*/
#define I915_CONTEXT_PARAM_ENGINES 0xa
@@ -2098,6 +2099,135 @@ struct i915_context_engines_bond {
struct i915_engine_class_instance engines[N__]; \
} __attribute__((packed)) name__
+/**
+ * struct i915_context_engines_parallel_submit - Configure engine for
+ * parallel submission.
+ *
+ * Setup a slot in the context engine map to allow multiple BBs to be submitted
+ * in a single execbuf IOCTL. Those BBs will then be scheduled to run on the GPU
+ * in parallel. Multiple hardware contexts are created internally in the i915 to
+ * run these BBs. Once a slot is configured for N BBs only N BBs can be
+ * submitted in each execbuf IOCTL and this is implicit behavior e.g. The user
+ * doesn't tell the execbuf IOCTL there are N BBs, the execbuf IOCTL knows how
+ * many BBs there are based on the slot's configuration. The N BBs are the last
+ * N buffer objects or first N if I915_EXEC_BATCH_FIRST is set.
+ *
+ * The default placement behavior is to create implicit bonds between each
+ * context if each context maps to more than 1 physical engine (e.g. context is
+ * a virtual engine). Also we only allow contexts of same engine class and these
+ * contexts must be in logically contiguous order. Examples of the placement
+ * behavior are described below. Lastly, the default is to not allow BBs to be
+ * preempted mid-batch. Rather insert coordinated preemption points on all
+ * hardware contexts between each set of BBs. Flags could be added in the future
+ * to change both of these default behaviors.
+ *
+ * Returns -EINVAL if hardware context placement configuration is invalid or if
+ * the placement configuration isn't supported on the platform / submission
+ * interface.
+ * Returns -ENODEV if extension isn't supported on the platform / submission
+ * interface.
+ *
+ * .. code-block:: none
+ *
+ * Examples syntax:
+ * CS[X] = generic engine of same class, logical instance X
+ * INVALID = I915_ENGINE_CLASS_INVALID, I915_ENGINE_CLASS_INVALID_NONE
+ *
+ * Example 1 pseudo code:
+ * set_engines(INVALID)
+ * set_parallel(engine_index=0, width=2, num_siblings=1,
+ * engines=CS[0],CS[1])
+ *
+ * Results in the following valid placement:
+ * CS[0], CS[1]
+ *
+ * Example 2 pseudo code:
+ * set_engines(INVALID)
+ * set_parallel(engine_index=0, width=2, num_siblings=2,
+ * engines=CS[0],CS[2],CS[1],CS[3])
+ *
+ * Results in the following valid placements:
+ * CS[0], CS[1]
+ * CS[2], CS[3]
+ *
+ * This can be thought of as two virtual engines, each containing two
+ * engines thereby making a 2D array. However, there are bonds tying the
+ * entries together and placing restrictions on how they can be scheduled.
+ * Specifically, the scheduler can choose only vertical columns from the 2D
+ * array. That is, CS[0] is bonded to CS[1] and CS[2] to CS[3]. So if the
+ * scheduler wants to submit to CS[0], it must also choose CS[1] and vice
+ * versa. Same for CS[2] requires also using CS[3].
+ * VE[0] = CS[0], CS[2]
+ * VE[1] = CS[1], CS[3]
+ *
+ * Example 3 pseudo code:
+ * set_engines(INVALID)
+ * set_parallel(engine_index=0, width=2, num_siblings=2,
+ * engines=CS[0],CS[1],CS[1],CS[3])
+ *
+ * Results in the following valid and invalid placements:
+ * CS[0], CS[1]
+ * CS[1], CS[3] - Not logically contiguous, return -EINVAL
+ */
+struct i915_context_engines_parallel_submit {
+ /**
+ * @base: base user extension.
+ */
+ struct i915_user_extension base;
+
+ /**
+ * @engine_index: slot for parallel engine
+ */
+ __u16 engine_index;
+
+ /**
+ * @width: number of contexts per parallel engine or in other words the
+ * number of batches in each submission
+ */
+ __u16 width;
+
+ /**
+ * @num_siblings: number of siblings per context or in other words the
+ * number of possible placements for each submission
+ */
+ __u16 num_siblings;
+
+ /**
+ * @mbz16: reserved for future use; must be zero
+ */
+ __u16 mbz16;
+
+ /**
+ * @flags: all undefined flags must be zero, currently not defined flags
+ */
+ __u64 flags;
+
+ /**
+ * @mbz64: reserved for future use; must be zero
+ */
+ __u64 mbz64[3];
+
+ /**
+ * @engines: 2-d array of engine instances to configure parallel engine
+ *
+ * length = width (i) * num_siblings (j)
+ * index = j + i * num_siblings
+ */
+ struct i915_engine_class_instance engines[0];
+
+} __packed;
+
+#define I915_DEFINE_CONTEXT_ENGINES_PARALLEL_SUBMIT(name__, N__) struct { \
+ struct i915_user_extension base; \
+ __u16 engine_index; \
+ __u16 width; \
+ __u16 num_siblings; \
+ __u16 mbz16; \
+ __u64 flags; \
+ __u64 mbz64[3]; \
+ struct i915_engine_class_instance engines[N__]; \
+} __attribute__((packed)) name__
+
/**
* DOC: Context Engine Map uAPI
*
@@ -2157,6 +2287,7 @@ struct i915_context_param_engines {
__u64 extensions; /* linked chain of extension blocks, 0 terminates */
#define I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE 0 /* see i915_context_engines_load_balance */
#define I915_CONTEXT_ENGINES_EXT_BOND 1 /* see i915_context_engines_bond */
+#define I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT 2 /* see i915_context_engines_parallel_submit */
struct i915_engine_class_instance engines[0];
} __attribute__((packed));
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [Intel-gfx] [PATCH 16/25] drm/i915/guc: Connect UAPI to GuC multi-lrc interface
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 18:24 ` John Harrison
-1 siblings, 0 replies; 69+ messages in thread
From: John Harrison @ 2021-10-14 18:24 UTC (permalink / raw)
To: Matthew Brost, intel-gfx, dri-devel
On 10/14/2021 10:19, Matthew Brost wrote:
> Introduce 'set parallel submit' extension to connect UAPI to GuC
> multi-lrc interface. Kernel doc in new uAPI should explain it all.
>
> IGT: https://patchwork.freedesktop.org/patch/447008/?series=93071&rev=1
> media UMD: https://github.com/intel/media-driver/pull/1252
>
> v2:
> (Daniel Vetter)
> - Add IGT link and placeholder for media UMD link
> v3:
> (Kernel test robot)
> - Fix warning in unpin engines call
> (John Harrison)
> - Reword a bunch of the kernel doc
> v4:
> (John Harrison)
> - Add comment why perma-pin is done after setting gem context
> - Update some comments / docs for proto contexts
> v5:
> (John Harrison)
> - Rework perma-pin comment
> - Add BUG_IN if context is pinned when setting gem context
IN?
>
> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/gem/i915_gem_context.c | 230 +++++++++++++++++-
> .../gpu/drm/i915/gem/i915_gem_context_types.h | 16 +-
> drivers/gpu/drm/i915/gt/intel_context_types.h | 9 +-
> drivers/gpu/drm/i915/gt/intel_engine.h | 12 +-
> drivers/gpu/drm/i915/gt/intel_engine_cs.c | 6 +-
> .../drm/i915/gt/intel_execlists_submission.c | 6 +-
> drivers/gpu/drm/i915/gt/selftest_execlists.c | 12 +-
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 114 ++++++++-
> include/uapi/drm/i915_drm.h | 131 ++++++++++
> 9 files changed, 505 insertions(+), 31 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> index d225d3dd0b40..9a00f11fef46 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> @@ -556,9 +556,150 @@ set_proto_ctx_engines_bond(struct i915_user_extension __user *base, void *data)
> return 0;
> }
>
> +static int
> +set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
> + void *data)
> +{
> + struct i915_context_engines_parallel_submit __user *ext =
> + container_of_user(base, typeof(*ext), base);
> + const struct set_proto_ctx_engines *set = data;
> + struct drm_i915_private *i915 = set->i915;
> + u64 flags;
> + int err = 0, n, i, j;
> + u16 slot, width, num_siblings;
> + struct intel_engine_cs **siblings = NULL;
> + intel_engine_mask_t prev_mask;
> +
> + /* Disabling for now */
> + return -ENODEV;
> +
> + /* FIXME: This is NIY for execlists */
> + if (!(intel_uc_uses_guc_submission(&i915->gt.uc)))
> + return -ENODEV;
> +
> + if (get_user(slot, &ext->engine_index))
> + return -EFAULT;
> +
> + if (get_user(width, &ext->width))
> + return -EFAULT;
> +
> + if (get_user(num_siblings, &ext->num_siblings))
> + return -EFAULT;
> +
> + if (slot >= set->num_engines) {
> + drm_dbg(&i915->drm, "Invalid placement value, %d >= %d\n",
> + slot, set->num_engines);
> + return -EINVAL;
> + }
> +
> + if (set->engines[slot].type != I915_GEM_ENGINE_TYPE_INVALID) {
> + drm_dbg(&i915->drm,
> + "Invalid placement[%d], already occupied\n", slot);
> + return -EINVAL;
> + }
> +
> + if (get_user(flags, &ext->flags))
> + return -EFAULT;
> +
> + if (flags) {
> + drm_dbg(&i915->drm, "Unknown flags 0x%02llx", flags);
> + return -EINVAL;
> + }
> +
> + for (n = 0; n < ARRAY_SIZE(ext->mbz64); n++) {
> + err = check_user_mbz(&ext->mbz64[n]);
> + if (err)
> + return err;
> + }
> +
> + if (width < 2) {
> + drm_dbg(&i915->drm, "Width (%d) < 2\n", width);
> + return -EINVAL;
> + }
> +
> + if (num_siblings < 1) {
> + drm_dbg(&i915->drm, "Number siblings (%d) < 1\n",
> + num_siblings);
> + return -EINVAL;
> + }
> +
> + siblings = kmalloc_array(num_siblings * width,
> + sizeof(*siblings),
> + GFP_KERNEL);
> + if (!siblings)
> + return -ENOMEM;
> +
> + /* Create contexts / engines */
> + for (i = 0; i < width; ++i) {
> + intel_engine_mask_t current_mask = 0;
> + struct i915_engine_class_instance prev_engine;
> +
> + for (j = 0; j < num_siblings; ++j) {
> + struct i915_engine_class_instance ci;
> +
> + n = i * num_siblings + j;
> + if (copy_from_user(&ci, &ext->engines[n], sizeof(ci))) {
> + err = -EFAULT;
> + goto out_err;
> + }
> +
> + siblings[n] =
> + intel_engine_lookup_user(i915, ci.engine_class,
> + ci.engine_instance);
> + if (!siblings[n]) {
> + drm_dbg(&i915->drm,
> + "Invalid sibling[%d]: { class:%d, inst:%d }\n",
> + n, ci.engine_class, ci.engine_instance);
> + err = -EINVAL;
> + goto out_err;
> + }
> +
> + if (n) {
> + if (prev_engine.engine_class !=
> + ci.engine_class) {
> + drm_dbg(&i915->drm,
> + "Mismatched class %d, %d\n",
> + prev_engine.engine_class,
> + ci.engine_class);
> + err = -EINVAL;
> + goto out_err;
> + }
> + }
> +
> + prev_engine = ci;
> + current_mask |= siblings[n]->logical_mask;
> + }
> +
> + if (i > 0) {
> + if (current_mask != prev_mask << 1) {
> + drm_dbg(&i915->drm,
> + "Non contiguous logical mask 0x%x, 0x%x\n",
> + prev_mask, current_mask);
> + err = -EINVAL;
> + goto out_err;
> + }
> + }
> + prev_mask = current_mask;
> + }
> +
> + set->engines[slot].type = I915_GEM_ENGINE_TYPE_PARALLEL;
> + set->engines[slot].num_siblings = num_siblings;
> + set->engines[slot].width = width;
> + set->engines[slot].siblings = siblings;
> +
> + return 0;
> +
> +out_err:
> + kfree(siblings);
> +
> + return err;
> +}
> +
> static const i915_user_extension_fn set_proto_ctx_engines_extensions[] = {
> [I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE] = set_proto_ctx_engines_balance,
> [I915_CONTEXT_ENGINES_EXT_BOND] = set_proto_ctx_engines_bond,
> + [I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT] =
> + set_proto_ctx_engines_parallel_submit,
> };
>
> static int set_proto_ctx_engines(struct drm_i915_file_private *fpriv,
> @@ -794,6 +935,7 @@ static int intel_context_set_gem(struct intel_context *ce,
> GEM_BUG_ON(rcu_access_pointer(ce->gem_context));
> RCU_INIT_POINTER(ce->gem_context, ctx);
>
> + GEM_BUG_ON(intel_context_is_pinned(ce));
> ce->ring_size = SZ_16K;
>
> i915_vm_put(ce->vm);
> @@ -818,6 +960,25 @@ static int intel_context_set_gem(struct intel_context *ce,
> return ret;
> }
>
> +static void __unpin_engines(struct i915_gem_engines *e, unsigned int count)
> +{
> + while (count--) {
> + struct intel_context *ce = e->engines[count], *child;
> +
> + if (!ce || !test_bit(CONTEXT_PERMA_PIN, &ce->flags))
> + continue;
> +
> + for_each_child(ce, child)
> + intel_context_unpin(child);
> + intel_context_unpin(ce);
> + }
> +}
> +
> +static void unpin_engines(struct i915_gem_engines *e)
> +{
> + __unpin_engines(e, e->num_engines);
> +}
> +
> static void __free_engines(struct i915_gem_engines *e, unsigned int count)
> {
> while (count--) {
> @@ -933,6 +1094,40 @@ static struct i915_gem_engines *default_engines(struct i915_gem_context *ctx,
> return err;
> }
>
> +static int perma_pin_contexts(struct intel_context *ce)
> +{
> + struct intel_context *child;
> + int i = 0, j = 0, ret;
> +
> + GEM_BUG_ON(!intel_context_is_parent(ce));
> +
> + ret = intel_context_pin(ce);
> + if (unlikely(ret))
> + return ret;
> +
> + for_each_child(ce, child) {
> + ret = intel_context_pin(child);
> + if (unlikely(ret))
> + goto unwind;
> + ++i;
> + }
> +
> + set_bit(CONTEXT_PERMA_PIN, &ce->flags);
> +
> + return 0;
> +
> +unwind:
> + intel_context_unpin(ce);
> + for_each_child(ce, child) {
> + if (j++ < i)
> + intel_context_unpin(child);
> + else
> + break;
> + }
> +
> + return ret;
> +}
> +
> static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
> unsigned int num_engines,
> struct i915_gem_proto_engine *pe)
> @@ -946,7 +1141,7 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
> e->num_engines = num_engines;
>
> for (n = 0; n < num_engines; n++) {
> - struct intel_context *ce;
> + struct intel_context *ce, *child;
> int ret;
>
> switch (pe[n].type) {
> @@ -956,7 +1151,13 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
>
> case I915_GEM_ENGINE_TYPE_BALANCED:
> ce = intel_engine_create_virtual(pe[n].siblings,
> - pe[n].num_siblings);
> + pe[n].num_siblings, 0);
> + break;
> +
> + case I915_GEM_ENGINE_TYPE_PARALLEL:
> + ce = intel_engine_create_parallel(pe[n].siblings,
> + pe[n].num_siblings,
> + pe[n].width);
> break;
>
> case I915_GEM_ENGINE_TYPE_INVALID:
> @@ -977,6 +1178,30 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
> err = ERR_PTR(ret);
> goto free_engines;
> }
> + for_each_child(ce, child) {
> + ret = intel_context_set_gem(child, ctx, pe->sseu);
> + if (ret) {
> + err = ERR_PTR(ret);
> + goto free_engines;
> + }
> + }
> +
> + /*
> + * XXX: Must be done after calling intel_context_set_gem as that
> + * function changes the ring size. The ring is allocated when
> + * the context is pinned. If the ring size is changed after
> + * allocation we have a mismatch of the ring size and will cause
> + * the context to hang. Presumably with a bit of reordering we
> + * could move the perma-pin step to the backend function
> + * intel_engine_create_parallel.
> + */
> + if (pe[n].type == I915_GEM_ENGINE_TYPE_PARALLEL) {
> + ret = perma_pin_contexts(ce);
> + if (ret) {
> + err = ERR_PTR(ret);
> + goto free_engines;
> + }
> + }
> }
>
> return e;
> @@ -1219,6 +1444,7 @@ static void context_close(struct i915_gem_context *ctx)
>
> /* Flush any concurrent set_engines() */
> mutex_lock(&ctx->engines_mutex);
> + unpin_engines(__context_engines_static(ctx));
> engines_idle_release(ctx, rcu_replace_pointer(ctx->engines, NULL, 1));
> i915_gem_context_set_closed(ctx);
> mutex_unlock(&ctx->engines_mutex);
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h
> index a627b09c4680..282cdb8a5c5a 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h
> @@ -78,13 +78,16 @@ enum i915_gem_engine_type {
>
> /** @I915_GEM_ENGINE_TYPE_BALANCED: A load-balanced engine set */
> I915_GEM_ENGINE_TYPE_BALANCED,
> +
> + /** @I915_GEM_ENGINE_TYPE_PARALLEL: A parallel engine set */
> + I915_GEM_ENGINE_TYPE_PARALLEL,
> };
>
> /**
> * struct i915_gem_proto_engine - prototype engine
> *
> * This struct describes an engine that a context may contain. Engines
> - * have three types:
> + * have four types:
> *
> * - I915_GEM_ENGINE_TYPE_INVALID: Invalid engines can be created but they
> * show up as a NULL in i915_gem_engines::engines[i] and any attempt to
> @@ -97,6 +100,10 @@ enum i915_gem_engine_type {
> *
> * - I915_GEM_ENGINE_TYPE_BALANCED: A load-balanced engine set, described
> * i915_gem_proto_engine::num_siblings and i915_gem_proto_engine::siblings.
> + *
> + * - I915_GEM_ENGINE_TYPE_PARALLEL: A parallel submission engine set, described
> + * i915_gem_proto_engine::width, i915_gem_proto_engine::num_siblings, and
> + * i915_gem_proto_engine::siblings.
> */
> struct i915_gem_proto_engine {
> /** @type: Type of this engine */
> @@ -105,10 +112,13 @@ struct i915_gem_proto_engine {
> /** @engine: Engine, for physical */
> struct intel_engine_cs *engine;
>
> - /** @num_siblings: Number of balanced siblings */
> + /** @num_siblings: Number of balanced or parallel siblings */
> unsigned int num_siblings;
>
> - /** @siblings: Balanced siblings */
> + /** @width: Width of each sibling */
> + unsigned int width;
> +
> + /** @siblings: Balanced siblings or num_siblings * width for parallel */
> struct intel_engine_cs **siblings;
>
> /** @sseu: Client-set SSEU parameters */
> diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
> index 8309d1141d0a..1d880303a7e4 100644
> --- a/drivers/gpu/drm/i915/gt/intel_context_types.h
> +++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
> @@ -55,9 +55,13 @@ struct intel_context_ops {
> void (*reset)(struct intel_context *ce);
> void (*destroy)(struct kref *kref);
>
> - /* virtual engine/context interface */
> + /* virtual/parallel engine/context interface */
> struct intel_context *(*create_virtual)(struct intel_engine_cs **engine,
> - unsigned int count);
> + unsigned int count,
> + unsigned long flags);
> + struct intel_context *(*create_parallel)(struct intel_engine_cs **engines,
> + unsigned int num_siblings,
> + unsigned int width);
> struct intel_engine_cs *(*get_sibling)(struct intel_engine_cs *engine,
> unsigned int sibling);
> };
> @@ -113,6 +117,7 @@ struct intel_context {
> #define CONTEXT_NOPREEMPT 8
> #define CONTEXT_LRCA_DIRTY 9
> #define CONTEXT_GUC_INIT 10
> +#define CONTEXT_PERMA_PIN 11
>
> struct {
> u64 timeout_us;
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
> index d5ac49c0691e..08559ace0ada 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine.h
> +++ b/drivers/gpu/drm/i915/gt/intel_engine.h
> @@ -282,9 +282,19 @@ intel_engine_has_preempt_reset(const struct intel_engine_cs *engine)
> return intel_engine_has_preemption(engine);
> }
>
> +#define FORCE_VIRTUAL BIT(0)
> struct intel_context *
> intel_engine_create_virtual(struct intel_engine_cs **siblings,
> - unsigned int count);
> + unsigned int count, unsigned long flags);
> +
> +static inline struct intel_context *
> +intel_engine_create_parallel(struct intel_engine_cs **engines,
> + unsigned int num_engines,
> + unsigned int width)
> +{
> + GEM_BUG_ON(!engines[0]->cops->create_parallel);
> + return engines[0]->cops->create_parallel(engines, num_engines, width);
> +}
>
> static inline bool
> intel_virtual_engine_has_heartbeat(const struct intel_engine_cs *engine)
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> index 2eb798ad068b..ff6753ccb129 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> @@ -1953,16 +1953,16 @@ ktime_t intel_engine_get_busy_time(struct intel_engine_cs *engine, ktime_t *now)
>
> struct intel_context *
> intel_engine_create_virtual(struct intel_engine_cs **siblings,
> - unsigned int count)
> + unsigned int count, unsigned long flags)
> {
> if (count == 0)
> return ERR_PTR(-EINVAL);
>
> - if (count == 1)
> + if (count == 1 && !(flags & FORCE_VIRTUAL))
> return intel_context_create(siblings[0]);
>
> GEM_BUG_ON(!siblings[0]->cops->create_virtual);
> - return siblings[0]->cops->create_virtual(siblings, count);
> + return siblings[0]->cops->create_virtual(siblings, count, flags);
> }
>
> struct i915_request *
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 43a74b216efb..bedb80057046 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -201,7 +201,8 @@ static struct virtual_engine *to_virtual_engine(struct intel_engine_cs *engine)
> }
>
> static struct intel_context *
> -execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count);
> +execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
> + unsigned long flags);
>
> static struct i915_request *
> __active_request(const struct intel_timeline * const tl,
> @@ -3784,7 +3785,8 @@ static void virtual_submit_request(struct i915_request *rq)
> }
>
> static struct intel_context *
> -execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count)
> +execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
> + unsigned long flags)
> {
> struct virtual_engine *ve;
> unsigned int n;
> diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> index 25a8c4f62b0d..b367ecfa42de 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> @@ -3733,7 +3733,7 @@ static int nop_virtual_engine(struct intel_gt *gt,
> GEM_BUG_ON(!nctx || nctx > ARRAY_SIZE(ve));
>
> for (n = 0; n < nctx; n++) {
> - ve[n] = intel_engine_create_virtual(siblings, nsibling);
> + ve[n] = intel_engine_create_virtual(siblings, nsibling, 0);
> if (IS_ERR(ve[n])) {
> err = PTR_ERR(ve[n]);
> nctx = n;
> @@ -3929,7 +3929,7 @@ static int mask_virtual_engine(struct intel_gt *gt,
> * restrict it to our desired engine within the virtual engine.
> */
>
> - ve = intel_engine_create_virtual(siblings, nsibling);
> + ve = intel_engine_create_virtual(siblings, nsibling, 0);
> if (IS_ERR(ve)) {
> err = PTR_ERR(ve);
> goto out_close;
> @@ -4060,7 +4060,7 @@ static int slicein_virtual_engine(struct intel_gt *gt,
> i915_request_add(rq);
> }
>
> - ce = intel_engine_create_virtual(siblings, nsibling);
> + ce = intel_engine_create_virtual(siblings, nsibling, 0);
> if (IS_ERR(ce)) {
> err = PTR_ERR(ce);
> goto out;
> @@ -4112,7 +4112,7 @@ static int sliceout_virtual_engine(struct intel_gt *gt,
>
> /* XXX We do not handle oversubscription and fairness with normal rq */
> for (n = 0; n < nsibling; n++) {
> - ce = intel_engine_create_virtual(siblings, nsibling);
> + ce = intel_engine_create_virtual(siblings, nsibling, 0);
> if (IS_ERR(ce)) {
> err = PTR_ERR(ce);
> goto out;
> @@ -4214,7 +4214,7 @@ static int preserved_virtual_engine(struct intel_gt *gt,
> if (err)
> goto out_scratch;
>
> - ve = intel_engine_create_virtual(siblings, nsibling);
> + ve = intel_engine_create_virtual(siblings, nsibling, 0);
> if (IS_ERR(ve)) {
> err = PTR_ERR(ve);
> goto out_scratch;
> @@ -4354,7 +4354,7 @@ static int reset_virtual_engine(struct intel_gt *gt,
> if (igt_spinner_init(&spin, gt))
> return -ENOMEM;
>
> - ve = intel_engine_create_virtual(siblings, nsibling);
> + ve = intel_engine_create_virtual(siblings, nsibling, 0);
> if (IS_ERR(ve)) {
> err = PTR_ERR(ve);
> goto out_spin;
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 938dc34e8d3a..7c12364a017a 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -124,7 +124,13 @@ struct guc_virtual_engine {
> };
>
> static struct intel_context *
> -guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count);
> +guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
> + unsigned long flags);
> +
> +static struct intel_context *
> +guc_create_parallel(struct intel_engine_cs **engines,
> + unsigned int num_siblings,
> + unsigned int width);
>
> #define GUC_REQUEST_SIZE 64 /* bytes */
>
> @@ -2609,6 +2615,7 @@ static const struct intel_context_ops guc_context_ops = {
> .destroy = guc_context_destroy,
>
> .create_virtual = guc_create_virtual,
> + .create_parallel = guc_create_parallel,
> };
>
> static void submit_work_cb(struct irq_work *wrk)
> @@ -2858,8 +2865,6 @@ static const struct intel_context_ops virtual_guc_context_ops = {
> .get_sibling = guc_virtual_get_sibling,
> };
>
> -/* Future patches will use this function */
> -__maybe_unused
> static int guc_parent_context_pin(struct intel_context *ce, void *vaddr)
> {
> struct intel_engine_cs *engine = guc_virtual_get_sibling(ce->engine, 0);
> @@ -2876,8 +2881,6 @@ static int guc_parent_context_pin(struct intel_context *ce, void *vaddr)
> return __guc_context_pin(ce, engine, vaddr);
> }
>
> -/* Future patches will use this function */
> -__maybe_unused
> static int guc_child_context_pin(struct intel_context *ce, void *vaddr)
> {
> struct intel_engine_cs *engine = guc_virtual_get_sibling(ce->engine, 0);
> @@ -2889,8 +2892,6 @@ static int guc_child_context_pin(struct intel_context *ce, void *vaddr)
> return __guc_context_pin(ce, engine, vaddr);
> }
>
> -/* Future patches will use this function */
> -__maybe_unused
> static void guc_parent_context_unpin(struct intel_context *ce)
> {
> struct intel_guc *guc = ce_to_guc(ce);
> @@ -2906,8 +2907,6 @@ static void guc_parent_context_unpin(struct intel_context *ce)
> lrc_unpin(ce);
> }
>
> -/* Future patches will use this function */
> -__maybe_unused
> static void guc_child_context_unpin(struct intel_context *ce)
> {
> GEM_BUG_ON(context_enabled(ce));
> @@ -2918,8 +2917,6 @@ static void guc_child_context_unpin(struct intel_context *ce)
> lrc_unpin(ce);
> }
>
> -/* Future patches will use this function */
> -__maybe_unused
> static void guc_child_context_post_unpin(struct intel_context *ce)
> {
> GEM_BUG_ON(!intel_context_is_child(ce));
> @@ -2930,6 +2927,98 @@ static void guc_child_context_post_unpin(struct intel_context *ce)
> intel_context_unpin(ce->parallel.parent);
> }
>
> +static void guc_child_context_destroy(struct kref *kref)
> +{
> + struct intel_context *ce = container_of(kref, typeof(*ce), ref);
> +
> + __guc_context_destroy(ce);
> +}
> +
> +static const struct intel_context_ops virtual_parent_context_ops = {
> + .alloc = guc_virtual_context_alloc,
> +
> + .pre_pin = guc_context_pre_pin,
> + .pin = guc_parent_context_pin,
> + .unpin = guc_parent_context_unpin,
> + .post_unpin = guc_context_post_unpin,
> +
> + .ban = guc_context_ban,
> +
> + .cancel_request = guc_context_cancel_request,
> +
> + .enter = guc_virtual_context_enter,
> + .exit = guc_virtual_context_exit,
> +
> + .sched_disable = guc_context_sched_disable,
> +
> + .destroy = guc_context_destroy,
> +
> + .get_sibling = guc_virtual_get_sibling,
> +};
> +
> +static const struct intel_context_ops virtual_child_context_ops = {
> + .alloc = guc_virtual_context_alloc,
> +
> + .pre_pin = guc_context_pre_pin,
> + .pin = guc_child_context_pin,
> + .unpin = guc_child_context_unpin,
> + .post_unpin = guc_child_context_post_unpin,
> +
> + .cancel_request = guc_context_cancel_request,
> +
> + .enter = guc_virtual_context_enter,
> + .exit = guc_virtual_context_exit,
> +
> + .destroy = guc_child_context_destroy,
> +
> + .get_sibling = guc_virtual_get_sibling,
> +};
> +
> +static struct intel_context *
> +guc_create_parallel(struct intel_engine_cs **engines,
> + unsigned int num_siblings,
> + unsigned int width)
> +{
> + struct intel_engine_cs **siblings = NULL;
> + struct intel_context *parent = NULL, *ce, *err;
> + int i, j;
> +
> + siblings = kmalloc_array(num_siblings,
> + sizeof(*siblings),
> + GFP_KERNEL);
> + if (!siblings)
> + return ERR_PTR(-ENOMEM);
> +
> + for (i = 0; i < width; ++i) {
> + for (j = 0; j < num_siblings; ++j)
> + siblings[j] = engines[i * num_siblings + j];
> +
> + ce = intel_engine_create_virtual(siblings, num_siblings,
> + FORCE_VIRTUAL);
> + if (!ce) {
> + err = ERR_PTR(-ENOMEM);
> + goto unwind;
> + }
> +
> + if (i == 0) {
> + parent = ce;
> + parent->ops = &virtual_parent_context_ops;
> + } else {
> + ce->ops = &virtual_child_context_ops;
> + intel_context_bind_parent_child(parent, ce);
> + }
> + }
> +
> + kfree(siblings);
> + return parent;
> +
> +unwind:
> + if (parent)
> + intel_context_put(parent);
> + kfree(siblings);
> + return err;
> +}
> +
> static bool
> guc_irq_enable_breadcrumbs(struct intel_breadcrumbs *b)
> {
> @@ -3756,7 +3845,8 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc,
> }
>
> static struct intel_context *
> -guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count)
> +guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
> + unsigned long flags)
> {
> struct guc_virtual_engine *ve;
> struct intel_guc *guc;
> diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
> index 0179f92e0916..c2a63e1584cb 100644
> --- a/include/uapi/drm/i915_drm.h
> +++ b/include/uapi/drm/i915_drm.h
> @@ -1824,6 +1824,7 @@ struct drm_i915_gem_context_param {
> * Extensions:
> * i915_context_engines_load_balance (I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE)
> * i915_context_engines_bond (I915_CONTEXT_ENGINES_EXT_BOND)
> + * i915_context_engines_parallel_submit (I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT)
> */
> #define I915_CONTEXT_PARAM_ENGINES 0xa
>
> @@ -2098,6 +2099,135 @@ struct i915_context_engines_bond {
> struct i915_engine_class_instance engines[N__]; \
> } __attribute__((packed)) name__
>
> +/**
> + * struct i915_context_engines_parallel_submit - Configure engine for
> + * parallel submission.
> + *
> + * Setup a slot in the context engine map to allow multiple BBs to be submitted
> + * in a single execbuf IOCTL. Those BBs will then be scheduled to run on the GPU
> + * in parallel. Multiple hardware contexts are created internally in the i915 to
> + * run these BBs. Once a slot is configured for N BBs only N BBs can be
> + * submitted in each execbuf IOCTL and this is implicit behavior e.g. The user
> + * doesn't tell the execbuf IOCTL there are N BBs, the execbuf IOCTL knows how
> + * many BBs there are based on the slot's configuration. The N BBs are the last
> + * N buffer objects or first N if I915_EXEC_BATCH_FIRST is set.
> + *
> + * The default placement behavior is to create implicit bonds between each
> + * context if each context maps to more than 1 physical engine (e.g. context is
> + * a virtual engine). Also we only allow contexts of same engine class and these
> + * contexts must be in logically contiguous order. Examples of the placement
> + * behavior are described below. Lastly, the default is to not allow BBs to be
> + * preempted mid-batch. Rather insert coordinated preemption points on all
> + * hardware contexts between each set of BBs. Flags could be added in the future
> + * to change both of these default behaviors.
> + *
> + * Returns -EINVAL if hardware context placement configuration is invalid or if
> + * the placement configuration isn't supported on the platform / submission
> + * interface.
> + * Returns -ENODEV if extension isn't supported on the platform / submission
> + * interface.
> + *
> + * .. code-block:: none
> + *
> + * Examples syntax:
> + * CS[X] = generic engine of same class, logical instance X
> + * INVALID = I915_ENGINE_CLASS_INVALID, I915_ENGINE_CLASS_INVALID_NONE
> + *
> + * Example 1 pseudo code:
> + * set_engines(INVALID)
> + * set_parallel(engine_index=0, width=2, num_siblings=1,
> + * engines=CS[0],CS[1])
> + *
> + * Results in the following valid placement:
> + * CS[0], CS[1]
> + *
> + * Example 2 pseudo code:
> + * set_engines(INVALID)
> + * set_parallel(engine_index=0, width=2, num_siblings=2,
> + * engines=CS[0],CS[2],CS[1],CS[3])
> + *
> + * Results in the following valid placements:
> + * CS[0], CS[1]
> + * CS[2], CS[3]
> + *
> + * This can be thought of as two virtual engines, each containing two
> + * engines thereby making a 2D array. However, there are bonds tying the
> + * entries together and placing restrictions on how they can be scheduled.
> + * Specifically, the scheduler can choose only vertical columns from the 2D
> + * array. That is, CS[0] is bonded to CS[1] and CS[2] to CS[3]. So if the
> + * scheduler wants to submit to CS[0], it must also choose CS[1] and vice
> + * versa. Same for CS[2] requires also using CS[3].
> + * VE[0] = CS[0], CS[2]
> + * VE[1] = CS[1], CS[3]
> + *
> + * Example 3 pseudo code:
> + * set_engines(INVALID)
> + * set_parallel(engine_index=0, width=2, num_siblings=2,
> + * engines=CS[0],CS[1],CS[1],CS[3])
> + *
> + * Results in the following valid and invalid placements:
> + * CS[0], CS[1]
> + * CS[1], CS[3] - Not logically contiguous, return -EINVAL
> + */
> +struct i915_context_engines_parallel_submit {
> + /**
> + * @base: base user extension.
> + */
> + struct i915_user_extension base;
> +
> + /**
> + * @engine_index: slot for parallel engine
> + */
> + __u16 engine_index;
> +
> + /**
> + * @width: number of contexts per parallel engine or in other words the
> + * number of batches in each submission
> + */
> + __u16 width;
> +
> + /**
> + * @num_siblings: number of siblings per context or in other words the
> + * number of possible placements for each submission
> + */
> + __u16 num_siblings;
> +
> + /**
> + * @mbz16: reserved for future use; must be zero
> + */
> + __u16 mbz16;
> +
> + /**
> + * @flags: all undefined flags must be zero, currently not defined flags
> + */
> + __u64 flags;
> +
> + /**
> + * @mbz64: reserved for future use; must be zero
> + */
> + __u64 mbz64[3];
> +
> + /**
> + * @engines: 2-d array of engine instances to configure parallel engine
> + *
> + * length = width (i) * num_siblings (j)
> + * index = j + i * num_siblings
> + */
> + struct i915_engine_class_instance engines[0];
> +
> +} __packed;
> +
> +#define I915_DEFINE_CONTEXT_ENGINES_PARALLEL_SUBMIT(name__, N__) struct { \
> + struct i915_user_extension base; \
> + __u16 engine_index; \
> + __u16 width; \
> + __u16 num_siblings; \
> + __u16 mbz16; \
> + __u64 flags; \
> + __u64 mbz64[3]; \
> + struct i915_engine_class_instance engines[N__]; \
> +} __attribute__((packed)) name__
> +
> /**
> * DOC: Context Engine Map uAPI
> *
> @@ -2157,6 +2287,7 @@ struct i915_context_param_engines {
> __u64 extensions; /* linked chain of extension blocks, 0 terminates */
> #define I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE 0 /* see i915_context_engines_load_balance */
> #define I915_CONTEXT_ENGINES_EXT_BOND 1 /* see i915_context_engines_bond */
> +#define I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT 2 /* see i915_context_engines_parallel_submit */
> struct i915_engine_class_instance engines[0];
> } __attribute__((packed));
>
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 16/25] drm/i915/guc: Connect UAPI to GuC multi-lrc interface
@ 2021-10-14 18:24 ` John Harrison
0 siblings, 0 replies; 69+ messages in thread
From: John Harrison @ 2021-10-14 18:24 UTC (permalink / raw)
To: Matthew Brost, intel-gfx, dri-devel
On 10/14/2021 10:19, Matthew Brost wrote:
> Introduce 'set parallel submit' extension to connect UAPI to GuC
> multi-lrc interface. Kernel doc in new uAPI should explain it all.
>
> IGT: https://patchwork.freedesktop.org/patch/447008/?series=93071&rev=1
> media UMD: https://github.com/intel/media-driver/pull/1252
>
> v2:
> (Daniel Vetter)
> - Add IGT link and placeholder for media UMD link
> v3:
> (Kernel test robot)
> - Fix warning in unpin engines call
> (John Harrison)
> - Reword a bunch of the kernel doc
> v4:
> (John Harrison)
> - Add comment why perma-pin is done after setting gem context
> - Update some comments / docs for proto contexts
> v5:
> (John Harrison)
> - Rework perma-pin comment
> - Add BUG_IN if context is pinned when setting gem context
IN?
>
> Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/gem/i915_gem_context.c | 230 +++++++++++++++++-
> .../gpu/drm/i915/gem/i915_gem_context_types.h | 16 +-
> drivers/gpu/drm/i915/gt/intel_context_types.h | 9 +-
> drivers/gpu/drm/i915/gt/intel_engine.h | 12 +-
> drivers/gpu/drm/i915/gt/intel_engine_cs.c | 6 +-
> .../drm/i915/gt/intel_execlists_submission.c | 6 +-
> drivers/gpu/drm/i915/gt/selftest_execlists.c | 12 +-
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 114 ++++++++-
> include/uapi/drm/i915_drm.h | 131 ++++++++++
> 9 files changed, 505 insertions(+), 31 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> index d225d3dd0b40..9a00f11fef46 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> @@ -556,9 +556,150 @@ set_proto_ctx_engines_bond(struct i915_user_extension __user *base, void *data)
> return 0;
> }
>
> +static int
> +set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
> + void *data)
> +{
> + struct i915_context_engines_parallel_submit __user *ext =
> + container_of_user(base, typeof(*ext), base);
> + const struct set_proto_ctx_engines *set = data;
> + struct drm_i915_private *i915 = set->i915;
> + u64 flags;
> + int err = 0, n, i, j;
> + u16 slot, width, num_siblings;
> + struct intel_engine_cs **siblings = NULL;
> + intel_engine_mask_t prev_mask;
> +
> + /* Disabling for now */
> + return -ENODEV;
> +
> + /* FIXME: This is NIY for execlists */
> + if (!(intel_uc_uses_guc_submission(&i915->gt.uc)))
> + return -ENODEV;
> +
> + if (get_user(slot, &ext->engine_index))
> + return -EFAULT;
> +
> + if (get_user(width, &ext->width))
> + return -EFAULT;
> +
> + if (get_user(num_siblings, &ext->num_siblings))
> + return -EFAULT;
> +
> + if (slot >= set->num_engines) {
> + drm_dbg(&i915->drm, "Invalid placement value, %d >= %d\n",
> + slot, set->num_engines);
> + return -EINVAL;
> + }
> +
> + if (set->engines[slot].type != I915_GEM_ENGINE_TYPE_INVALID) {
> + drm_dbg(&i915->drm,
> + "Invalid placement[%d], already occupied\n", slot);
> + return -EINVAL;
> + }
> +
> + if (get_user(flags, &ext->flags))
> + return -EFAULT;
> +
> + if (flags) {
> + drm_dbg(&i915->drm, "Unknown flags 0x%02llx", flags);
> + return -EINVAL;
> + }
> +
> + for (n = 0; n < ARRAY_SIZE(ext->mbz64); n++) {
> + err = check_user_mbz(&ext->mbz64[n]);
> + if (err)
> + return err;
> + }
> +
> + if (width < 2) {
> + drm_dbg(&i915->drm, "Width (%d) < 2\n", width);
> + return -EINVAL;
> + }
> +
> + if (num_siblings < 1) {
> + drm_dbg(&i915->drm, "Number siblings (%d) < 1\n",
> + num_siblings);
> + return -EINVAL;
> + }
> +
> + siblings = kmalloc_array(num_siblings * width,
> + sizeof(*siblings),
> + GFP_KERNEL);
> + if (!siblings)
> + return -ENOMEM;
> +
> + /* Create contexts / engines */
> + for (i = 0; i < width; ++i) {
> + intel_engine_mask_t current_mask = 0;
> + struct i915_engine_class_instance prev_engine;
> +
> + for (j = 0; j < num_siblings; ++j) {
> + struct i915_engine_class_instance ci;
> +
> + n = i * num_siblings + j;
> + if (copy_from_user(&ci, &ext->engines[n], sizeof(ci))) {
> + err = -EFAULT;
> + goto out_err;
> + }
> +
> + siblings[n] =
> + intel_engine_lookup_user(i915, ci.engine_class,
> + ci.engine_instance);
> + if (!siblings[n]) {
> + drm_dbg(&i915->drm,
> + "Invalid sibling[%d]: { class:%d, inst:%d }\n",
> + n, ci.engine_class, ci.engine_instance);
> + err = -EINVAL;
> + goto out_err;
> + }
> +
> + if (n) {
> + if (prev_engine.engine_class !=
> + ci.engine_class) {
> + drm_dbg(&i915->drm,
> + "Mismatched class %d, %d\n",
> + prev_engine.engine_class,
> + ci.engine_class);
> + err = -EINVAL;
> + goto out_err;
> + }
> + }
> +
> + prev_engine = ci;
> + current_mask |= siblings[n]->logical_mask;
> + }
> +
> + if (i > 0) {
> + if (current_mask != prev_mask << 1) {
> + drm_dbg(&i915->drm,
> + "Non contiguous logical mask 0x%x, 0x%x\n",
> + prev_mask, current_mask);
> + err = -EINVAL;
> + goto out_err;
> + }
> + }
> + prev_mask = current_mask;
> + }
> +
> + set->engines[slot].type = I915_GEM_ENGINE_TYPE_PARALLEL;
> + set->engines[slot].num_siblings = num_siblings;
> + set->engines[slot].width = width;
> + set->engines[slot].siblings = siblings;
> +
> + return 0;
> +
> +out_err:
> + kfree(siblings);
> +
> + return err;
> +}
> +
> static const i915_user_extension_fn set_proto_ctx_engines_extensions[] = {
> [I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE] = set_proto_ctx_engines_balance,
> [I915_CONTEXT_ENGINES_EXT_BOND] = set_proto_ctx_engines_bond,
> + [I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT] =
> + set_proto_ctx_engines_parallel_submit,
> };
>
> static int set_proto_ctx_engines(struct drm_i915_file_private *fpriv,
> @@ -794,6 +935,7 @@ static int intel_context_set_gem(struct intel_context *ce,
> GEM_BUG_ON(rcu_access_pointer(ce->gem_context));
> RCU_INIT_POINTER(ce->gem_context, ctx);
>
> + GEM_BUG_ON(intel_context_is_pinned(ce));
> ce->ring_size = SZ_16K;
>
> i915_vm_put(ce->vm);
> @@ -818,6 +960,25 @@ static int intel_context_set_gem(struct intel_context *ce,
> return ret;
> }
>
> +static void __unpin_engines(struct i915_gem_engines *e, unsigned int count)
> +{
> + while (count--) {
> + struct intel_context *ce = e->engines[count], *child;
> +
> + if (!ce || !test_bit(CONTEXT_PERMA_PIN, &ce->flags))
> + continue;
> +
> + for_each_child(ce, child)
> + intel_context_unpin(child);
> + intel_context_unpin(ce);
> + }
> +}
> +
> +static void unpin_engines(struct i915_gem_engines *e)
> +{
> + __unpin_engines(e, e->num_engines);
> +}
> +
> static void __free_engines(struct i915_gem_engines *e, unsigned int count)
> {
> while (count--) {
> @@ -933,6 +1094,40 @@ static struct i915_gem_engines *default_engines(struct i915_gem_context *ctx,
> return err;
> }
>
> +static int perma_pin_contexts(struct intel_context *ce)
> +{
> + struct intel_context *child;
> + int i = 0, j = 0, ret;
> +
> + GEM_BUG_ON(!intel_context_is_parent(ce));
> +
> + ret = intel_context_pin(ce);
> + if (unlikely(ret))
> + return ret;
> +
> + for_each_child(ce, child) {
> + ret = intel_context_pin(child);
> + if (unlikely(ret))
> + goto unwind;
> + ++i;
> + }
> +
> + set_bit(CONTEXT_PERMA_PIN, &ce->flags);
> +
> + return 0;
> +
> +unwind:
> + intel_context_unpin(ce);
> + for_each_child(ce, child) {
> + if (j++ < i)
> + intel_context_unpin(child);
> + else
> + break;
> + }
> +
> + return ret;
> +}
> +
> static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
> unsigned int num_engines,
> struct i915_gem_proto_engine *pe)
> @@ -946,7 +1141,7 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
> e->num_engines = num_engines;
>
> for (n = 0; n < num_engines; n++) {
> - struct intel_context *ce;
> + struct intel_context *ce, *child;
> int ret;
>
> switch (pe[n].type) {
> @@ -956,7 +1151,13 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
>
> case I915_GEM_ENGINE_TYPE_BALANCED:
> ce = intel_engine_create_virtual(pe[n].siblings,
> - pe[n].num_siblings);
> + pe[n].num_siblings, 0);
> + break;
> +
> + case I915_GEM_ENGINE_TYPE_PARALLEL:
> + ce = intel_engine_create_parallel(pe[n].siblings,
> + pe[n].num_siblings,
> + pe[n].width);
> break;
>
> case I915_GEM_ENGINE_TYPE_INVALID:
> @@ -977,6 +1178,30 @@ static struct i915_gem_engines *user_engines(struct i915_gem_context *ctx,
> err = ERR_PTR(ret);
> goto free_engines;
> }
> + for_each_child(ce, child) {
> + ret = intel_context_set_gem(child, ctx, pe->sseu);
> + if (ret) {
> + err = ERR_PTR(ret);
> + goto free_engines;
> + }
> + }
> +
> + /*
> + * XXX: Must be done after calling intel_context_set_gem as that
> + * function changes the ring size. The ring is allocated when
> + * the context is pinned. If the ring size is changed after
> + * allocation we have a mismatch of the ring size and will cause
> + * the context to hang. Presumably with a bit of reordering we
> + * could move the perma-pin step to the backend function
> + * intel_engine_create_parallel.
> + */
> + if (pe[n].type == I915_GEM_ENGINE_TYPE_PARALLEL) {
> + ret = perma_pin_contexts(ce);
> + if (ret) {
> + err = ERR_PTR(ret);
> + goto free_engines;
> + }
> + }
> }
>
> return e;
> @@ -1219,6 +1444,7 @@ static void context_close(struct i915_gem_context *ctx)
>
> /* Flush any concurrent set_engines() */
> mutex_lock(&ctx->engines_mutex);
> + unpin_engines(__context_engines_static(ctx));
> engines_idle_release(ctx, rcu_replace_pointer(ctx->engines, NULL, 1));
> i915_gem_context_set_closed(ctx);
> mutex_unlock(&ctx->engines_mutex);
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h
> index a627b09c4680..282cdb8a5c5a 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_context_types.h
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_context_types.h
> @@ -78,13 +78,16 @@ enum i915_gem_engine_type {
>
> /** @I915_GEM_ENGINE_TYPE_BALANCED: A load-balanced engine set */
> I915_GEM_ENGINE_TYPE_BALANCED,
> +
> + /** @I915_GEM_ENGINE_TYPE_PARALLEL: A parallel engine set */
> + I915_GEM_ENGINE_TYPE_PARALLEL,
> };
>
> /**
> * struct i915_gem_proto_engine - prototype engine
> *
> * This struct describes an engine that a context may contain. Engines
> - * have three types:
> + * have four types:
> *
> * - I915_GEM_ENGINE_TYPE_INVALID: Invalid engines can be created but they
> * show up as a NULL in i915_gem_engines::engines[i] and any attempt to
> @@ -97,6 +100,10 @@ enum i915_gem_engine_type {
> *
> * - I915_GEM_ENGINE_TYPE_BALANCED: A load-balanced engine set, described
> * i915_gem_proto_engine::num_siblings and i915_gem_proto_engine::siblings.
> + *
> + * - I915_GEM_ENGINE_TYPE_PARALLEL: A parallel submission engine set, described
> + * i915_gem_proto_engine::width, i915_gem_proto_engine::num_siblings, and
> + * i915_gem_proto_engine::siblings.
> */
> struct i915_gem_proto_engine {
> /** @type: Type of this engine */
> @@ -105,10 +112,13 @@ struct i915_gem_proto_engine {
> /** @engine: Engine, for physical */
> struct intel_engine_cs *engine;
>
> - /** @num_siblings: Number of balanced siblings */
> + /** @num_siblings: Number of balanced or parallel siblings */
> unsigned int num_siblings;
>
> - /** @siblings: Balanced siblings */
> + /** @width: Width of each sibling */
> + unsigned int width;
> +
> + /** @siblings: Balanced siblings or num_siblings * width for parallel */
> struct intel_engine_cs **siblings;
>
> /** @sseu: Client-set SSEU parameters */
> diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
> index 8309d1141d0a..1d880303a7e4 100644
> --- a/drivers/gpu/drm/i915/gt/intel_context_types.h
> +++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
> @@ -55,9 +55,13 @@ struct intel_context_ops {
> void (*reset)(struct intel_context *ce);
> void (*destroy)(struct kref *kref);
>
> - /* virtual engine/context interface */
> + /* virtual/parallel engine/context interface */
> struct intel_context *(*create_virtual)(struct intel_engine_cs **engine,
> - unsigned int count);
> + unsigned int count,
> + unsigned long flags);
> + struct intel_context *(*create_parallel)(struct intel_engine_cs **engines,
> + unsigned int num_siblings,
> + unsigned int width);
> struct intel_engine_cs *(*get_sibling)(struct intel_engine_cs *engine,
> unsigned int sibling);
> };
> @@ -113,6 +117,7 @@ struct intel_context {
> #define CONTEXT_NOPREEMPT 8
> #define CONTEXT_LRCA_DIRTY 9
> #define CONTEXT_GUC_INIT 10
> +#define CONTEXT_PERMA_PIN 11
>
> struct {
> u64 timeout_us;
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine.h b/drivers/gpu/drm/i915/gt/intel_engine.h
> index d5ac49c0691e..08559ace0ada 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine.h
> +++ b/drivers/gpu/drm/i915/gt/intel_engine.h
> @@ -282,9 +282,19 @@ intel_engine_has_preempt_reset(const struct intel_engine_cs *engine)
> return intel_engine_has_preemption(engine);
> }
>
> +#define FORCE_VIRTUAL BIT(0)
> struct intel_context *
> intel_engine_create_virtual(struct intel_engine_cs **siblings,
> - unsigned int count);
> + unsigned int count, unsigned long flags);
> +
> +static inline struct intel_context *
> +intel_engine_create_parallel(struct intel_engine_cs **engines,
> + unsigned int num_engines,
> + unsigned int width)
> +{
> + GEM_BUG_ON(!engines[0]->cops->create_parallel);
> + return engines[0]->cops->create_parallel(engines, num_engines, width);
> +}
>
> static inline bool
> intel_virtual_engine_has_heartbeat(const struct intel_engine_cs *engine)
> diff --git a/drivers/gpu/drm/i915/gt/intel_engine_cs.c b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> index 2eb798ad068b..ff6753ccb129 100644
> --- a/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> +++ b/drivers/gpu/drm/i915/gt/intel_engine_cs.c
> @@ -1953,16 +1953,16 @@ ktime_t intel_engine_get_busy_time(struct intel_engine_cs *engine, ktime_t *now)
>
> struct intel_context *
> intel_engine_create_virtual(struct intel_engine_cs **siblings,
> - unsigned int count)
> + unsigned int count, unsigned long flags)
> {
> if (count == 0)
> return ERR_PTR(-EINVAL);
>
> - if (count == 1)
> + if (count == 1 && !(flags & FORCE_VIRTUAL))
> return intel_context_create(siblings[0]);
>
> GEM_BUG_ON(!siblings[0]->cops->create_virtual);
> - return siblings[0]->cops->create_virtual(siblings, count);
> + return siblings[0]->cops->create_virtual(siblings, count, flags);
> }
>
> struct i915_request *
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index 43a74b216efb..bedb80057046 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -201,7 +201,8 @@ static struct virtual_engine *to_virtual_engine(struct intel_engine_cs *engine)
> }
>
> static struct intel_context *
> -execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count);
> +execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
> + unsigned long flags);
>
> static struct i915_request *
> __active_request(const struct intel_timeline * const tl,
> @@ -3784,7 +3785,8 @@ static void virtual_submit_request(struct i915_request *rq)
> }
>
> static struct intel_context *
> -execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count)
> +execlists_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
> + unsigned long flags)
> {
> struct virtual_engine *ve;
> unsigned int n;
> diff --git a/drivers/gpu/drm/i915/gt/selftest_execlists.c b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> index 25a8c4f62b0d..b367ecfa42de 100644
> --- a/drivers/gpu/drm/i915/gt/selftest_execlists.c
> +++ b/drivers/gpu/drm/i915/gt/selftest_execlists.c
> @@ -3733,7 +3733,7 @@ static int nop_virtual_engine(struct intel_gt *gt,
> GEM_BUG_ON(!nctx || nctx > ARRAY_SIZE(ve));
>
> for (n = 0; n < nctx; n++) {
> - ve[n] = intel_engine_create_virtual(siblings, nsibling);
> + ve[n] = intel_engine_create_virtual(siblings, nsibling, 0);
> if (IS_ERR(ve[n])) {
> err = PTR_ERR(ve[n]);
> nctx = n;
> @@ -3929,7 +3929,7 @@ static int mask_virtual_engine(struct intel_gt *gt,
> * restrict it to our desired engine within the virtual engine.
> */
>
> - ve = intel_engine_create_virtual(siblings, nsibling);
> + ve = intel_engine_create_virtual(siblings, nsibling, 0);
> if (IS_ERR(ve)) {
> err = PTR_ERR(ve);
> goto out_close;
> @@ -4060,7 +4060,7 @@ static int slicein_virtual_engine(struct intel_gt *gt,
> i915_request_add(rq);
> }
>
> - ce = intel_engine_create_virtual(siblings, nsibling);
> + ce = intel_engine_create_virtual(siblings, nsibling, 0);
> if (IS_ERR(ce)) {
> err = PTR_ERR(ce);
> goto out;
> @@ -4112,7 +4112,7 @@ static int sliceout_virtual_engine(struct intel_gt *gt,
>
> /* XXX We do not handle oversubscription and fairness with normal rq */
> for (n = 0; n < nsibling; n++) {
> - ce = intel_engine_create_virtual(siblings, nsibling);
> + ce = intel_engine_create_virtual(siblings, nsibling, 0);
> if (IS_ERR(ce)) {
> err = PTR_ERR(ce);
> goto out;
> @@ -4214,7 +4214,7 @@ static int preserved_virtual_engine(struct intel_gt *gt,
> if (err)
> goto out_scratch;
>
> - ve = intel_engine_create_virtual(siblings, nsibling);
> + ve = intel_engine_create_virtual(siblings, nsibling, 0);
> if (IS_ERR(ve)) {
> err = PTR_ERR(ve);
> goto out_scratch;
> @@ -4354,7 +4354,7 @@ static int reset_virtual_engine(struct intel_gt *gt,
> if (igt_spinner_init(&spin, gt))
> return -ENOMEM;
>
> - ve = intel_engine_create_virtual(siblings, nsibling);
> + ve = intel_engine_create_virtual(siblings, nsibling, 0);
> if (IS_ERR(ve)) {
> err = PTR_ERR(ve);
> goto out_spin;
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 938dc34e8d3a..7c12364a017a 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -124,7 +124,13 @@ struct guc_virtual_engine {
> };
>
> static struct intel_context *
> -guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count);
> +guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
> + unsigned long flags);
> +
> +static struct intel_context *
> +guc_create_parallel(struct intel_engine_cs **engines,
> + unsigned int num_siblings,
> + unsigned int width);
>
> #define GUC_REQUEST_SIZE 64 /* bytes */
>
> @@ -2609,6 +2615,7 @@ static const struct intel_context_ops guc_context_ops = {
> .destroy = guc_context_destroy,
>
> .create_virtual = guc_create_virtual,
> + .create_parallel = guc_create_parallel,
> };
>
> static void submit_work_cb(struct irq_work *wrk)
> @@ -2858,8 +2865,6 @@ static const struct intel_context_ops virtual_guc_context_ops = {
> .get_sibling = guc_virtual_get_sibling,
> };
>
> -/* Future patches will use this function */
> -__maybe_unused
> static int guc_parent_context_pin(struct intel_context *ce, void *vaddr)
> {
> struct intel_engine_cs *engine = guc_virtual_get_sibling(ce->engine, 0);
> @@ -2876,8 +2881,6 @@ static int guc_parent_context_pin(struct intel_context *ce, void *vaddr)
> return __guc_context_pin(ce, engine, vaddr);
> }
>
> -/* Future patches will use this function */
> -__maybe_unused
> static int guc_child_context_pin(struct intel_context *ce, void *vaddr)
> {
> struct intel_engine_cs *engine = guc_virtual_get_sibling(ce->engine, 0);
> @@ -2889,8 +2892,6 @@ static int guc_child_context_pin(struct intel_context *ce, void *vaddr)
> return __guc_context_pin(ce, engine, vaddr);
> }
>
> -/* Future patches will use this function */
> -__maybe_unused
> static void guc_parent_context_unpin(struct intel_context *ce)
> {
> struct intel_guc *guc = ce_to_guc(ce);
> @@ -2906,8 +2907,6 @@ static void guc_parent_context_unpin(struct intel_context *ce)
> lrc_unpin(ce);
> }
>
> -/* Future patches will use this function */
> -__maybe_unused
> static void guc_child_context_unpin(struct intel_context *ce)
> {
> GEM_BUG_ON(context_enabled(ce));
> @@ -2918,8 +2917,6 @@ static void guc_child_context_unpin(struct intel_context *ce)
> lrc_unpin(ce);
> }
>
> -/* Future patches will use this function */
> -__maybe_unused
> static void guc_child_context_post_unpin(struct intel_context *ce)
> {
> GEM_BUG_ON(!intel_context_is_child(ce));
> @@ -2930,6 +2927,98 @@ static void guc_child_context_post_unpin(struct intel_context *ce)
> intel_context_unpin(ce->parallel.parent);
> }
>
> +static void guc_child_context_destroy(struct kref *kref)
> +{
> + struct intel_context *ce = container_of(kref, typeof(*ce), ref);
> +
> + __guc_context_destroy(ce);
> +}
> +
> +static const struct intel_context_ops virtual_parent_context_ops = {
> + .alloc = guc_virtual_context_alloc,
> +
> + .pre_pin = guc_context_pre_pin,
> + .pin = guc_parent_context_pin,
> + .unpin = guc_parent_context_unpin,
> + .post_unpin = guc_context_post_unpin,
> +
> + .ban = guc_context_ban,
> +
> + .cancel_request = guc_context_cancel_request,
> +
> + .enter = guc_virtual_context_enter,
> + .exit = guc_virtual_context_exit,
> +
> + .sched_disable = guc_context_sched_disable,
> +
> + .destroy = guc_context_destroy,
> +
> + .get_sibling = guc_virtual_get_sibling,
> +};
> +
> +static const struct intel_context_ops virtual_child_context_ops = {
> + .alloc = guc_virtual_context_alloc,
> +
> + .pre_pin = guc_context_pre_pin,
> + .pin = guc_child_context_pin,
> + .unpin = guc_child_context_unpin,
> + .post_unpin = guc_child_context_post_unpin,
> +
> + .cancel_request = guc_context_cancel_request,
> +
> + .enter = guc_virtual_context_enter,
> + .exit = guc_virtual_context_exit,
> +
> + .destroy = guc_child_context_destroy,
> +
> + .get_sibling = guc_virtual_get_sibling,
> +};
> +
> +static struct intel_context *
> +guc_create_parallel(struct intel_engine_cs **engines,
> + unsigned int num_siblings,
> + unsigned int width)
> +{
> + struct intel_engine_cs **siblings = NULL;
> + struct intel_context *parent = NULL, *ce, *err;
> + int i, j;
> +
> + siblings = kmalloc_array(num_siblings,
> + sizeof(*siblings),
> + GFP_KERNEL);
> + if (!siblings)
> + return ERR_PTR(-ENOMEM);
> +
> + for (i = 0; i < width; ++i) {
> + for (j = 0; j < num_siblings; ++j)
> + siblings[j] = engines[i * num_siblings + j];
> +
> + ce = intel_engine_create_virtual(siblings, num_siblings,
> + FORCE_VIRTUAL);
> + if (!ce) {
> + err = ERR_PTR(-ENOMEM);
> + goto unwind;
> + }
> +
> + if (i == 0) {
> + parent = ce;
> + parent->ops = &virtual_parent_context_ops;
> + } else {
> + ce->ops = &virtual_child_context_ops;
> + intel_context_bind_parent_child(parent, ce);
> + }
> + }
> +
> + kfree(siblings);
> + return parent;
> +
> +unwind:
> + if (parent)
> + intel_context_put(parent);
> + kfree(siblings);
> + return err;
> +}
> +
> static bool
> guc_irq_enable_breadcrumbs(struct intel_breadcrumbs *b)
> {
> @@ -3756,7 +3845,8 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc,
> }
>
> static struct intel_context *
> -guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count)
> +guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
> + unsigned long flags)
> {
> struct guc_virtual_engine *ve;
> struct intel_guc *guc;
> diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
> index 0179f92e0916..c2a63e1584cb 100644
> --- a/include/uapi/drm/i915_drm.h
> +++ b/include/uapi/drm/i915_drm.h
> @@ -1824,6 +1824,7 @@ struct drm_i915_gem_context_param {
> * Extensions:
> * i915_context_engines_load_balance (I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE)
> * i915_context_engines_bond (I915_CONTEXT_ENGINES_EXT_BOND)
> + * i915_context_engines_parallel_submit (I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT)
> */
> #define I915_CONTEXT_PARAM_ENGINES 0xa
>
> @@ -2098,6 +2099,135 @@ struct i915_context_engines_bond {
> struct i915_engine_class_instance engines[N__]; \
> } __attribute__((packed)) name__
>
> +/**
> + * struct i915_context_engines_parallel_submit - Configure engine for
> + * parallel submission.
> + *
> + * Setup a slot in the context engine map to allow multiple BBs to be submitted
> + * in a single execbuf IOCTL. Those BBs will then be scheduled to run on the GPU
> + * in parallel. Multiple hardware contexts are created internally in the i915 to
> + * run these BBs. Once a slot is configured for N BBs only N BBs can be
> + * submitted in each execbuf IOCTL and this is implicit behavior e.g. The user
> + * doesn't tell the execbuf IOCTL there are N BBs, the execbuf IOCTL knows how
> + * many BBs there are based on the slot's configuration. The N BBs are the last
> + * N buffer objects or first N if I915_EXEC_BATCH_FIRST is set.
> + *
> + * The default placement behavior is to create implicit bonds between each
> + * context if each context maps to more than 1 physical engine (e.g. context is
> + * a virtual engine). Also we only allow contexts of same engine class and these
> + * contexts must be in logically contiguous order. Examples of the placement
> + * behavior are described below. Lastly, the default is to not allow BBs to be
> + * preempted mid-batch. Rather insert coordinated preemption points on all
> + * hardware contexts between each set of BBs. Flags could be added in the future
> + * to change both of these default behaviors.
> + *
> + * Returns -EINVAL if hardware context placement configuration is invalid or if
> + * the placement configuration isn't supported on the platform / submission
> + * interface.
> + * Returns -ENODEV if extension isn't supported on the platform / submission
> + * interface.
> + *
> + * .. code-block:: none
> + *
> + * Examples syntax:
> + * CS[X] = generic engine of same class, logical instance X
> + * INVALID = I915_ENGINE_CLASS_INVALID, I915_ENGINE_CLASS_INVALID_NONE
> + *
> + * Example 1 pseudo code:
> + * set_engines(INVALID)
> + * set_parallel(engine_index=0, width=2, num_siblings=1,
> + * engines=CS[0],CS[1])
> + *
> + * Results in the following valid placement:
> + * CS[0], CS[1]
> + *
> + * Example 2 pseudo code:
> + * set_engines(INVALID)
> + * set_parallel(engine_index=0, width=2, num_siblings=2,
> + * engines=CS[0],CS[2],CS[1],CS[3])
> + *
> + * Results in the following valid placements:
> + * CS[0], CS[1]
> + * CS[2], CS[3]
> + *
> + * This can be thought of as two virtual engines, each containing two
> + * engines thereby making a 2D array. However, there are bonds tying the
> + * entries together and placing restrictions on how they can be scheduled.
> + * Specifically, the scheduler can choose only vertical columns from the 2D
> + * array. That is, CS[0] is bonded to CS[1] and CS[2] to CS[3]. So if the
> + * scheduler wants to submit to CS[0], it must also choose CS[1] and vice
> + * versa. Same for CS[2] requires also using CS[3].
> + * VE[0] = CS[0], CS[2]
> + * VE[1] = CS[1], CS[3]
> + *
> + * Example 3 pseudo code:
> + * set_engines(INVALID)
> + * set_parallel(engine_index=0, width=2, num_siblings=2,
> + * engines=CS[0],CS[1],CS[1],CS[3])
> + *
> + * Results in the following valid and invalid placements:
> + * CS[0], CS[1]
> + * CS[1], CS[3] - Not logically contiguous, return -EINVAL
> + */
> +struct i915_context_engines_parallel_submit {
> + /**
> + * @base: base user extension.
> + */
> + struct i915_user_extension base;
> +
> + /**
> + * @engine_index: slot for parallel engine
> + */
> + __u16 engine_index;
> +
> + /**
> + * @width: number of contexts per parallel engine or in other words the
> + * number of batches in each submission
> + */
> + __u16 width;
> +
> + /**
> + * @num_siblings: number of siblings per context or in other words the
> + * number of possible placements for each submission
> + */
> + __u16 num_siblings;
> +
> + /**
> + * @mbz16: reserved for future use; must be zero
> + */
> + __u16 mbz16;
> +
> + /**
> + * @flags: all undefined flags must be zero, currently not defined flags
> + */
> + __u64 flags;
> +
> + /**
> + * @mbz64: reserved for future use; must be zero
> + */
> + __u64 mbz64[3];
> +
> + /**
> + * @engines: 2-d array of engine instances to configure parallel engine
> + *
> + * length = width (i) * num_siblings (j)
> + * index = j + i * num_siblings
> + */
> + struct i915_engine_class_instance engines[0];
> +
> +} __packed;
> +
> +#define I915_DEFINE_CONTEXT_ENGINES_PARALLEL_SUBMIT(name__, N__) struct { \
> + struct i915_user_extension base; \
> + __u16 engine_index; \
> + __u16 width; \
> + __u16 num_siblings; \
> + __u16 mbz16; \
> + __u64 flags; \
> + __u64 mbz64[3]; \
> + struct i915_engine_class_instance engines[N__]; \
> +} __attribute__((packed)) name__
> +
> /**
> * DOC: Context Engine Map uAPI
> *
> @@ -2157,6 +2287,7 @@ struct i915_context_param_engines {
> __u64 extensions; /* linked chain of extension blocks, 0 terminates */
> #define I915_CONTEXT_ENGINES_EXT_LOAD_BALANCE 0 /* see i915_context_engines_load_balance */
> #define I915_CONTEXT_ENGINES_EXT_BOND 1 /* see i915_context_engines_bond */
> +#define I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT 2 /* see i915_context_engines_parallel_submit */
> struct i915_engine_class_instance engines[0];
> } __attribute__((packed));
>
^ permalink raw reply [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 17/25] drm/i915/doc: Update parallel submit doc to point to i915_drm.h
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Update parallel submit doc to point to i915_drm.h
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
Documentation/gpu/rfc/i915_parallel_execbuf.h | 122 ------------------
Documentation/gpu/rfc/i915_scheduler.rst | 4 +-
2 files changed, 2 insertions(+), 124 deletions(-)
delete mode 100644 Documentation/gpu/rfc/i915_parallel_execbuf.h
diff --git a/Documentation/gpu/rfc/i915_parallel_execbuf.h b/Documentation/gpu/rfc/i915_parallel_execbuf.h
deleted file mode 100644
index 8cbe2c4e0172..000000000000
--- a/Documentation/gpu/rfc/i915_parallel_execbuf.h
+++ /dev/null
@@ -1,122 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-/*
- * Copyright © 2021 Intel Corporation
- */
-
-#define I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT 2 /* see i915_context_engines_parallel_submit */
-
-/**
- * struct drm_i915_context_engines_parallel_submit - Configure engine for
- * parallel submission.
- *
- * Setup a slot in the context engine map to allow multiple BBs to be submitted
- * in a single execbuf IOCTL. Those BBs will then be scheduled to run on the GPU
- * in parallel. Multiple hardware contexts are created internally in the i915
- * run these BBs. Once a slot is configured for N BBs only N BBs can be
- * submitted in each execbuf IOCTL and this is implicit behavior e.g. The user
- * doesn't tell the execbuf IOCTL there are N BBs, the execbuf IOCTL knows how
- * many BBs there are based on the slot's configuration. The N BBs are the last
- * N buffer objects or first N if I915_EXEC_BATCH_FIRST is set.
- *
- * The default placement behavior is to create implicit bonds between each
- * context if each context maps to more than 1 physical engine (e.g. context is
- * a virtual engine). Also we only allow contexts of same engine class and these
- * contexts must be in logically contiguous order. Examples of the placement
- * behavior described below. Lastly, the default is to not allow BBs to
- * preempted mid BB rather insert coordinated preemption on all hardware
- * contexts between each set of BBs. Flags may be added in the future to change
- * both of these default behaviors.
- *
- * Returns -EINVAL if hardware context placement configuration is invalid or if
- * the placement configuration isn't supported on the platform / submission
- * interface.
- * Returns -ENODEV if extension isn't supported on the platform / submission
- * interface.
- *
- * .. code-block:: none
- *
- * Example 1 pseudo code:
- * CS[X] = generic engine of same class, logical instance X
- * INVALID = I915_ENGINE_CLASS_INVALID, I915_ENGINE_CLASS_INVALID_NONE
- * set_engines(INVALID)
- * set_parallel(engine_index=0, width=2, num_siblings=1,
- * engines=CS[0],CS[1])
- *
- * Results in the following valid placement:
- * CS[0], CS[1]
- *
- * Example 2 pseudo code:
- * CS[X] = generic engine of same class, logical instance X
- * INVALID = I915_ENGINE_CLASS_INVALID, I915_ENGINE_CLASS_INVALID_NONE
- * set_engines(INVALID)
- * set_parallel(engine_index=0, width=2, num_siblings=2,
- * engines=CS[0],CS[2],CS[1],CS[3])
- *
- * Results in the following valid placements:
- * CS[0], CS[1]
- * CS[2], CS[3]
- *
- * This can also be thought of as 2 virtual engines described by 2-D array
- * in the engines the field with bonds placed between each index of the
- * virtual engines. e.g. CS[0] is bonded to CS[1], CS[2] is bonded to
- * CS[3].
- * VE[0] = CS[0], CS[2]
- * VE[1] = CS[1], CS[3]
- *
- * Example 3 pseudo code:
- * CS[X] = generic engine of same class, logical instance X
- * INVALID = I915_ENGINE_CLASS_INVALID, I915_ENGINE_CLASS_INVALID_NONE
- * set_engines(INVALID)
- * set_parallel(engine_index=0, width=2, num_siblings=2,
- * engines=CS[0],CS[1],CS[1],CS[3])
- *
- * Results in the following valid and invalid placements:
- * CS[0], CS[1]
- * CS[1], CS[3] - Not logical contiguous, return -EINVAL
- */
-struct drm_i915_context_engines_parallel_submit {
- /**
- * @base: base user extension.
- */
- struct i915_user_extension base;
-
- /**
- * @engine_index: slot for parallel engine
- */
- __u16 engine_index;
-
- /**
- * @width: number of contexts per parallel engine
- */
- __u16 width;
-
- /**
- * @num_siblings: number of siblings per context
- */
- __u16 num_siblings;
-
- /**
- * @mbz16: reserved for future use; must be zero
- */
- __u16 mbz16;
-
- /**
- * @flags: all undefined flags must be zero, currently not defined flags
- */
- __u64 flags;
-
- /**
- * @mbz64: reserved for future use; must be zero
- */
- __u64 mbz64[3];
-
- /**
- * @engines: 2-d array of engine instances to configure parallel engine
- *
- * length = width (i) * num_siblings (j)
- * index = j + i * num_siblings
- */
- struct i915_engine_class_instance engines[0];
-
-} __packed;
-
diff --git a/Documentation/gpu/rfc/i915_scheduler.rst b/Documentation/gpu/rfc/i915_scheduler.rst
index cbda75065dad..d630f15ab795 100644
--- a/Documentation/gpu/rfc/i915_scheduler.rst
+++ b/Documentation/gpu/rfc/i915_scheduler.rst
@@ -135,8 +135,8 @@ Add I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT and
drm_i915_context_engines_parallel_submit to the uAPI to implement this
extension.
-.. kernel-doc:: Documentation/gpu/rfc/i915_parallel_execbuf.h
- :functions: drm_i915_context_engines_parallel_submit
+.. kernel-doc:: include/uapi/drm/i915_drm.h
+ :functions: i915_context_engines_parallel_submit
Extend execbuf2 IOCTL to support submitting N BBs in a single IOCTL
-------------------------------------------------------------------
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 17/25] drm/i915/doc: Update parallel submit doc to point to i915_drm.h
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Update parallel submit doc to point to i915_drm.h
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
Documentation/gpu/rfc/i915_parallel_execbuf.h | 122 ------------------
Documentation/gpu/rfc/i915_scheduler.rst | 4 +-
2 files changed, 2 insertions(+), 124 deletions(-)
delete mode 100644 Documentation/gpu/rfc/i915_parallel_execbuf.h
diff --git a/Documentation/gpu/rfc/i915_parallel_execbuf.h b/Documentation/gpu/rfc/i915_parallel_execbuf.h
deleted file mode 100644
index 8cbe2c4e0172..000000000000
--- a/Documentation/gpu/rfc/i915_parallel_execbuf.h
+++ /dev/null
@@ -1,122 +0,0 @@
-/* SPDX-License-Identifier: MIT */
-/*
- * Copyright © 2021 Intel Corporation
- */
-
-#define I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT 2 /* see i915_context_engines_parallel_submit */
-
-/**
- * struct drm_i915_context_engines_parallel_submit - Configure engine for
- * parallel submission.
- *
- * Setup a slot in the context engine map to allow multiple BBs to be submitted
- * in a single execbuf IOCTL. Those BBs will then be scheduled to run on the GPU
- * in parallel. Multiple hardware contexts are created internally in the i915
- * run these BBs. Once a slot is configured for N BBs only N BBs can be
- * submitted in each execbuf IOCTL and this is implicit behavior e.g. The user
- * doesn't tell the execbuf IOCTL there are N BBs, the execbuf IOCTL knows how
- * many BBs there are based on the slot's configuration. The N BBs are the last
- * N buffer objects or first N if I915_EXEC_BATCH_FIRST is set.
- *
- * The default placement behavior is to create implicit bonds between each
- * context if each context maps to more than 1 physical engine (e.g. context is
- * a virtual engine). Also we only allow contexts of same engine class and these
- * contexts must be in logically contiguous order. Examples of the placement
- * behavior described below. Lastly, the default is to not allow BBs to
- * preempted mid BB rather insert coordinated preemption on all hardware
- * contexts between each set of BBs. Flags may be added in the future to change
- * both of these default behaviors.
- *
- * Returns -EINVAL if hardware context placement configuration is invalid or if
- * the placement configuration isn't supported on the platform / submission
- * interface.
- * Returns -ENODEV if extension isn't supported on the platform / submission
- * interface.
- *
- * .. code-block:: none
- *
- * Example 1 pseudo code:
- * CS[X] = generic engine of same class, logical instance X
- * INVALID = I915_ENGINE_CLASS_INVALID, I915_ENGINE_CLASS_INVALID_NONE
- * set_engines(INVALID)
- * set_parallel(engine_index=0, width=2, num_siblings=1,
- * engines=CS[0],CS[1])
- *
- * Results in the following valid placement:
- * CS[0], CS[1]
- *
- * Example 2 pseudo code:
- * CS[X] = generic engine of same class, logical instance X
- * INVALID = I915_ENGINE_CLASS_INVALID, I915_ENGINE_CLASS_INVALID_NONE
- * set_engines(INVALID)
- * set_parallel(engine_index=0, width=2, num_siblings=2,
- * engines=CS[0],CS[2],CS[1],CS[3])
- *
- * Results in the following valid placements:
- * CS[0], CS[1]
- * CS[2], CS[3]
- *
- * This can also be thought of as 2 virtual engines described by 2-D array
- * in the engines the field with bonds placed between each index of the
- * virtual engines. e.g. CS[0] is bonded to CS[1], CS[2] is bonded to
- * CS[3].
- * VE[0] = CS[0], CS[2]
- * VE[1] = CS[1], CS[3]
- *
- * Example 3 pseudo code:
- * CS[X] = generic engine of same class, logical instance X
- * INVALID = I915_ENGINE_CLASS_INVALID, I915_ENGINE_CLASS_INVALID_NONE
- * set_engines(INVALID)
- * set_parallel(engine_index=0, width=2, num_siblings=2,
- * engines=CS[0],CS[1],CS[1],CS[3])
- *
- * Results in the following valid and invalid placements:
- * CS[0], CS[1]
- * CS[1], CS[3] - Not logical contiguous, return -EINVAL
- */
-struct drm_i915_context_engines_parallel_submit {
- /**
- * @base: base user extension.
- */
- struct i915_user_extension base;
-
- /**
- * @engine_index: slot for parallel engine
- */
- __u16 engine_index;
-
- /**
- * @width: number of contexts per parallel engine
- */
- __u16 width;
-
- /**
- * @num_siblings: number of siblings per context
- */
- __u16 num_siblings;
-
- /**
- * @mbz16: reserved for future use; must be zero
- */
- __u16 mbz16;
-
- /**
- * @flags: all undefined flags must be zero, currently not defined flags
- */
- __u64 flags;
-
- /**
- * @mbz64: reserved for future use; must be zero
- */
- __u64 mbz64[3];
-
- /**
- * @engines: 2-d array of engine instances to configure parallel engine
- *
- * length = width (i) * num_siblings (j)
- * index = j + i * num_siblings
- */
- struct i915_engine_class_instance engines[0];
-
-} __packed;
-
diff --git a/Documentation/gpu/rfc/i915_scheduler.rst b/Documentation/gpu/rfc/i915_scheduler.rst
index cbda75065dad..d630f15ab795 100644
--- a/Documentation/gpu/rfc/i915_scheduler.rst
+++ b/Documentation/gpu/rfc/i915_scheduler.rst
@@ -135,8 +135,8 @@ Add I915_CONTEXT_ENGINES_EXT_PARALLEL_SUBMIT and
drm_i915_context_engines_parallel_submit to the uAPI to implement this
extension.
-.. kernel-doc:: Documentation/gpu/rfc/i915_parallel_execbuf.h
- :functions: drm_i915_context_engines_parallel_submit
+.. kernel-doc:: include/uapi/drm/i915_drm.h
+ :functions: i915_context_engines_parallel_submit
Extend execbuf2 IOCTL to support submitting N BBs in a single IOCTL
-------------------------------------------------------------------
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 18/25] drm/i915/guc: Add basic GuC multi-lrc selftest
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Add very basic (single submission) multi-lrc selftest.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 1 +
.../drm/i915/gt/uc/selftest_guc_multi_lrc.c | 179 ++++++++++++++++++
.../drm/i915/selftests/i915_live_selftests.h | 1 +
3 files changed, 181 insertions(+)
create mode 100644 drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 7c12364a017a..57eb5f8bc8bb 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -3954,4 +3954,5 @@ bool intel_guc_virtual_engine_has_heartbeat(const struct intel_engine_cs *ve)
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
#include "selftest_guc.c"
+#include "selftest_guc_multi_lrc.c"
#endif
diff --git a/drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c b/drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c
new file mode 100644
index 000000000000..50953c8e8b53
--- /dev/null
+++ b/drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c
@@ -0,0 +1,179 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright �� 2019 Intel Corporation
+ */
+
+#include "selftests/igt_spinner.h"
+#include "selftests/igt_reset.h"
+#include "selftests/intel_scheduler_helpers.h"
+#include "gt/intel_engine_heartbeat.h"
+#include "gem/selftests/mock_context.h"
+
+static void logical_sort(struct intel_engine_cs **engines, int num_engines)
+{
+ struct intel_engine_cs *sorted[MAX_ENGINE_INSTANCE + 1];
+ int i, j;
+
+ for (i = 0; i < num_engines; ++i)
+ for (j = 0; j < MAX_ENGINE_INSTANCE + 1; ++j) {
+ if (engines[j]->logical_mask & BIT(i)) {
+ sorted[i] = engines[j];
+ break;
+ }
+ }
+
+ memcpy(*engines, *sorted,
+ sizeof(struct intel_engine_cs *) * num_engines);
+}
+
+static struct intel_context *
+multi_lrc_create_parent(struct intel_gt *gt, u8 class,
+ unsigned long flags)
+{
+ struct intel_engine_cs *siblings[MAX_ENGINE_INSTANCE + 1];
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+ int i = 0;
+
+ for_each_engine(engine, gt, id) {
+ if (engine->class != class)
+ continue;
+
+ siblings[i++] = engine;
+ }
+
+ if (i <= 1)
+ return ERR_PTR(0);
+
+ logical_sort(siblings, i);
+
+ return intel_engine_create_parallel(siblings, 1, i);
+}
+
+static void multi_lrc_context_unpin(struct intel_context *ce)
+{
+ struct intel_context *child;
+
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ for_each_child(ce, child)
+ intel_context_unpin(child);
+ intel_context_unpin(ce);
+}
+
+static void multi_lrc_context_put(struct intel_context *ce)
+{
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ /*
+ * Only the parent gets the creation ref put in the uAPI, the parent
+ * itself is responsible for creation ref put on the children.
+ */
+ intel_context_put(ce);
+}
+
+static struct i915_request *
+multi_lrc_nop_request(struct intel_context *ce)
+{
+ struct intel_context *child;
+ struct i915_request *rq, *child_rq;
+ int i = 0;
+
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ rq = intel_context_create_request(ce);
+ if (IS_ERR(rq))
+ return rq;
+
+ i915_request_get(rq);
+ i915_request_add(rq);
+
+ for_each_child(ce, child) {
+ child_rq = intel_context_create_request(child);
+ if (IS_ERR(child_rq))
+ goto child_error;
+
+ if (++i == ce->parallel.number_children)
+ set_bit(I915_FENCE_FLAG_SUBMIT_PARALLEL,
+ &child_rq->fence.flags);
+ i915_request_add(child_rq);
+ }
+
+ return rq;
+
+child_error:
+ i915_request_put(rq);
+
+ return ERR_PTR(-ENOMEM);
+}
+
+static int __intel_guc_multi_lrc_basic(struct intel_gt *gt, unsigned int class)
+{
+ struct intel_context *parent;
+ struct i915_request *rq;
+ int ret;
+
+ parent = multi_lrc_create_parent(gt, class, 0);
+ if (IS_ERR(parent)) {
+ pr_err("Failed creating contexts: %ld", PTR_ERR(parent));
+ return PTR_ERR(parent);
+ } else if (!parent) {
+ pr_debug("Not enough engines in class: %d", class);
+ return 0;
+ }
+
+ rq = multi_lrc_nop_request(parent);
+ if (IS_ERR(rq)) {
+ ret = PTR_ERR(rq);
+ pr_err("Failed creating requests: %d", ret);
+ goto out;
+ }
+
+ ret = intel_selftest_wait_for_rq(rq);
+ if (ret)
+ pr_err("Failed waiting on request: %d", ret);
+
+ i915_request_put(rq);
+
+ if (ret >= 0) {
+ ret = intel_gt_wait_for_idle(gt, HZ * 5);
+ if (ret < 0)
+ pr_err("GT failed to idle: %d\n", ret);
+ }
+
+out:
+ multi_lrc_context_unpin(parent);
+ multi_lrc_context_put(parent);
+ return ret;
+}
+
+static int intel_guc_multi_lrc_basic(void *arg)
+{
+ struct intel_gt *gt = arg;
+ unsigned int class;
+ int ret;
+
+ for (class = 0; class < MAX_ENGINE_CLASS + 1; ++class) {
+ ret = __intel_guc_multi_lrc_basic(gt, class);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+int intel_guc_multi_lrc_live_selftests(struct drm_i915_private *i915)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(intel_guc_multi_lrc_basic),
+ };
+ struct intel_gt *gt = &i915->gt;
+
+ if (intel_gt_is_wedged(gt))
+ return 0;
+
+ if (!intel_uc_uses_guc_submission(>->uc))
+ return 0;
+
+ return intel_gt_live_subtests(tests, gt);
+}
diff --git a/drivers/gpu/drm/i915/selftests/i915_live_selftests.h b/drivers/gpu/drm/i915/selftests/i915_live_selftests.h
index 3cf6758931f9..bdd290f2bf3c 100644
--- a/drivers/gpu/drm/i915/selftests/i915_live_selftests.h
+++ b/drivers/gpu/drm/i915/selftests/i915_live_selftests.h
@@ -48,5 +48,6 @@ selftest(ring_submission, intel_ring_submission_live_selftests)
selftest(perf, i915_perf_live_selftests)
selftest(slpc, intel_slpc_live_selftests)
selftest(guc, intel_guc_live_selftests)
+selftest(guc_multi_lrc, intel_guc_multi_lrc_live_selftests)
/* Here be dragons: keep last to run last! */
selftest(late_gt_pm, intel_gt_pm_late_selftests)
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 18/25] drm/i915/guc: Add basic GuC multi-lrc selftest
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Add very basic (single submission) multi-lrc selftest.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 1 +
.../drm/i915/gt/uc/selftest_guc_multi_lrc.c | 179 ++++++++++++++++++
.../drm/i915/selftests/i915_live_selftests.h | 1 +
3 files changed, 181 insertions(+)
create mode 100644 drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 7c12364a017a..57eb5f8bc8bb 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -3954,4 +3954,5 @@ bool intel_guc_virtual_engine_has_heartbeat(const struct intel_engine_cs *ve)
#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
#include "selftest_guc.c"
+#include "selftest_guc_multi_lrc.c"
#endif
diff --git a/drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c b/drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c
new file mode 100644
index 000000000000..50953c8e8b53
--- /dev/null
+++ b/drivers/gpu/drm/i915/gt/uc/selftest_guc_multi_lrc.c
@@ -0,0 +1,179 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright �� 2019 Intel Corporation
+ */
+
+#include "selftests/igt_spinner.h"
+#include "selftests/igt_reset.h"
+#include "selftests/intel_scheduler_helpers.h"
+#include "gt/intel_engine_heartbeat.h"
+#include "gem/selftests/mock_context.h"
+
+static void logical_sort(struct intel_engine_cs **engines, int num_engines)
+{
+ struct intel_engine_cs *sorted[MAX_ENGINE_INSTANCE + 1];
+ int i, j;
+
+ for (i = 0; i < num_engines; ++i)
+ for (j = 0; j < MAX_ENGINE_INSTANCE + 1; ++j) {
+ if (engines[j]->logical_mask & BIT(i)) {
+ sorted[i] = engines[j];
+ break;
+ }
+ }
+
+ memcpy(*engines, *sorted,
+ sizeof(struct intel_engine_cs *) * num_engines);
+}
+
+static struct intel_context *
+multi_lrc_create_parent(struct intel_gt *gt, u8 class,
+ unsigned long flags)
+{
+ struct intel_engine_cs *siblings[MAX_ENGINE_INSTANCE + 1];
+ struct intel_engine_cs *engine;
+ enum intel_engine_id id;
+ int i = 0;
+
+ for_each_engine(engine, gt, id) {
+ if (engine->class != class)
+ continue;
+
+ siblings[i++] = engine;
+ }
+
+ if (i <= 1)
+ return ERR_PTR(0);
+
+ logical_sort(siblings, i);
+
+ return intel_engine_create_parallel(siblings, 1, i);
+}
+
+static void multi_lrc_context_unpin(struct intel_context *ce)
+{
+ struct intel_context *child;
+
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ for_each_child(ce, child)
+ intel_context_unpin(child);
+ intel_context_unpin(ce);
+}
+
+static void multi_lrc_context_put(struct intel_context *ce)
+{
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ /*
+ * Only the parent gets the creation ref put in the uAPI, the parent
+ * itself is responsible for creation ref put on the children.
+ */
+ intel_context_put(ce);
+}
+
+static struct i915_request *
+multi_lrc_nop_request(struct intel_context *ce)
+{
+ struct intel_context *child;
+ struct i915_request *rq, *child_rq;
+ int i = 0;
+
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ rq = intel_context_create_request(ce);
+ if (IS_ERR(rq))
+ return rq;
+
+ i915_request_get(rq);
+ i915_request_add(rq);
+
+ for_each_child(ce, child) {
+ child_rq = intel_context_create_request(child);
+ if (IS_ERR(child_rq))
+ goto child_error;
+
+ if (++i == ce->parallel.number_children)
+ set_bit(I915_FENCE_FLAG_SUBMIT_PARALLEL,
+ &child_rq->fence.flags);
+ i915_request_add(child_rq);
+ }
+
+ return rq;
+
+child_error:
+ i915_request_put(rq);
+
+ return ERR_PTR(-ENOMEM);
+}
+
+static int __intel_guc_multi_lrc_basic(struct intel_gt *gt, unsigned int class)
+{
+ struct intel_context *parent;
+ struct i915_request *rq;
+ int ret;
+
+ parent = multi_lrc_create_parent(gt, class, 0);
+ if (IS_ERR(parent)) {
+ pr_err("Failed creating contexts: %ld", PTR_ERR(parent));
+ return PTR_ERR(parent);
+ } else if (!parent) {
+ pr_debug("Not enough engines in class: %d", class);
+ return 0;
+ }
+
+ rq = multi_lrc_nop_request(parent);
+ if (IS_ERR(rq)) {
+ ret = PTR_ERR(rq);
+ pr_err("Failed creating requests: %d", ret);
+ goto out;
+ }
+
+ ret = intel_selftest_wait_for_rq(rq);
+ if (ret)
+ pr_err("Failed waiting on request: %d", ret);
+
+ i915_request_put(rq);
+
+ if (ret >= 0) {
+ ret = intel_gt_wait_for_idle(gt, HZ * 5);
+ if (ret < 0)
+ pr_err("GT failed to idle: %d\n", ret);
+ }
+
+out:
+ multi_lrc_context_unpin(parent);
+ multi_lrc_context_put(parent);
+ return ret;
+}
+
+static int intel_guc_multi_lrc_basic(void *arg)
+{
+ struct intel_gt *gt = arg;
+ unsigned int class;
+ int ret;
+
+ for (class = 0; class < MAX_ENGINE_CLASS + 1; ++class) {
+ ret = __intel_guc_multi_lrc_basic(gt, class);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+int intel_guc_multi_lrc_live_selftests(struct drm_i915_private *i915)
+{
+ static const struct i915_subtest tests[] = {
+ SUBTEST(intel_guc_multi_lrc_basic),
+ };
+ struct intel_gt *gt = &i915->gt;
+
+ if (intel_gt_is_wedged(gt))
+ return 0;
+
+ if (!intel_uc_uses_guc_submission(>->uc))
+ return 0;
+
+ return intel_gt_live_subtests(tests, gt);
+}
diff --git a/drivers/gpu/drm/i915/selftests/i915_live_selftests.h b/drivers/gpu/drm/i915/selftests/i915_live_selftests.h
index 3cf6758931f9..bdd290f2bf3c 100644
--- a/drivers/gpu/drm/i915/selftests/i915_live_selftests.h
+++ b/drivers/gpu/drm/i915/selftests/i915_live_selftests.h
@@ -48,5 +48,6 @@ selftest(ring_submission, intel_ring_submission_live_selftests)
selftest(perf, i915_perf_live_selftests)
selftest(slpc, intel_slpc_live_selftests)
selftest(guc, intel_guc_live_selftests)
+selftest(guc_multi_lrc, intel_guc_multi_lrc_live_selftests)
/* Here be dragons: keep last to run last! */
selftest(late_gt_pm, intel_gt_pm_late_selftests)
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 19/25] drm/i915/guc: Implement no mid batch preemption for multi-lrc
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:19 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
For some users of multi-lrc, e.g. split frame, it isn't safe to preempt
mid BB. To safely enable preemption at the BB boundary, a handshake
between parent and child is needed, syncing the set of BBs at the
beginning and end of each batch. This is implemented via custom
emit_bb_start & emit_fini_breadcrumb functions and enabled by default if
a context is configured by set parallel extension.
Lastly, this patch updates the process descriptor to the correct size as
the memory used in the handshake is directly after the process
descriptor.
v2:
(John Harrison)
- Fix a few comments wording
- Add struture for parent page layout
v3:
(John Harrison)
- A structure for sync semaphore
- Use offsetof to calc address
- Update commit message
v4:
(John Harrison)
- Fix typos in comment explaining memory map of scratch page
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.c | 2 +-
drivers/gpu/drm/i915/gt/intel_context_types.h | 2 +
drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 2 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 333 +++++++++++++++++-
4 files changed, 326 insertions(+), 13 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index 6aab60584ee5..5634d14052bc 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -570,7 +570,7 @@ void intel_context_bind_parent_child(struct intel_context *parent,
GEM_BUG_ON(intel_context_is_child(child));
GEM_BUG_ON(intel_context_is_parent(child));
- parent->parallel.number_children++;
+ parent->parallel.child_index = parent->parallel.number_children++;
list_add_tail(&child->parallel.child_link,
&parent->parallel.child_list);
child->parallel.parent = parent;
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 1d880303a7e4..95a5b94b4ece 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -250,6 +250,8 @@ struct intel_context {
struct i915_request *last_rq;
/** @number_children: number of children if parent */
u8 number_children;
+ /** @child_index: index into child_list if child */
+ u8 child_index;
/** @guc: GuC specific members for parallel submission */
struct {
/** @wqi_head: head pointer in work queue */
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
index 18da67cfcd92..722933e26347 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
@@ -186,7 +186,7 @@ struct guc_process_desc {
u32 wq_status;
u32 engine_presence;
u32 priority;
- u32 reserved[30];
+ u32 reserved[36];
} __packed;
#define CONTEXT_REGISTRATION_FLAG_KMD BIT(0)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 57eb5f8bc8bb..50f0f4eba03b 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -11,6 +11,7 @@
#include "gt/intel_context.h"
#include "gt/intel_engine_pm.h"
#include "gt/intel_engine_heartbeat.h"
+#include "gt/intel_gpu_commands.h"
#include "gt/intel_gt.h"
#include "gt/intel_gt_irq.h"
#include "gt/intel_gt_pm.h"
@@ -368,11 +369,16 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
/*
* When using multi-lrc submission a scratch memory area is reserved in the
- * parent's context state for the process descriptor and work queue. Currently
- * the scratch area is sized to a page.
+ * parent's context state for the process descriptor, work queue, and handshake
+ * between the parent + children contexts to insert safe preemption points
+ * between each of the BBs. Currently the scratch area is sized to a page.
*
* The layout of this scratch area is below:
* 0 guc_process_desc
+ * + sizeof(struct guc_process_desc) child go
+ * + CACHELINE_BYTES child join[0]
+ * ...
+ * + CACHELINE_BYTES child join[n - 1]
* ... unused
* PARENT_SCRATCH_SIZE / 2 work queue start
* ... work queue
@@ -380,7 +386,25 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
*/
#define WQ_SIZE (PARENT_SCRATCH_SIZE / 2)
#define WQ_OFFSET (PARENT_SCRATCH_SIZE - WQ_SIZE)
-static u32 __get_process_desc_offset(struct intel_context *ce)
+
+struct sync_semaphore {
+ u32 semaphore;
+ u8 unused[CACHELINE_BYTES - sizeof(u32)];
+};
+
+struct parent_scratch {
+ struct guc_process_desc pdesc;
+
+ struct sync_semaphore go;
+ struct sync_semaphore join[MAX_ENGINE_INSTANCE + 1];
+
+ u8 unused[WQ_OFFSET - sizeof(struct guc_process_desc) -
+ sizeof(struct sync_semaphore) * (MAX_ENGINE_INSTANCE + 2)];
+
+ u32 wq[WQ_SIZE / sizeof(u32)];
+};
+
+static u32 __get_parent_scratch_offset(struct intel_context *ce)
{
GEM_BUG_ON(!ce->parallel.guc.parent_page);
@@ -389,23 +413,36 @@ static u32 __get_process_desc_offset(struct intel_context *ce)
static u32 __get_wq_offset(struct intel_context *ce)
{
- return __get_process_desc_offset(ce) + WQ_OFFSET;
+ BUILD_BUG_ON(offsetof(struct parent_scratch, wq) != WQ_OFFSET);
+
+ return __get_parent_scratch_offset(ce) + WQ_OFFSET;
}
-static struct guc_process_desc *
-__get_process_desc(struct intel_context *ce)
+static struct parent_scratch *
+__get_parent_scratch(struct intel_context *ce)
{
+ BUILD_BUG_ON(sizeof(struct parent_scratch) != PARENT_SCRATCH_SIZE);
+ BUILD_BUG_ON(sizeof(struct sync_semaphore) != CACHELINE_BYTES);
+
/*
* Need to subtract LRC_STATE_OFFSET here as the
* parallel.guc.parent_page is the offset into ce->state while
* ce->lrc_reg_reg is ce->state + LRC_STATE_OFFSET.
*/
- return (struct guc_process_desc *)
+ return (struct parent_scratch *)
(ce->lrc_reg_state +
- ((__get_process_desc_offset(ce) -
+ ((__get_parent_scratch_offset(ce) -
LRC_STATE_OFFSET) / sizeof(u32)));
}
+static struct guc_process_desc *
+__get_process_desc(struct intel_context *ce)
+{
+ struct parent_scratch *ps = __get_parent_scratch(ce);
+
+ return &ps->pdesc;
+}
+
static u32 *get_wq_pointer(struct guc_process_desc *desc,
struct intel_context *ce,
u32 wqi_size)
@@ -425,8 +462,7 @@ static u32 *get_wq_pointer(struct guc_process_desc *desc,
}
#undef AVAILABLE_SPACE
- return ((u32 *)__get_process_desc(ce)) +
- ((WQ_OFFSET + ce->parallel.guc.wqi_tail) / sizeof(u32));
+ return &__get_parent_scratch(ce)->wq[ce->parallel.guc.wqi_tail / sizeof(u32)];
}
static struct guc_lrc_desc *__get_lrc_desc(struct intel_guc *guc, u32 index)
@@ -1827,6 +1863,27 @@ static int deregister_context(struct intel_context *ce, u32 guc_id)
return __guc_action_deregister_context(guc, guc_id);
}
+static inline void clear_children_join_go_memory(struct intel_context *ce)
+{
+ struct parent_scratch *ps = __get_parent_scratch(ce);
+ int i;
+
+ ps->go.semaphore = 0;
+ for (i = 0; i < ce->parallel.number_children + 1; ++i)
+ ps->join[i].semaphore = 0;
+}
+
+static inline u32 get_children_go_value(struct intel_context *ce)
+{
+ return __get_parent_scratch(ce)->go.semaphore;
+}
+
+static inline u32 get_children_join_value(struct intel_context *ce,
+ u8 child_index)
+{
+ return __get_parent_scratch(ce)->join[child_index].semaphore;
+}
+
static void guc_context_policy_init(struct intel_engine_cs *engine,
struct guc_lrc_desc *desc)
{
@@ -1886,7 +1943,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)
ce->parallel.guc.wqi_head = 0;
desc->process_desc = i915_ggtt_offset(ce->state) +
- __get_process_desc_offset(ce);
+ __get_parent_scratch_offset(ce);
desc->wq_addr = i915_ggtt_offset(ce->state) +
__get_wq_offset(ce);
desc->wq_size = WQ_SIZE;
@@ -1908,6 +1965,8 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)
desc->context_flags = CONTEXT_REGISTRATION_FLAG_KMD;
guc_context_policy_init(engine, desc);
}
+
+ clear_children_join_go_memory(ce);
}
/*
@@ -2974,6 +3033,31 @@ static const struct intel_context_ops virtual_child_context_ops = {
.get_sibling = guc_virtual_get_sibling,
};
+/*
+ * The below override of the breadcrumbs is enabled when the user configures a
+ * context for parallel submission (multi-lrc, parent-child).
+ *
+ * The overridden breadcrumbs implements an algorithm which allows the GuC to
+ * safely preempt all the hw contexts configured for parallel submission
+ * between each BB. The contract between the i915 and GuC is if the parent
+ * context can be preempted, all the children can be preempted, and the GuC will
+ * always try to preempt the parent before the children. A handshake between the
+ * parent / children breadcrumbs ensures the i915 holds up its end of the deal
+ * creating a window to preempt between each set of BBs.
+ */
+static int emit_bb_start_parent_no_preempt_mid_batch(struct i915_request *rq,
+ u64 offset, u32 len,
+ const unsigned int flags);
+static int emit_bb_start_child_no_preempt_mid_batch(struct i915_request *rq,
+ u64 offset, u32 len,
+ const unsigned int flags);
+static u32 *
+emit_fini_breadcrumb_parent_no_preempt_mid_batch(struct i915_request *rq,
+ u32 *cs);
+static u32 *
+emit_fini_breadcrumb_child_no_preempt_mid_batch(struct i915_request *rq,
+ u32 *cs);
+
static struct intel_context *
guc_create_parallel(struct intel_engine_cs **engines,
unsigned int num_siblings,
@@ -3009,6 +3093,20 @@ guc_create_parallel(struct intel_engine_cs **engines,
}
}
+ parent->engine->emit_bb_start =
+ emit_bb_start_parent_no_preempt_mid_batch;
+ parent->engine->emit_fini_breadcrumb =
+ emit_fini_breadcrumb_parent_no_preempt_mid_batch;
+ parent->engine->emit_fini_breadcrumb_dw =
+ 12 + 4 * parent->parallel.number_children;
+ for_each_child(parent, ce) {
+ ce->engine->emit_bb_start =
+ emit_bb_start_child_no_preempt_mid_batch;
+ ce->engine->emit_fini_breadcrumb =
+ emit_fini_breadcrumb_child_no_preempt_mid_batch;
+ ce->engine->emit_fini_breadcrumb_dw = 16;
+ }
+
kfree(siblings);
return parent;
@@ -3837,6 +3935,17 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc,
drm_printf(p, "\t\tWQI Status: %u\n\n",
READ_ONCE(desc->wq_status));
+ if (ce->engine->emit_bb_start ==
+ emit_bb_start_parent_no_preempt_mid_batch) {
+ u8 i;
+
+ drm_printf(p, "\t\tChildren Go: %u\n\n",
+ get_children_go_value(ce));
+ for (i = 0; i < ce->parallel.number_children; ++i)
+ drm_printf(p, "\t\tChildren Join: %u\n",
+ get_children_join_value(ce, i));
+ }
+
for_each_child(ce, child)
guc_log_context(p, child);
}
@@ -3844,6 +3953,208 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc,
xa_unlock_irqrestore(&guc->context_lookup, flags);
}
+static inline u32 get_children_go_addr(struct intel_context *ce)
+{
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ return i915_ggtt_offset(ce->state) +
+ __get_parent_scratch_offset(ce) +
+ offsetof(struct parent_scratch, go.semaphore);
+}
+
+static inline u32 get_children_join_addr(struct intel_context *ce,
+ u8 child_index)
+{
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ return i915_ggtt_offset(ce->state) +
+ __get_parent_scratch_offset(ce) +
+ offsetof(struct parent_scratch, join[child_index].semaphore);
+}
+
+#define PARENT_GO_BB 1
+#define PARENT_GO_FINI_BREADCRUMB 0
+#define CHILD_GO_BB 1
+#define CHILD_GO_FINI_BREADCRUMB 0
+static int emit_bb_start_parent_no_preempt_mid_batch(struct i915_request *rq,
+ u64 offset, u32 len,
+ const unsigned int flags)
+{
+ struct intel_context *ce = rq->context;
+ u32 *cs;
+ u8 i;
+
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ cs = intel_ring_begin(rq, 10 + 4 * ce->parallel.number_children);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
+
+ /* Wait on children */
+ for (i = 0; i < ce->parallel.number_children; ++i) {
+ *cs++ = (MI_SEMAPHORE_WAIT |
+ MI_SEMAPHORE_GLOBAL_GTT |
+ MI_SEMAPHORE_POLL |
+ MI_SEMAPHORE_SAD_EQ_SDD);
+ *cs++ = PARENT_GO_BB;
+ *cs++ = get_children_join_addr(ce, i);
+ *cs++ = 0;
+ }
+
+ /* Turn off preemption */
+ *cs++ = MI_ARB_ON_OFF | MI_ARB_DISABLE;
+ *cs++ = MI_NOOP;
+
+ /* Tell children go */
+ cs = gen8_emit_ggtt_write(cs,
+ CHILD_GO_BB,
+ get_children_go_addr(ce),
+ 0);
+
+ /* Jump to batch */
+ *cs++ = MI_BATCH_BUFFER_START_GEN8 |
+ (flags & I915_DISPATCH_SECURE ? 0 : BIT(8));
+ *cs++ = lower_32_bits(offset);
+ *cs++ = upper_32_bits(offset);
+ *cs++ = MI_NOOP;
+
+ intel_ring_advance(rq, cs);
+
+ return 0;
+}
+
+static int emit_bb_start_child_no_preempt_mid_batch(struct i915_request *rq,
+ u64 offset, u32 len,
+ const unsigned int flags)
+{
+ struct intel_context *ce = rq->context;
+ struct intel_context *parent = intel_context_to_parent(ce);
+ u32 *cs;
+
+ GEM_BUG_ON(!intel_context_is_child(ce));
+
+ cs = intel_ring_begin(rq, 12);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
+
+ /* Signal parent */
+ cs = gen8_emit_ggtt_write(cs,
+ PARENT_GO_BB,
+ get_children_join_addr(parent,
+ ce->parallel.child_index),
+ 0);
+
+ /* Wait on parent for go */
+ *cs++ = (MI_SEMAPHORE_WAIT |
+ MI_SEMAPHORE_GLOBAL_GTT |
+ MI_SEMAPHORE_POLL |
+ MI_SEMAPHORE_SAD_EQ_SDD);
+ *cs++ = CHILD_GO_BB;
+ *cs++ = get_children_go_addr(parent);
+ *cs++ = 0;
+
+ /* Turn off preemption */
+ *cs++ = MI_ARB_ON_OFF | MI_ARB_DISABLE;
+
+ /* Jump to batch */
+ *cs++ = MI_BATCH_BUFFER_START_GEN8 |
+ (flags & I915_DISPATCH_SECURE ? 0 : BIT(8));
+ *cs++ = lower_32_bits(offset);
+ *cs++ = upper_32_bits(offset);
+
+ intel_ring_advance(rq, cs);
+
+ return 0;
+}
+
+static u32 *
+emit_fini_breadcrumb_parent_no_preempt_mid_batch(struct i915_request *rq,
+ u32 *cs)
+{
+ struct intel_context *ce = rq->context;
+ u8 i;
+
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ /* Wait on children */
+ for (i = 0; i < ce->parallel.number_children; ++i) {
+ *cs++ = (MI_SEMAPHORE_WAIT |
+ MI_SEMAPHORE_GLOBAL_GTT |
+ MI_SEMAPHORE_POLL |
+ MI_SEMAPHORE_SAD_EQ_SDD);
+ *cs++ = PARENT_GO_FINI_BREADCRUMB;
+ *cs++ = get_children_join_addr(ce, i);
+ *cs++ = 0;
+ }
+
+ /* Turn on preemption */
+ *cs++ = MI_ARB_ON_OFF | MI_ARB_ENABLE;
+ *cs++ = MI_NOOP;
+
+ /* Tell children go */
+ cs = gen8_emit_ggtt_write(cs,
+ CHILD_GO_FINI_BREADCRUMB,
+ get_children_go_addr(ce),
+ 0);
+
+ /* Emit fini breadcrumb */
+ cs = gen8_emit_ggtt_write(cs,
+ rq->fence.seqno,
+ i915_request_active_timeline(rq)->hwsp_offset,
+ 0);
+
+ /* User interrupt */
+ *cs++ = MI_USER_INTERRUPT;
+ *cs++ = MI_NOOP;
+
+ rq->tail = intel_ring_offset(rq, cs);
+
+ return cs;
+}
+
+static u32 *
+emit_fini_breadcrumb_child_no_preempt_mid_batch(struct i915_request *rq, u32 *cs)
+{
+ struct intel_context *ce = rq->context;
+ struct intel_context *parent = intel_context_to_parent(ce);
+
+ GEM_BUG_ON(!intel_context_is_child(ce));
+
+ /* Turn on preemption */
+ *cs++ = MI_ARB_ON_OFF | MI_ARB_ENABLE;
+ *cs++ = MI_NOOP;
+
+ /* Signal parent */
+ cs = gen8_emit_ggtt_write(cs,
+ PARENT_GO_FINI_BREADCRUMB,
+ get_children_join_addr(parent,
+ ce->parallel.child_index),
+ 0);
+
+ /* Wait parent on for go */
+ *cs++ = (MI_SEMAPHORE_WAIT |
+ MI_SEMAPHORE_GLOBAL_GTT |
+ MI_SEMAPHORE_POLL |
+ MI_SEMAPHORE_SAD_EQ_SDD);
+ *cs++ = CHILD_GO_FINI_BREADCRUMB;
+ *cs++ = get_children_go_addr(parent);
+ *cs++ = 0;
+
+ /* Emit fini breadcrumb */
+ cs = gen8_emit_ggtt_write(cs,
+ rq->fence.seqno,
+ i915_request_active_timeline(rq)->hwsp_offset,
+ 0);
+
+ /* User interrupt */
+ *cs++ = MI_USER_INTERRUPT;
+ *cs++ = MI_NOOP;
+
+ rq->tail = intel_ring_offset(rq, cs);
+
+ return cs;
+}
+
static struct intel_context *
guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
unsigned long flags)
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 19/25] drm/i915/guc: Implement no mid batch preemption for multi-lrc
@ 2021-10-14 17:19 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:19 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
For some users of multi-lrc, e.g. split frame, it isn't safe to preempt
mid BB. To safely enable preemption at the BB boundary, a handshake
between parent and child is needed, syncing the set of BBs at the
beginning and end of each batch. This is implemented via custom
emit_bb_start & emit_fini_breadcrumb functions and enabled by default if
a context is configured by set parallel extension.
Lastly, this patch updates the process descriptor to the correct size as
the memory used in the handshake is directly after the process
descriptor.
v2:
(John Harrison)
- Fix a few comments wording
- Add struture for parent page layout
v3:
(John Harrison)
- A structure for sync semaphore
- Use offsetof to calc address
- Update commit message
v4:
(John Harrison)
- Fix typos in comment explaining memory map of scratch page
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/gt/intel_context.c | 2 +-
drivers/gpu/drm/i915/gt/intel_context_types.h | 2 +
drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h | 2 +-
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 333 +++++++++++++++++-
4 files changed, 326 insertions(+), 13 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index 6aab60584ee5..5634d14052bc 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -570,7 +570,7 @@ void intel_context_bind_parent_child(struct intel_context *parent,
GEM_BUG_ON(intel_context_is_child(child));
GEM_BUG_ON(intel_context_is_parent(child));
- parent->parallel.number_children++;
+ parent->parallel.child_index = parent->parallel.number_children++;
list_add_tail(&child->parallel.child_link,
&parent->parallel.child_list);
child->parallel.parent = parent;
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 1d880303a7e4..95a5b94b4ece 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -250,6 +250,8 @@ struct intel_context {
struct i915_request *last_rq;
/** @number_children: number of children if parent */
u8 number_children;
+ /** @child_index: index into child_list if child */
+ u8 child_index;
/** @guc: GuC specific members for parallel submission */
struct {
/** @wqi_head: head pointer in work queue */
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
index 18da67cfcd92..722933e26347 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
@@ -186,7 +186,7 @@ struct guc_process_desc {
u32 wq_status;
u32 engine_presence;
u32 priority;
- u32 reserved[30];
+ u32 reserved[36];
} __packed;
#define CONTEXT_REGISTRATION_FLAG_KMD BIT(0)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 57eb5f8bc8bb..50f0f4eba03b 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -11,6 +11,7 @@
#include "gt/intel_context.h"
#include "gt/intel_engine_pm.h"
#include "gt/intel_engine_heartbeat.h"
+#include "gt/intel_gpu_commands.h"
#include "gt/intel_gt.h"
#include "gt/intel_gt_irq.h"
#include "gt/intel_gt_pm.h"
@@ -368,11 +369,16 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
/*
* When using multi-lrc submission a scratch memory area is reserved in the
- * parent's context state for the process descriptor and work queue. Currently
- * the scratch area is sized to a page.
+ * parent's context state for the process descriptor, work queue, and handshake
+ * between the parent + children contexts to insert safe preemption points
+ * between each of the BBs. Currently the scratch area is sized to a page.
*
* The layout of this scratch area is below:
* 0 guc_process_desc
+ * + sizeof(struct guc_process_desc) child go
+ * + CACHELINE_BYTES child join[0]
+ * ...
+ * + CACHELINE_BYTES child join[n - 1]
* ... unused
* PARENT_SCRATCH_SIZE / 2 work queue start
* ... work queue
@@ -380,7 +386,25 @@ static inline struct i915_priolist *to_priolist(struct rb_node *rb)
*/
#define WQ_SIZE (PARENT_SCRATCH_SIZE / 2)
#define WQ_OFFSET (PARENT_SCRATCH_SIZE - WQ_SIZE)
-static u32 __get_process_desc_offset(struct intel_context *ce)
+
+struct sync_semaphore {
+ u32 semaphore;
+ u8 unused[CACHELINE_BYTES - sizeof(u32)];
+};
+
+struct parent_scratch {
+ struct guc_process_desc pdesc;
+
+ struct sync_semaphore go;
+ struct sync_semaphore join[MAX_ENGINE_INSTANCE + 1];
+
+ u8 unused[WQ_OFFSET - sizeof(struct guc_process_desc) -
+ sizeof(struct sync_semaphore) * (MAX_ENGINE_INSTANCE + 2)];
+
+ u32 wq[WQ_SIZE / sizeof(u32)];
+};
+
+static u32 __get_parent_scratch_offset(struct intel_context *ce)
{
GEM_BUG_ON(!ce->parallel.guc.parent_page);
@@ -389,23 +413,36 @@ static u32 __get_process_desc_offset(struct intel_context *ce)
static u32 __get_wq_offset(struct intel_context *ce)
{
- return __get_process_desc_offset(ce) + WQ_OFFSET;
+ BUILD_BUG_ON(offsetof(struct parent_scratch, wq) != WQ_OFFSET);
+
+ return __get_parent_scratch_offset(ce) + WQ_OFFSET;
}
-static struct guc_process_desc *
-__get_process_desc(struct intel_context *ce)
+static struct parent_scratch *
+__get_parent_scratch(struct intel_context *ce)
{
+ BUILD_BUG_ON(sizeof(struct parent_scratch) != PARENT_SCRATCH_SIZE);
+ BUILD_BUG_ON(sizeof(struct sync_semaphore) != CACHELINE_BYTES);
+
/*
* Need to subtract LRC_STATE_OFFSET here as the
* parallel.guc.parent_page is the offset into ce->state while
* ce->lrc_reg_reg is ce->state + LRC_STATE_OFFSET.
*/
- return (struct guc_process_desc *)
+ return (struct parent_scratch *)
(ce->lrc_reg_state +
- ((__get_process_desc_offset(ce) -
+ ((__get_parent_scratch_offset(ce) -
LRC_STATE_OFFSET) / sizeof(u32)));
}
+static struct guc_process_desc *
+__get_process_desc(struct intel_context *ce)
+{
+ struct parent_scratch *ps = __get_parent_scratch(ce);
+
+ return &ps->pdesc;
+}
+
static u32 *get_wq_pointer(struct guc_process_desc *desc,
struct intel_context *ce,
u32 wqi_size)
@@ -425,8 +462,7 @@ static u32 *get_wq_pointer(struct guc_process_desc *desc,
}
#undef AVAILABLE_SPACE
- return ((u32 *)__get_process_desc(ce)) +
- ((WQ_OFFSET + ce->parallel.guc.wqi_tail) / sizeof(u32));
+ return &__get_parent_scratch(ce)->wq[ce->parallel.guc.wqi_tail / sizeof(u32)];
}
static struct guc_lrc_desc *__get_lrc_desc(struct intel_guc *guc, u32 index)
@@ -1827,6 +1863,27 @@ static int deregister_context(struct intel_context *ce, u32 guc_id)
return __guc_action_deregister_context(guc, guc_id);
}
+static inline void clear_children_join_go_memory(struct intel_context *ce)
+{
+ struct parent_scratch *ps = __get_parent_scratch(ce);
+ int i;
+
+ ps->go.semaphore = 0;
+ for (i = 0; i < ce->parallel.number_children + 1; ++i)
+ ps->join[i].semaphore = 0;
+}
+
+static inline u32 get_children_go_value(struct intel_context *ce)
+{
+ return __get_parent_scratch(ce)->go.semaphore;
+}
+
+static inline u32 get_children_join_value(struct intel_context *ce,
+ u8 child_index)
+{
+ return __get_parent_scratch(ce)->join[child_index].semaphore;
+}
+
static void guc_context_policy_init(struct intel_engine_cs *engine,
struct guc_lrc_desc *desc)
{
@@ -1886,7 +1943,7 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)
ce->parallel.guc.wqi_head = 0;
desc->process_desc = i915_ggtt_offset(ce->state) +
- __get_process_desc_offset(ce);
+ __get_parent_scratch_offset(ce);
desc->wq_addr = i915_ggtt_offset(ce->state) +
__get_wq_offset(ce);
desc->wq_size = WQ_SIZE;
@@ -1908,6 +1965,8 @@ static int guc_lrc_desc_pin(struct intel_context *ce, bool loop)
desc->context_flags = CONTEXT_REGISTRATION_FLAG_KMD;
guc_context_policy_init(engine, desc);
}
+
+ clear_children_join_go_memory(ce);
}
/*
@@ -2974,6 +3033,31 @@ static const struct intel_context_ops virtual_child_context_ops = {
.get_sibling = guc_virtual_get_sibling,
};
+/*
+ * The below override of the breadcrumbs is enabled when the user configures a
+ * context for parallel submission (multi-lrc, parent-child).
+ *
+ * The overridden breadcrumbs implements an algorithm which allows the GuC to
+ * safely preempt all the hw contexts configured for parallel submission
+ * between each BB. The contract between the i915 and GuC is if the parent
+ * context can be preempted, all the children can be preempted, and the GuC will
+ * always try to preempt the parent before the children. A handshake between the
+ * parent / children breadcrumbs ensures the i915 holds up its end of the deal
+ * creating a window to preempt between each set of BBs.
+ */
+static int emit_bb_start_parent_no_preempt_mid_batch(struct i915_request *rq,
+ u64 offset, u32 len,
+ const unsigned int flags);
+static int emit_bb_start_child_no_preempt_mid_batch(struct i915_request *rq,
+ u64 offset, u32 len,
+ const unsigned int flags);
+static u32 *
+emit_fini_breadcrumb_parent_no_preempt_mid_batch(struct i915_request *rq,
+ u32 *cs);
+static u32 *
+emit_fini_breadcrumb_child_no_preempt_mid_batch(struct i915_request *rq,
+ u32 *cs);
+
static struct intel_context *
guc_create_parallel(struct intel_engine_cs **engines,
unsigned int num_siblings,
@@ -3009,6 +3093,20 @@ guc_create_parallel(struct intel_engine_cs **engines,
}
}
+ parent->engine->emit_bb_start =
+ emit_bb_start_parent_no_preempt_mid_batch;
+ parent->engine->emit_fini_breadcrumb =
+ emit_fini_breadcrumb_parent_no_preempt_mid_batch;
+ parent->engine->emit_fini_breadcrumb_dw =
+ 12 + 4 * parent->parallel.number_children;
+ for_each_child(parent, ce) {
+ ce->engine->emit_bb_start =
+ emit_bb_start_child_no_preempt_mid_batch;
+ ce->engine->emit_fini_breadcrumb =
+ emit_fini_breadcrumb_child_no_preempt_mid_batch;
+ ce->engine->emit_fini_breadcrumb_dw = 16;
+ }
+
kfree(siblings);
return parent;
@@ -3837,6 +3935,17 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc,
drm_printf(p, "\t\tWQI Status: %u\n\n",
READ_ONCE(desc->wq_status));
+ if (ce->engine->emit_bb_start ==
+ emit_bb_start_parent_no_preempt_mid_batch) {
+ u8 i;
+
+ drm_printf(p, "\t\tChildren Go: %u\n\n",
+ get_children_go_value(ce));
+ for (i = 0; i < ce->parallel.number_children; ++i)
+ drm_printf(p, "\t\tChildren Join: %u\n",
+ get_children_join_value(ce, i));
+ }
+
for_each_child(ce, child)
guc_log_context(p, child);
}
@@ -3844,6 +3953,208 @@ void intel_guc_submission_print_context_info(struct intel_guc *guc,
xa_unlock_irqrestore(&guc->context_lookup, flags);
}
+static inline u32 get_children_go_addr(struct intel_context *ce)
+{
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ return i915_ggtt_offset(ce->state) +
+ __get_parent_scratch_offset(ce) +
+ offsetof(struct parent_scratch, go.semaphore);
+}
+
+static inline u32 get_children_join_addr(struct intel_context *ce,
+ u8 child_index)
+{
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ return i915_ggtt_offset(ce->state) +
+ __get_parent_scratch_offset(ce) +
+ offsetof(struct parent_scratch, join[child_index].semaphore);
+}
+
+#define PARENT_GO_BB 1
+#define PARENT_GO_FINI_BREADCRUMB 0
+#define CHILD_GO_BB 1
+#define CHILD_GO_FINI_BREADCRUMB 0
+static int emit_bb_start_parent_no_preempt_mid_batch(struct i915_request *rq,
+ u64 offset, u32 len,
+ const unsigned int flags)
+{
+ struct intel_context *ce = rq->context;
+ u32 *cs;
+ u8 i;
+
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ cs = intel_ring_begin(rq, 10 + 4 * ce->parallel.number_children);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
+
+ /* Wait on children */
+ for (i = 0; i < ce->parallel.number_children; ++i) {
+ *cs++ = (MI_SEMAPHORE_WAIT |
+ MI_SEMAPHORE_GLOBAL_GTT |
+ MI_SEMAPHORE_POLL |
+ MI_SEMAPHORE_SAD_EQ_SDD);
+ *cs++ = PARENT_GO_BB;
+ *cs++ = get_children_join_addr(ce, i);
+ *cs++ = 0;
+ }
+
+ /* Turn off preemption */
+ *cs++ = MI_ARB_ON_OFF | MI_ARB_DISABLE;
+ *cs++ = MI_NOOP;
+
+ /* Tell children go */
+ cs = gen8_emit_ggtt_write(cs,
+ CHILD_GO_BB,
+ get_children_go_addr(ce),
+ 0);
+
+ /* Jump to batch */
+ *cs++ = MI_BATCH_BUFFER_START_GEN8 |
+ (flags & I915_DISPATCH_SECURE ? 0 : BIT(8));
+ *cs++ = lower_32_bits(offset);
+ *cs++ = upper_32_bits(offset);
+ *cs++ = MI_NOOP;
+
+ intel_ring_advance(rq, cs);
+
+ return 0;
+}
+
+static int emit_bb_start_child_no_preempt_mid_batch(struct i915_request *rq,
+ u64 offset, u32 len,
+ const unsigned int flags)
+{
+ struct intel_context *ce = rq->context;
+ struct intel_context *parent = intel_context_to_parent(ce);
+ u32 *cs;
+
+ GEM_BUG_ON(!intel_context_is_child(ce));
+
+ cs = intel_ring_begin(rq, 12);
+ if (IS_ERR(cs))
+ return PTR_ERR(cs);
+
+ /* Signal parent */
+ cs = gen8_emit_ggtt_write(cs,
+ PARENT_GO_BB,
+ get_children_join_addr(parent,
+ ce->parallel.child_index),
+ 0);
+
+ /* Wait on parent for go */
+ *cs++ = (MI_SEMAPHORE_WAIT |
+ MI_SEMAPHORE_GLOBAL_GTT |
+ MI_SEMAPHORE_POLL |
+ MI_SEMAPHORE_SAD_EQ_SDD);
+ *cs++ = CHILD_GO_BB;
+ *cs++ = get_children_go_addr(parent);
+ *cs++ = 0;
+
+ /* Turn off preemption */
+ *cs++ = MI_ARB_ON_OFF | MI_ARB_DISABLE;
+
+ /* Jump to batch */
+ *cs++ = MI_BATCH_BUFFER_START_GEN8 |
+ (flags & I915_DISPATCH_SECURE ? 0 : BIT(8));
+ *cs++ = lower_32_bits(offset);
+ *cs++ = upper_32_bits(offset);
+
+ intel_ring_advance(rq, cs);
+
+ return 0;
+}
+
+static u32 *
+emit_fini_breadcrumb_parent_no_preempt_mid_batch(struct i915_request *rq,
+ u32 *cs)
+{
+ struct intel_context *ce = rq->context;
+ u8 i;
+
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ /* Wait on children */
+ for (i = 0; i < ce->parallel.number_children; ++i) {
+ *cs++ = (MI_SEMAPHORE_WAIT |
+ MI_SEMAPHORE_GLOBAL_GTT |
+ MI_SEMAPHORE_POLL |
+ MI_SEMAPHORE_SAD_EQ_SDD);
+ *cs++ = PARENT_GO_FINI_BREADCRUMB;
+ *cs++ = get_children_join_addr(ce, i);
+ *cs++ = 0;
+ }
+
+ /* Turn on preemption */
+ *cs++ = MI_ARB_ON_OFF | MI_ARB_ENABLE;
+ *cs++ = MI_NOOP;
+
+ /* Tell children go */
+ cs = gen8_emit_ggtt_write(cs,
+ CHILD_GO_FINI_BREADCRUMB,
+ get_children_go_addr(ce),
+ 0);
+
+ /* Emit fini breadcrumb */
+ cs = gen8_emit_ggtt_write(cs,
+ rq->fence.seqno,
+ i915_request_active_timeline(rq)->hwsp_offset,
+ 0);
+
+ /* User interrupt */
+ *cs++ = MI_USER_INTERRUPT;
+ *cs++ = MI_NOOP;
+
+ rq->tail = intel_ring_offset(rq, cs);
+
+ return cs;
+}
+
+static u32 *
+emit_fini_breadcrumb_child_no_preempt_mid_batch(struct i915_request *rq, u32 *cs)
+{
+ struct intel_context *ce = rq->context;
+ struct intel_context *parent = intel_context_to_parent(ce);
+
+ GEM_BUG_ON(!intel_context_is_child(ce));
+
+ /* Turn on preemption */
+ *cs++ = MI_ARB_ON_OFF | MI_ARB_ENABLE;
+ *cs++ = MI_NOOP;
+
+ /* Signal parent */
+ cs = gen8_emit_ggtt_write(cs,
+ PARENT_GO_FINI_BREADCRUMB,
+ get_children_join_addr(parent,
+ ce->parallel.child_index),
+ 0);
+
+ /* Wait parent on for go */
+ *cs++ = (MI_SEMAPHORE_WAIT |
+ MI_SEMAPHORE_GLOBAL_GTT |
+ MI_SEMAPHORE_POLL |
+ MI_SEMAPHORE_SAD_EQ_SDD);
+ *cs++ = CHILD_GO_FINI_BREADCRUMB;
+ *cs++ = get_children_go_addr(parent);
+ *cs++ = 0;
+
+ /* Emit fini breadcrumb */
+ cs = gen8_emit_ggtt_write(cs,
+ rq->fence.seqno,
+ i915_request_active_timeline(rq)->hwsp_offset,
+ 0);
+
+ /* User interrupt */
+ *cs++ = MI_USER_INTERRUPT;
+ *cs++ = MI_NOOP;
+
+ rq->tail = intel_ring_offset(rq, cs);
+
+ return cs;
+}
+
static struct intel_context *
guc_create_virtual(struct intel_engine_cs **siblings, unsigned int count,
unsigned long flags)
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 20/25] drm/i915: Multi-BB execbuf
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:20 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:20 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Allow multiple batch buffers to be submitted in a single execbuf IOCTL
after a context has been configured with the 'set_parallel' extension.
The number batches is implicit based on the contexts configuration.
This is implemented with a series of loops. First a loop is used to find
all the batches, a loop to pin all the HW contexts, a loop to create all
the requests, a loop to submit (emit BB start, etc...) all the requests,
a loop to tie the requests to the VMAs they touch, and finally a loop to
commit the requests to the backend.
A composite fence is also created for the generated requests to return
to the user and to stick in dma resv slots.
No behavior from the existing IOCTL should be changed aside from when
throttling because the ring for a context is full. In this situation,
i915 will now wait while holding the object locks. This change was done
because the code is much simpler to wait while holding the locks and we
believe there isn't a huge benefit of dropping these locks. If this
proves false we can restructure the code to drop the locks during the
wait.
IGT: https://patchwork.freedesktop.org/patch/447008/?series=93071&rev=1
media UMD: https://github.com/intel/media-driver/pull/1252
v2:
(Matthew Brost)
- Return proper error value if i915_request_create fails
v3:
(John Harrison)
- Add comment explaining create / add order loops + locking
- Update commit message explaining different in IOCTL behavior
- Line wrap some comments
- eb_add_request returns void
- Return -EINVAL rather triggering BUG_ON if cmd parser used
(Checkpatch)
- Check eb->batch_len[*current_batch]
v4:
(CI)
- Set batch len if passed if via execbuf args
- Call __i915_request_skip after __i915_request_commit
(Kernel test robot)
- Initialize rq to NULL in eb_pin_timeline
v5:
(John Harrison)
- Fix typo in comments near bb order loops
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
.../gpu/drm/i915/gem/i915_gem_execbuffer.c | 783 ++++++++++++------
drivers/gpu/drm/i915/gt/intel_context.h | 8 +-
drivers/gpu/drm/i915/gt/intel_context_types.h | 10 +
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +
drivers/gpu/drm/i915/i915_request.h | 9 +
drivers/gpu/drm/i915/i915_vma.c | 21 +-
drivers/gpu/drm/i915/i915_vma.h | 13 +-
7 files changed, 595 insertions(+), 251 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index c75afc8784e3..fc30856e81fa 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -246,17 +246,25 @@ struct i915_execbuffer {
struct drm_i915_gem_exec_object2 *exec; /** ioctl execobj[] */
struct eb_vma *vma;
- struct intel_engine_cs *engine; /** engine to queue the request to */
+ struct intel_gt *gt; /* gt for the execbuf */
struct intel_context *context; /* logical state for the request */
struct i915_gem_context *gem_context; /** caller's context */
- struct i915_request *request; /** our request to build */
- struct eb_vma *batch; /** identity of the batch obj/vma */
+ /** our requests to build */
+ struct i915_request *requests[MAX_ENGINE_INSTANCE + 1];
+ /** identity of the batch obj/vma */
+ struct eb_vma *batches[MAX_ENGINE_INSTANCE + 1];
struct i915_vma *trampoline; /** trampoline used for chaining */
+ /** used for excl fence in dma_resv objects when > 1 BB submitted */
+ struct dma_fence *composite_fence;
+
/** actual size of execobj[] as we may extend it for the cmdparser */
unsigned int buffer_count;
+ /* number of batches in execbuf IOCTL */
+ unsigned int num_batches;
+
/** list of vma not yet bound during reservation phase */
struct list_head unbound;
@@ -283,7 +291,8 @@ struct i915_execbuffer {
u64 invalid_flags; /** Set of execobj.flags that are invalid */
- u64 batch_len; /** Length of batch within object */
+ /** Length of batch within object */
+ u64 batch_len[MAX_ENGINE_INSTANCE + 1];
u32 batch_start_offset; /** Location within object of batch */
u32 batch_flags; /** Flags composed for emit_bb_start() */
struct intel_gt_buffer_pool_node *batch_pool; /** pool node for batch buffer */
@@ -301,14 +310,13 @@ struct i915_execbuffer {
};
static int eb_parse(struct i915_execbuffer *eb);
-static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb,
- bool throttle);
+static int eb_pin_engine(struct i915_execbuffer *eb, bool throttle);
static void eb_unpin_engine(struct i915_execbuffer *eb);
static inline bool eb_use_cmdparser(const struct i915_execbuffer *eb)
{
- return intel_engine_requires_cmd_parser(eb->engine) ||
- (intel_engine_using_cmd_parser(eb->engine) &&
+ return intel_engine_requires_cmd_parser(eb->context->engine) ||
+ (intel_engine_using_cmd_parser(eb->context->engine) &&
eb->args->batch_len);
}
@@ -546,11 +554,21 @@ eb_validate_vma(struct i915_execbuffer *eb,
return 0;
}
-static void
+static inline bool
+is_batch_buffer(struct i915_execbuffer *eb, unsigned int buffer_idx)
+{
+ return eb->args->flags & I915_EXEC_BATCH_FIRST ?
+ buffer_idx < eb->num_batches :
+ buffer_idx >= eb->args->buffer_count - eb->num_batches;
+}
+
+static int
eb_add_vma(struct i915_execbuffer *eb,
- unsigned int i, unsigned batch_idx,
+ unsigned int *current_batch,
+ unsigned int i,
struct i915_vma *vma)
{
+ struct drm_i915_private *i915 = eb->i915;
struct drm_i915_gem_exec_object2 *entry = &eb->exec[i];
struct eb_vma *ev = &eb->vma[i];
@@ -577,15 +595,43 @@ eb_add_vma(struct i915_execbuffer *eb,
* Note that actual hangs have only been observed on gen7, but for
* paranoia do it everywhere.
*/
- if (i == batch_idx) {
+ if (is_batch_buffer(eb, i)) {
if (entry->relocation_count &&
!(ev->flags & EXEC_OBJECT_PINNED))
ev->flags |= __EXEC_OBJECT_NEEDS_BIAS;
if (eb->reloc_cache.has_fence)
ev->flags |= EXEC_OBJECT_NEEDS_FENCE;
- eb->batch = ev;
+ eb->batches[*current_batch] = ev;
+
+ if (unlikely(ev->flags & EXEC_OBJECT_WRITE)) {
+ drm_dbg(&i915->drm,
+ "Attempting to use self-modifying batch buffer\n");
+ return -EINVAL;
+ }
+
+ if (range_overflows_t(u64,
+ eb->batch_start_offset,
+ eb->args->batch_len,
+ ev->vma->size)) {
+ drm_dbg(&i915->drm, "Attempting to use out-of-bounds batch\n");
+ return -EINVAL;
+ }
+
+ if (eb->args->batch_len == 0)
+ eb->batch_len[*current_batch] = ev->vma->size -
+ eb->batch_start_offset;
+ else
+ eb->batch_len[*current_batch] = eb->args->batch_len;
+ if (unlikely(eb->batch_len[*current_batch] == 0)) { /* impossible! */
+ drm_dbg(&i915->drm, "Invalid batch length\n");
+ return -EINVAL;
+ }
+
+ ++*current_batch;
}
+
+ return 0;
}
static inline int use_cpu_reloc(const struct reloc_cache *cache,
@@ -729,14 +775,6 @@ static int eb_reserve(struct i915_execbuffer *eb)
} while (1);
}
-static unsigned int eb_batch_index(const struct i915_execbuffer *eb)
-{
- if (eb->args->flags & I915_EXEC_BATCH_FIRST)
- return 0;
- else
- return eb->buffer_count - 1;
-}
-
static int eb_select_context(struct i915_execbuffer *eb)
{
struct i915_gem_context *ctx;
@@ -857,9 +895,7 @@ static struct i915_vma *eb_lookup_vma(struct i915_execbuffer *eb, u32 handle)
static int eb_lookup_vmas(struct i915_execbuffer *eb)
{
- struct drm_i915_private *i915 = eb->i915;
- unsigned int batch = eb_batch_index(eb);
- unsigned int i;
+ unsigned int i, current_batch = 0;
int err = 0;
INIT_LIST_HEAD(&eb->relocs);
@@ -879,7 +915,9 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
goto err;
}
- eb_add_vma(eb, i, batch, vma);
+ err = eb_add_vma(eb, ¤t_batch, i, vma);
+ if (err)
+ return err;
if (i915_gem_object_is_userptr(vma->obj)) {
err = i915_gem_object_userptr_submit_init(vma->obj);
@@ -902,26 +940,6 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
}
}
- if (unlikely(eb->batch->flags & EXEC_OBJECT_WRITE)) {
- drm_dbg(&i915->drm,
- "Attempting to use self-modifying batch buffer\n");
- return -EINVAL;
- }
-
- if (range_overflows_t(u64,
- eb->batch_start_offset, eb->batch_len,
- eb->batch->vma->size)) {
- drm_dbg(&i915->drm, "Attempting to use out-of-bounds batch\n");
- return -EINVAL;
- }
-
- if (eb->batch_len == 0)
- eb->batch_len = eb->batch->vma->size - eb->batch_start_offset;
- if (unlikely(eb->batch_len == 0)) { /* impossible! */
- drm_dbg(&i915->drm, "Invalid batch length\n");
- return -EINVAL;
- }
-
return 0;
err:
@@ -1654,8 +1672,7 @@ static int eb_reinit_userptr(struct i915_execbuffer *eb)
return 0;
}
-static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
- struct i915_request *rq)
+static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb)
{
bool have_copy = false;
struct eb_vma *ev;
@@ -1671,21 +1688,6 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
eb_release_vmas(eb, false);
i915_gem_ww_ctx_fini(&eb->ww);
- if (rq) {
- /* nonblocking is always false */
- if (i915_request_wait(rq, I915_WAIT_INTERRUPTIBLE,
- MAX_SCHEDULE_TIMEOUT) < 0) {
- i915_request_put(rq);
- rq = NULL;
-
- err = -EINTR;
- goto err_relock;
- }
-
- i915_request_put(rq);
- rq = NULL;
- }
-
/*
* We take 3 passes through the slowpatch.
*
@@ -1712,28 +1714,21 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
if (!err)
err = eb_reinit_userptr(eb);
-err_relock:
i915_gem_ww_ctx_init(&eb->ww, true);
if (err)
goto out;
/* reacquire the objects */
repeat_validate:
- rq = eb_pin_engine(eb, false);
- if (IS_ERR(rq)) {
- err = PTR_ERR(rq);
- rq = NULL;
+ err = eb_pin_engine(eb, false);
+ if (err)
goto err;
- }
-
- /* We didn't throttle, should be NULL */
- GEM_WARN_ON(rq);
err = eb_validate_vmas(eb);
if (err)
goto err;
- GEM_BUG_ON(!eb->batch);
+ GEM_BUG_ON(!eb->batches[0]);
list_for_each_entry(ev, &eb->relocs, reloc_link) {
if (!have_copy) {
@@ -1797,46 +1792,23 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
}
}
- if (rq)
- i915_request_put(rq);
-
return err;
}
static int eb_relocate_parse(struct i915_execbuffer *eb)
{
int err;
- struct i915_request *rq = NULL;
bool throttle = true;
retry:
- rq = eb_pin_engine(eb, throttle);
- if (IS_ERR(rq)) {
- err = PTR_ERR(rq);
- rq = NULL;
+ err = eb_pin_engine(eb, throttle);
+ if (err) {
if (err != -EDEADLK)
return err;
goto err;
}
- if (rq) {
- bool nonblock = eb->file->filp->f_flags & O_NONBLOCK;
-
- /* Need to drop all locks now for throttling, take slowpath */
- err = i915_request_wait(rq, I915_WAIT_INTERRUPTIBLE, 0);
- if (err == -ETIME) {
- if (nonblock) {
- err = -EWOULDBLOCK;
- i915_request_put(rq);
- goto err;
- }
- goto slow;
- }
- i915_request_put(rq);
- rq = NULL;
- }
-
/* only throttle once, even if we didn't need to throttle */
throttle = false;
@@ -1876,7 +1848,7 @@ static int eb_relocate_parse(struct i915_execbuffer *eb)
return err;
slow:
- err = eb_relocate_parse_slow(eb, rq);
+ err = eb_relocate_parse_slow(eb);
if (err)
/*
* If the user expects the execobject.offset and
@@ -1890,11 +1862,40 @@ static int eb_relocate_parse(struct i915_execbuffer *eb)
return err;
}
+/*
+ * Using two helper loops for the order of which requests / batches are created
+ * and added the to backend. Requests are created in order from the parent to
+ * the last child. Requests are added in the reverse order, from the last child
+ * to parent. This is done for locking reasons as the timeline lock is acquired
+ * during request creation and released when the request is added to the
+ * backend. To make lockdep happy (see intel_context_timeline_lock) this must be
+ * the ordering.
+ */
+#define for_each_batch_create_order(_eb, _i) \
+ for ((_i) = 0; (_i) < (_eb)->num_batches; ++(_i))
+#define for_each_batch_add_order(_eb, _i) \
+ BUILD_BUG_ON(!typecheck(int, _i)); \
+ for ((_i) = (_eb)->num_batches - 1; (_i) >= 0; --(_i))
+
+static struct i915_request *
+eb_find_first_request_added(struct i915_execbuffer *eb)
+{
+ int i;
+
+ for_each_batch_add_order(eb, i)
+ if (eb->requests[i])
+ return eb->requests[i];
+
+ GEM_BUG_ON("Request not found");
+
+ return NULL;
+}
+
static int eb_move_to_gpu(struct i915_execbuffer *eb)
{
const unsigned int count = eb->buffer_count;
unsigned int i = count;
- int err = 0;
+ int err = 0, j;
while (i--) {
struct eb_vma *ev = &eb->vma[i];
@@ -1907,11 +1908,17 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
if (flags & EXEC_OBJECT_CAPTURE) {
struct i915_capture_list *capture;
- capture = kmalloc(sizeof(*capture), GFP_KERNEL);
- if (capture) {
- capture->next = eb->request->capture_list;
- capture->vma = vma;
- eb->request->capture_list = capture;
+ for_each_batch_create_order(eb, j) {
+ if (!eb->requests[j])
+ break;
+
+ capture = kmalloc(sizeof(*capture), GFP_KERNEL);
+ if (capture) {
+ capture->next =
+ eb->requests[j]->capture_list;
+ capture->vma = vma;
+ eb->requests[j]->capture_list = capture;
+ }
}
}
@@ -1932,14 +1939,26 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
flags &= ~EXEC_OBJECT_ASYNC;
}
+ /* We only need to await on the first request */
if (err == 0 && !(flags & EXEC_OBJECT_ASYNC)) {
err = i915_request_await_object
- (eb->request, obj, flags & EXEC_OBJECT_WRITE);
+ (eb_find_first_request_added(eb), obj,
+ flags & EXEC_OBJECT_WRITE);
}
- if (err == 0)
- err = i915_vma_move_to_active(vma, eb->request,
- flags | __EXEC_OBJECT_NO_RESERVE);
+ for_each_batch_add_order(eb, j) {
+ if (err)
+ break;
+ if (!eb->requests[j])
+ continue;
+
+ err = _i915_vma_move_to_active(vma, eb->requests[j],
+ j ? NULL :
+ eb->composite_fence ?
+ eb->composite_fence :
+ &eb->requests[j]->fence,
+ flags | __EXEC_OBJECT_NO_RESERVE);
+ }
}
#ifdef CONFIG_MMU_NOTIFIER
@@ -1970,11 +1989,16 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
goto err_skip;
/* Unconditionally flush any chipset caches (for streaming writes). */
- intel_gt_chipset_flush(eb->engine->gt);
+ intel_gt_chipset_flush(eb->gt);
return 0;
err_skip:
- i915_request_set_error_once(eb->request, err);
+ for_each_batch_create_order(eb, j) {
+ if (!eb->requests[j])
+ break;
+
+ i915_request_set_error_once(eb->requests[j], err);
+ }
return err;
}
@@ -2069,14 +2093,17 @@ static int eb_parse(struct i915_execbuffer *eb)
int err;
if (!eb_use_cmdparser(eb)) {
- batch = eb_dispatch_secure(eb, eb->batch->vma);
+ batch = eb_dispatch_secure(eb, eb->batches[0]->vma);
if (IS_ERR(batch))
return PTR_ERR(batch);
goto secure_batch;
}
- len = eb->batch_len;
+ if (intel_context_is_parallel(eb->context))
+ return -EINVAL;
+
+ len = eb->batch_len[0];
if (!CMDPARSER_USES_GGTT(eb->i915)) {
/*
* ppGTT backed shadow buffers must be mapped RO, to prevent
@@ -2090,11 +2117,11 @@ static int eb_parse(struct i915_execbuffer *eb)
} else {
len += I915_CMD_PARSER_TRAMPOLINE_SIZE;
}
- if (unlikely(len < eb->batch_len)) /* last paranoid check of overflow */
+ if (unlikely(len < eb->batch_len[0])) /* last paranoid check of overflow */
return -EINVAL;
if (!pool) {
- pool = intel_gt_get_buffer_pool(eb->engine->gt, len,
+ pool = intel_gt_get_buffer_pool(eb->gt, len,
I915_MAP_WB);
if (IS_ERR(pool))
return PTR_ERR(pool);
@@ -2119,7 +2146,7 @@ static int eb_parse(struct i915_execbuffer *eb)
trampoline = shadow;
shadow = shadow_batch_pin(eb, pool->obj,
- &eb->engine->gt->ggtt->vm,
+ &eb->gt->ggtt->vm,
PIN_GLOBAL);
if (IS_ERR(shadow)) {
err = PTR_ERR(shadow);
@@ -2141,26 +2168,29 @@ static int eb_parse(struct i915_execbuffer *eb)
if (err)
goto err_trampoline;
- err = intel_engine_cmd_parser(eb->engine,
- eb->batch->vma,
+ err = intel_engine_cmd_parser(eb->context->engine,
+ eb->batches[0]->vma,
eb->batch_start_offset,
- eb->batch_len,
+ eb->batch_len[0],
shadow, trampoline);
if (err)
goto err_unpin_batch;
- eb->batch = &eb->vma[eb->buffer_count++];
- eb->batch->vma = i915_vma_get(shadow);
- eb->batch->flags = __EXEC_OBJECT_HAS_PIN;
+ eb->batches[0] = &eb->vma[eb->buffer_count++];
+ eb->batches[0]->vma = i915_vma_get(shadow);
+ eb->batches[0]->flags = __EXEC_OBJECT_HAS_PIN;
eb->trampoline = trampoline;
eb->batch_start_offset = 0;
secure_batch:
if (batch) {
- eb->batch = &eb->vma[eb->buffer_count++];
- eb->batch->flags = __EXEC_OBJECT_HAS_PIN;
- eb->batch->vma = i915_vma_get(batch);
+ if (intel_context_is_parallel(eb->context))
+ return -EINVAL;
+
+ eb->batches[0] = &eb->vma[eb->buffer_count++];
+ eb->batches[0]->flags = __EXEC_OBJECT_HAS_PIN;
+ eb->batches[0]->vma = i915_vma_get(batch);
}
return 0;
@@ -2176,19 +2206,18 @@ static int eb_parse(struct i915_execbuffer *eb)
return err;
}
-static int eb_submit(struct i915_execbuffer *eb, struct i915_vma *batch)
+static int eb_request_submit(struct i915_execbuffer *eb,
+ struct i915_request *rq,
+ struct i915_vma *batch,
+ u64 batch_len)
{
int err;
- if (intel_context_nopreempt(eb->context))
- __set_bit(I915_FENCE_FLAG_NOPREEMPT, &eb->request->fence.flags);
-
- err = eb_move_to_gpu(eb);
- if (err)
- return err;
+ if (intel_context_nopreempt(rq->context))
+ __set_bit(I915_FENCE_FLAG_NOPREEMPT, &rq->fence.flags);
if (eb->args->flags & I915_EXEC_GEN7_SOL_RESET) {
- err = i915_reset_gen7_sol_offsets(eb->request);
+ err = i915_reset_gen7_sol_offsets(rq);
if (err)
return err;
}
@@ -2199,26 +2228,26 @@ static int eb_submit(struct i915_execbuffer *eb, struct i915_vma *batch)
* allows us to determine if the batch is still waiting on the GPU
* or actually running by checking the breadcrumb.
*/
- if (eb->engine->emit_init_breadcrumb) {
- err = eb->engine->emit_init_breadcrumb(eb->request);
+ if (rq->context->engine->emit_init_breadcrumb) {
+ err = rq->context->engine->emit_init_breadcrumb(rq);
if (err)
return err;
}
- err = eb->engine->emit_bb_start(eb->request,
- batch->node.start +
- eb->batch_start_offset,
- eb->batch_len,
- eb->batch_flags);
+ err = rq->context->engine->emit_bb_start(rq,
+ batch->node.start +
+ eb->batch_start_offset,
+ batch_len,
+ eb->batch_flags);
if (err)
return err;
if (eb->trampoline) {
+ GEM_BUG_ON(intel_context_is_parallel(rq->context));
GEM_BUG_ON(eb->batch_start_offset);
- err = eb->engine->emit_bb_start(eb->request,
- eb->trampoline->node.start +
- eb->batch_len,
- 0, 0);
+ err = rq->context->engine->emit_bb_start(rq,
+ eb->trampoline->node.start +
+ batch_len, 0, 0);
if (err)
return err;
}
@@ -2226,6 +2255,27 @@ static int eb_submit(struct i915_execbuffer *eb, struct i915_vma *batch)
return 0;
}
+static int eb_submit(struct i915_execbuffer *eb)
+{
+ unsigned int i;
+ int err;
+
+ err = eb_move_to_gpu(eb);
+
+ for_each_batch_create_order(eb, i) {
+ if (!eb->requests[i])
+ break;
+
+ trace_i915_request_queue(eb->requests[i], eb->batch_flags);
+ if (!err)
+ err = eb_request_submit(eb, eb->requests[i],
+ eb->batches[i]->vma,
+ eb->batch_len[i]);
+ }
+
+ return err;
+}
+
static int num_vcs_engines(const struct drm_i915_private *i915)
{
return hweight_long(VDBOX_MASK(&i915->gt));
@@ -2291,26 +2341,11 @@ static struct i915_request *eb_throttle(struct i915_execbuffer *eb, struct intel
return i915_request_get(rq);
}
-static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb, bool throttle)
+static int eb_pin_timeline(struct i915_execbuffer *eb, struct intel_context *ce,
+ bool throttle)
{
- struct intel_context *ce = eb->context;
struct intel_timeline *tl;
struct i915_request *rq = NULL;
- int err;
-
- GEM_BUG_ON(eb->args->flags & __EXEC_ENGINE_PINNED);
-
- if (unlikely(intel_context_is_banned(ce)))
- return ERR_PTR(-EIO);
-
- /*
- * Pinning the contexts may generate requests in order to acquire
- * GGTT space, so do this first before we reserve a seqno for
- * ourselves.
- */
- err = intel_context_pin_ww(ce, &eb->ww);
- if (err)
- return ERR_PTR(err);
/*
* Take a local wakeref for preparing to dispatch the execbuf as
@@ -2321,33 +2356,108 @@ static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb, bool throt
* taken on the engine, and the parent device.
*/
tl = intel_context_timeline_lock(ce);
- if (IS_ERR(tl)) {
- intel_context_unpin(ce);
- return ERR_CAST(tl);
- }
+ if (IS_ERR(tl))
+ return PTR_ERR(tl);
intel_context_enter(ce);
if (throttle)
rq = eb_throttle(eb, ce);
intel_context_timeline_unlock(tl);
+ if (rq) {
+ bool nonblock = eb->file->filp->f_flags & O_NONBLOCK;
+ long timeout = nonblock ? 0 : MAX_SCHEDULE_TIMEOUT;
+
+ if (i915_request_wait(rq, I915_WAIT_INTERRUPTIBLE,
+ timeout) < 0) {
+ i915_request_put(rq);
+
+ tl = intel_context_timeline_lock(ce);
+ intel_context_exit(ce);
+ intel_context_timeline_unlock(tl);
+
+ if (nonblock)
+ return -EWOULDBLOCK;
+ else
+ return -EINTR;
+ }
+ i915_request_put(rq);
+ }
+
+ return 0;
+}
+
+static int eb_pin_engine(struct i915_execbuffer *eb, bool throttle)
+{
+ struct intel_context *ce = eb->context, *child;
+ int err;
+ int i = 0, j = 0;
+
+ GEM_BUG_ON(eb->args->flags & __EXEC_ENGINE_PINNED);
+
+ if (unlikely(intel_context_is_banned(ce)))
+ return -EIO;
+
+ /*
+ * Pinning the contexts may generate requests in order to acquire
+ * GGTT space, so do this first before we reserve a seqno for
+ * ourselves.
+ */
+ err = intel_context_pin_ww(ce, &eb->ww);
+ if (err)
+ return err;
+ for_each_child(ce, child) {
+ err = intel_context_pin_ww(child, &eb->ww);
+ GEM_BUG_ON(err); /* perma-pinned should incr a counter */
+ }
+
+ for_each_child(ce, child) {
+ err = eb_pin_timeline(eb, child, throttle);
+ if (err)
+ goto unwind;
+ ++i;
+ }
+ err = eb_pin_timeline(eb, ce, throttle);
+ if (err)
+ goto unwind;
+
eb->args->flags |= __EXEC_ENGINE_PINNED;
- return rq;
+ return 0;
+
+unwind:
+ for_each_child(ce, child) {
+ if (j++ < i) {
+ mutex_lock(&child->timeline->mutex);
+ intel_context_exit(child);
+ mutex_unlock(&child->timeline->mutex);
+ }
+ }
+ for_each_child(ce, child)
+ intel_context_unpin(child);
+ intel_context_unpin(ce);
+ return err;
}
static void eb_unpin_engine(struct i915_execbuffer *eb)
{
- struct intel_context *ce = eb->context;
- struct intel_timeline *tl = ce->timeline;
+ struct intel_context *ce = eb->context, *child;
if (!(eb->args->flags & __EXEC_ENGINE_PINNED))
return;
eb->args->flags &= ~__EXEC_ENGINE_PINNED;
- mutex_lock(&tl->mutex);
+ for_each_child(ce, child) {
+ mutex_lock(&child->timeline->mutex);
+ intel_context_exit(child);
+ mutex_unlock(&child->timeline->mutex);
+
+ intel_context_unpin(child);
+ }
+
+ mutex_lock(&ce->timeline->mutex);
intel_context_exit(ce);
- mutex_unlock(&tl->mutex);
+ mutex_unlock(&ce->timeline->mutex);
intel_context_unpin(ce);
}
@@ -2398,7 +2508,7 @@ eb_select_legacy_ring(struct i915_execbuffer *eb)
static int
eb_select_engine(struct i915_execbuffer *eb)
{
- struct intel_context *ce;
+ struct intel_context *ce, *child;
unsigned int idx;
int err;
@@ -2411,6 +2521,20 @@ eb_select_engine(struct i915_execbuffer *eb)
if (IS_ERR(ce))
return PTR_ERR(ce);
+ if (intel_context_is_parallel(ce)) {
+ if (eb->buffer_count < ce->parallel.number_children + 1) {
+ intel_context_put(ce);
+ return -EINVAL;
+ }
+ if (eb->batch_start_offset || eb->args->batch_len) {
+ intel_context_put(ce);
+ return -EINVAL;
+ }
+ }
+ eb->num_batches = ce->parallel.number_children + 1;
+
+ for_each_child(ce, child)
+ intel_context_get(child);
intel_gt_pm_get(ce->engine->gt);
if (!test_bit(CONTEXT_ALLOC_BIT, &ce->flags)) {
@@ -2418,6 +2542,13 @@ eb_select_engine(struct i915_execbuffer *eb)
if (err)
goto err;
}
+ for_each_child(ce, child) {
+ if (!test_bit(CONTEXT_ALLOC_BIT, &child->flags)) {
+ err = intel_context_alloc_state(child);
+ if (err)
+ goto err;
+ }
+ }
/*
* ABI: Before userspace accesses the GPU (e.g. execbuffer), report
@@ -2428,7 +2559,7 @@ eb_select_engine(struct i915_execbuffer *eb)
goto err;
eb->context = ce;
- eb->engine = ce->engine;
+ eb->gt = ce->engine->gt;
/*
* Make sure engine pool stays alive even if we call intel_context_put
@@ -2439,6 +2570,8 @@ eb_select_engine(struct i915_execbuffer *eb)
err:
intel_gt_pm_put(ce->engine->gt);
+ for_each_child(ce, child)
+ intel_context_put(child);
intel_context_put(ce);
return err;
}
@@ -2446,7 +2579,11 @@ eb_select_engine(struct i915_execbuffer *eb)
static void
eb_put_engine(struct i915_execbuffer *eb)
{
- intel_gt_pm_put(eb->engine->gt);
+ struct intel_context *child;
+
+ intel_gt_pm_put(eb->gt);
+ for_each_child(eb->context, child)
+ intel_context_put(child);
intel_context_put(eb->context);
}
@@ -2669,7 +2806,8 @@ static void put_fence_array(struct eb_fence *fences, int num_fences)
}
static int
-await_fence_array(struct i915_execbuffer *eb)
+await_fence_array(struct i915_execbuffer *eb,
+ struct i915_request *rq)
{
unsigned int n;
int err;
@@ -2683,8 +2821,7 @@ await_fence_array(struct i915_execbuffer *eb)
if (!eb->fences[n].dma_fence)
continue;
- err = i915_request_await_dma_fence(eb->request,
- eb->fences[n].dma_fence);
+ err = i915_request_await_dma_fence(rq, eb->fences[n].dma_fence);
if (err < 0)
return err;
}
@@ -2692,9 +2829,9 @@ await_fence_array(struct i915_execbuffer *eb)
return 0;
}
-static void signal_fence_array(const struct i915_execbuffer *eb)
+static void signal_fence_array(const struct i915_execbuffer *eb,
+ struct dma_fence * const fence)
{
- struct dma_fence * const fence = &eb->request->fence;
unsigned int n;
for (n = 0; n < eb->num_fences; n++) {
@@ -2742,9 +2879,9 @@ static void retire_requests(struct intel_timeline *tl, struct i915_request *end)
break;
}
-static int eb_request_add(struct i915_execbuffer *eb, int err)
+static int eb_request_add(struct i915_execbuffer *eb, struct i915_request *rq,
+ int err, bool last_parallel)
{
- struct i915_request *rq = eb->request;
struct intel_timeline * const tl = i915_request_timeline(rq);
struct i915_sched_attr attr = {};
struct i915_request *prev;
@@ -2766,6 +2903,17 @@ static int eb_request_add(struct i915_execbuffer *eb, int err)
err = -ENOENT; /* override any transient errors */
}
+ if (intel_context_is_parallel(eb->context)) {
+ if (err) {
+ __i915_request_skip(rq);
+ set_bit(I915_FENCE_FLAG_SKIP_PARALLEL,
+ &rq->fence.flags);
+ }
+ if (last_parallel)
+ set_bit(I915_FENCE_FLAG_SUBMIT_PARALLEL,
+ &rq->fence.flags);
+ }
+
__i915_request_queue(rq, &attr);
/* Try to clean up the client's timeline after submitting the request */
@@ -2777,6 +2925,25 @@ static int eb_request_add(struct i915_execbuffer *eb, int err)
return err;
}
+static int eb_requests_add(struct i915_execbuffer *eb, int err)
+{
+ int i;
+
+ /*
+ * We iterate in reverse order of creation to release timeline mutexes in
+ * same order.
+ */
+ for_each_batch_add_order(eb, i) {
+ struct i915_request *rq = eb->requests[i];
+
+ if (!rq)
+ continue;
+ err |= eb_request_add(eb, rq, err, i == 0);
+ }
+
+ return err;
+}
+
static const i915_user_extension_fn execbuf_extensions[] = {
[DRM_I915_GEM_EXECBUFFER_EXT_TIMELINE_FENCES] = parse_timeline_fences,
};
@@ -2803,6 +2970,182 @@ parse_execbuf2_extensions(struct drm_i915_gem_execbuffer2 *args,
eb);
}
+static void eb_requests_get(struct i915_execbuffer *eb)
+{
+ unsigned int i;
+
+ for_each_batch_create_order(eb, i) {
+ if (!eb->requests[i])
+ break;
+
+ i915_request_get(eb->requests[i]);
+ }
+}
+
+static void eb_requests_put(struct i915_execbuffer *eb)
+{
+ unsigned int i;
+
+ for_each_batch_create_order(eb, i) {
+ if (!eb->requests[i])
+ break;
+
+ i915_request_put(eb->requests[i]);
+ }
+}
+
+static struct sync_file *
+eb_composite_fence_create(struct i915_execbuffer *eb, int out_fence_fd)
+{
+ struct sync_file *out_fence = NULL;
+ struct dma_fence_array *fence_array;
+ struct dma_fence **fences;
+ unsigned int i;
+
+ GEM_BUG_ON(!intel_context_is_parent(eb->context));
+
+ fences = kmalloc_array(eb->num_batches, sizeof(*fences), GFP_KERNEL);
+ if (!fences)
+ return ERR_PTR(-ENOMEM);
+
+ for_each_batch_create_order(eb, i)
+ fences[i] = &eb->requests[i]->fence;
+
+ fence_array = dma_fence_array_create(eb->num_batches,
+ fences,
+ eb->context->parallel.fence_context,
+ eb->context->parallel.seqno,
+ false);
+ if (!fence_array) {
+ kfree(fences);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ /* Move ownership to the dma_fence_array created above */
+ for_each_batch_create_order(eb, i)
+ dma_fence_get(fences[i]);
+
+ if (out_fence_fd != -1) {
+ out_fence = sync_file_create(&fence_array->base);
+ /* sync_file now owns fence_arry, drop creation ref */
+ dma_fence_put(&fence_array->base);
+ if (!out_fence)
+ return ERR_PTR(-ENOMEM);
+ }
+
+ eb->composite_fence = &fence_array->base;
+
+ return out_fence;
+}
+
+static struct sync_file *
+eb_fences_add(struct i915_execbuffer *eb, struct i915_request *rq,
+ struct dma_fence *in_fence, int out_fence_fd)
+{
+ struct sync_file *out_fence = NULL;
+ int err;
+
+ if (unlikely(eb->gem_context->syncobj)) {
+ struct dma_fence *fence;
+
+ fence = drm_syncobj_fence_get(eb->gem_context->syncobj);
+ err = i915_request_await_dma_fence(rq, fence);
+ dma_fence_put(fence);
+ if (err)
+ return ERR_PTR(err);
+ }
+
+ if (in_fence) {
+ if (eb->args->flags & I915_EXEC_FENCE_SUBMIT)
+ err = i915_request_await_execution(rq, in_fence);
+ else
+ err = i915_request_await_dma_fence(rq, in_fence);
+ if (err < 0)
+ return ERR_PTR(err);
+ }
+
+ if (eb->fences) {
+ err = await_fence_array(eb, rq);
+ if (err)
+ return ERR_PTR(err);
+ }
+
+ if (intel_context_is_parallel(eb->context)) {
+ out_fence = eb_composite_fence_create(eb, out_fence_fd);
+ if (IS_ERR(out_fence))
+ return ERR_PTR(-ENOMEM);
+ } else if (out_fence_fd != -1) {
+ out_fence = sync_file_create(&rq->fence);
+ if (!out_fence)
+ return ERR_PTR(-ENOMEM);
+ }
+
+ return out_fence;
+}
+
+static struct intel_context *
+eb_find_context(struct i915_execbuffer *eb, unsigned int context_number)
+{
+ struct intel_context *child;
+
+ if (likely(context_number == 0))
+ return eb->context;
+
+ for_each_child(eb->context, child)
+ if (!--context_number)
+ return child;
+
+ GEM_BUG_ON("Context not found");
+
+ return NULL;
+}
+
+static struct sync_file *
+eb_requests_create(struct i915_execbuffer *eb, struct dma_fence *in_fence,
+ int out_fence_fd)
+{
+ struct sync_file *out_fence = NULL;
+ unsigned int i;
+
+ for_each_batch_create_order(eb, i) {
+ /* Allocate a request for this batch buffer nice and early. */
+ eb->requests[i] = i915_request_create(eb_find_context(eb, i));
+ if (IS_ERR(eb->requests[i])) {
+ out_fence = ERR_PTR(PTR_ERR(eb->requests[i]));
+ eb->requests[i] = NULL;
+ return out_fence;
+ }
+
+ /*
+ * Only the first request added (committed to backend) has to
+ * take the in fences into account as all subsequent requests
+ * will have fences inserted inbetween them.
+ */
+ if (i + 1 == eb->num_batches) {
+ out_fence = eb_fences_add(eb, eb->requests[i],
+ in_fence, out_fence_fd);
+ if (IS_ERR(out_fence))
+ return out_fence;
+ }
+
+ /*
+ * Whilst this request exists, batch_obj will be on the
+ * active_list, and so will hold the active reference. Only when
+ * this request is retired will the batch_obj be moved onto
+ * the inactive_list and lose its active reference. Hence we do
+ * not need to explicitly hold another reference here.
+ */
+ eb->requests[i]->batch = eb->batches[i]->vma;
+ if (eb->batch_pool) {
+ GEM_BUG_ON(intel_context_is_parallel(eb->context));
+ intel_gt_buffer_pool_mark_active(eb->batch_pool,
+ eb->requests[i]);
+ }
+ }
+
+ return out_fence;
+}
+
static int
i915_gem_do_execbuffer(struct drm_device *dev,
struct drm_file *file,
@@ -2813,7 +3156,6 @@ i915_gem_do_execbuffer(struct drm_device *dev,
struct i915_execbuffer eb;
struct dma_fence *in_fence = NULL;
struct sync_file *out_fence = NULL;
- struct i915_vma *batch;
int out_fence_fd = -1;
int err;
@@ -2837,12 +3179,15 @@ i915_gem_do_execbuffer(struct drm_device *dev,
eb.buffer_count = args->buffer_count;
eb.batch_start_offset = args->batch_start_offset;
- eb.batch_len = args->batch_len;
eb.trampoline = NULL;
eb.fences = NULL;
eb.num_fences = 0;
+ memset(eb.requests, 0, sizeof(struct i915_request *) *
+ ARRAY_SIZE(eb.requests));
+ eb.composite_fence = NULL;
+
eb.batch_flags = 0;
if (args->flags & I915_EXEC_SECURE) {
if (GRAPHICS_VER(i915) >= 11)
@@ -2926,70 +3271,25 @@ i915_gem_do_execbuffer(struct drm_device *dev,
ww_acquire_done(&eb.ww.ctx);
- batch = eb.batch->vma;
-
- /* Allocate a request for this batch buffer nice and early. */
- eb.request = i915_request_create(eb.context);
- if (IS_ERR(eb.request)) {
- err = PTR_ERR(eb.request);
- goto err_vma;
- }
-
- if (unlikely(eb.gem_context->syncobj)) {
- struct dma_fence *fence;
-
- fence = drm_syncobj_fence_get(eb.gem_context->syncobj);
- err = i915_request_await_dma_fence(eb.request, fence);
- dma_fence_put(fence);
- if (err)
- goto err_ext;
- }
-
- if (in_fence) {
- if (args->flags & I915_EXEC_FENCE_SUBMIT)
- err = i915_request_await_execution(eb.request,
- in_fence);
- else
- err = i915_request_await_dma_fence(eb.request,
- in_fence);
- if (err < 0)
- goto err_request;
- }
-
- if (eb.fences) {
- err = await_fence_array(&eb);
- if (err)
+ out_fence = eb_requests_create(&eb, in_fence, out_fence_fd);
+ if (IS_ERR(out_fence)) {
+ err = PTR_ERR(out_fence);
+ if (eb.requests[0])
goto err_request;
+ else
+ goto err_vma;
}
- if (out_fence_fd != -1) {
- out_fence = sync_file_create(&eb.request->fence);
- if (!out_fence) {
- err = -ENOMEM;
- goto err_request;
- }
- }
-
- /*
- * Whilst this request exists, batch_obj will be on the
- * active_list, and so will hold the active reference. Only when this
- * request is retired will the the batch_obj be moved onto the
- * inactive_list and lose its active reference. Hence we do not need
- * to explicitly hold another reference here.
- */
- eb.request->batch = batch;
- if (eb.batch_pool)
- intel_gt_buffer_pool_mark_active(eb.batch_pool, eb.request);
-
- trace_i915_request_queue(eb.request, eb.batch_flags);
- err = eb_submit(&eb, batch);
+ err = eb_submit(&eb);
err_request:
- i915_request_get(eb.request);
- err = eb_request_add(&eb, err);
+ eb_requests_get(&eb);
+ err = eb_requests_add(&eb, err);
if (eb.fences)
- signal_fence_array(&eb);
+ signal_fence_array(&eb, eb.composite_fence ?
+ eb.composite_fence :
+ &eb.requests[0]->fence);
if (out_fence) {
if (err == 0) {
@@ -3004,10 +3304,15 @@ i915_gem_do_execbuffer(struct drm_device *dev,
if (unlikely(eb.gem_context->syncobj)) {
drm_syncobj_replace_fence(eb.gem_context->syncobj,
- &eb.request->fence);
+ eb.composite_fence ?
+ eb.composite_fence :
+ &eb.requests[0]->fence);
}
- i915_request_put(eb.request);
+ if (!out_fence && eb.composite_fence)
+ dma_fence_put(eb.composite_fence);
+
+ eb_requests_put(&eb);
err_vma:
eb_release_vmas(&eb, true);
diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
index edf12caaade3..246c37d72cd7 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.h
+++ b/drivers/gpu/drm/i915/gt/intel_context.h
@@ -241,7 +241,13 @@ intel_context_timeline_lock(struct intel_context *ce)
struct intel_timeline *tl = ce->timeline;
int err;
- err = mutex_lock_interruptible(&tl->mutex);
+ if (intel_context_is_parent(ce))
+ err = mutex_lock_interruptible_nested(&tl->mutex, 0);
+ else if (intel_context_is_child(ce))
+ err = mutex_lock_interruptible_nested(&tl->mutex,
+ ce->parallel.child_index + 1);
+ else
+ err = mutex_lock_interruptible(&tl->mutex);
if (err)
return ERR_PTR(err);
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 95a5b94b4ece..9e0177dc5484 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -248,6 +248,16 @@ struct intel_context {
* context
*/
struct i915_request *last_rq;
+ /**
+ * @fence_context: fence context composite fence when doing
+ * parallel submission
+ */
+ u64 fence_context;
+ /**
+ * @seqno: seqno for composite fence when doing parallel
+ * submission
+ */
+ u32 seqno;
/** @number_children: number of children if parent */
u8 number_children;
/** @child_index: index into child_list if child */
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 50f0f4eba03b..361fab2cae99 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -3093,6 +3093,8 @@ guc_create_parallel(struct intel_engine_cs **engines,
}
}
+ parent->parallel.fence_context = dma_fence_context_alloc(1);
+
parent->engine->emit_bb_start =
emit_bb_start_parent_no_preempt_mid_batch;
parent->engine->emit_fini_breadcrumb =
diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
index 8950785e55d6..24db8459376b 100644
--- a/drivers/gpu/drm/i915/i915_request.h
+++ b/drivers/gpu/drm/i915/i915_request.h
@@ -147,6 +147,15 @@ enum {
* tail.
*/
I915_FENCE_FLAG_SUBMIT_PARALLEL,
+
+ /*
+ * I915_FENCE_FLAG_SKIP_PARALLEL - request with a context in a
+ * parent-child relationship (parallel submission, multi-lrc) that
+ * hit an error while generating requests in the execbuf IOCTL.
+ * Indicates this request should be skipped as another request in
+ * submission / relationship encoutered an error.
+ */
+ I915_FENCE_FLAG_SKIP_PARALLEL,
};
/**
diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c
index 4b7fc4647e46..90546fa58fc1 100644
--- a/drivers/gpu/drm/i915/i915_vma.c
+++ b/drivers/gpu/drm/i915/i915_vma.c
@@ -1234,9 +1234,10 @@ int __i915_vma_move_to_active(struct i915_vma *vma, struct i915_request *rq)
return i915_active_add_request(&vma->active, rq);
}
-int i915_vma_move_to_active(struct i915_vma *vma,
- struct i915_request *rq,
- unsigned int flags)
+int _i915_vma_move_to_active(struct i915_vma *vma,
+ struct i915_request *rq,
+ struct dma_fence *fence,
+ unsigned int flags)
{
struct drm_i915_gem_object *obj = vma->obj;
int err;
@@ -1257,9 +1258,11 @@ int i915_vma_move_to_active(struct i915_vma *vma,
intel_frontbuffer_put(front);
}
- dma_resv_add_excl_fence(vma->resv, &rq->fence);
- obj->write_domain = I915_GEM_DOMAIN_RENDER;
- obj->read_domains = 0;
+ if (fence) {
+ dma_resv_add_excl_fence(vma->resv, fence);
+ obj->write_domain = I915_GEM_DOMAIN_RENDER;
+ obj->read_domains = 0;
+ }
} else {
if (!(flags & __EXEC_OBJECT_NO_RESERVE)) {
err = dma_resv_reserve_shared(vma->resv, 1);
@@ -1267,8 +1270,10 @@ int i915_vma_move_to_active(struct i915_vma *vma,
return err;
}
- dma_resv_add_shared_fence(vma->resv, &rq->fence);
- obj->write_domain = 0;
+ if (fence) {
+ dma_resv_add_shared_fence(vma->resv, fence);
+ obj->write_domain = 0;
+ }
}
if (flags & EXEC_OBJECT_NEEDS_FENCE && vma->fence)
diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vma.h
index ed69f66c7ab0..648dbe744c96 100644
--- a/drivers/gpu/drm/i915/i915_vma.h
+++ b/drivers/gpu/drm/i915/i915_vma.h
@@ -57,9 +57,16 @@ static inline bool i915_vma_is_active(const struct i915_vma *vma)
int __must_check __i915_vma_move_to_active(struct i915_vma *vma,
struct i915_request *rq);
-int __must_check i915_vma_move_to_active(struct i915_vma *vma,
- struct i915_request *rq,
- unsigned int flags);
+int __must_check _i915_vma_move_to_active(struct i915_vma *vma,
+ struct i915_request *rq,
+ struct dma_fence *fence,
+ unsigned int flags);
+static inline int __must_check
+i915_vma_move_to_active(struct i915_vma *vma, struct i915_request *rq,
+ unsigned int flags)
+{
+ return _i915_vma_move_to_active(vma, rq, &rq->fence, flags);
+}
#define __i915_vma_flags(v) ((unsigned long *)&(v)->flags.counter)
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 20/25] drm/i915: Multi-BB execbuf
@ 2021-10-14 17:20 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:20 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Allow multiple batch buffers to be submitted in a single execbuf IOCTL
after a context has been configured with the 'set_parallel' extension.
The number batches is implicit based on the contexts configuration.
This is implemented with a series of loops. First a loop is used to find
all the batches, a loop to pin all the HW contexts, a loop to create all
the requests, a loop to submit (emit BB start, etc...) all the requests,
a loop to tie the requests to the VMAs they touch, and finally a loop to
commit the requests to the backend.
A composite fence is also created for the generated requests to return
to the user and to stick in dma resv slots.
No behavior from the existing IOCTL should be changed aside from when
throttling because the ring for a context is full. In this situation,
i915 will now wait while holding the object locks. This change was done
because the code is much simpler to wait while holding the locks and we
believe there isn't a huge benefit of dropping these locks. If this
proves false we can restructure the code to drop the locks during the
wait.
IGT: https://patchwork.freedesktop.org/patch/447008/?series=93071&rev=1
media UMD: https://github.com/intel/media-driver/pull/1252
v2:
(Matthew Brost)
- Return proper error value if i915_request_create fails
v3:
(John Harrison)
- Add comment explaining create / add order loops + locking
- Update commit message explaining different in IOCTL behavior
- Line wrap some comments
- eb_add_request returns void
- Return -EINVAL rather triggering BUG_ON if cmd parser used
(Checkpatch)
- Check eb->batch_len[*current_batch]
v4:
(CI)
- Set batch len if passed if via execbuf args
- Call __i915_request_skip after __i915_request_commit
(Kernel test robot)
- Initialize rq to NULL in eb_pin_timeline
v5:
(John Harrison)
- Fix typo in comments near bb order loops
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
.../gpu/drm/i915/gem/i915_gem_execbuffer.c | 783 ++++++++++++------
drivers/gpu/drm/i915/gt/intel_context.h | 8 +-
drivers/gpu/drm/i915/gt/intel_context_types.h | 10 +
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +
drivers/gpu/drm/i915/i915_request.h | 9 +
drivers/gpu/drm/i915/i915_vma.c | 21 +-
drivers/gpu/drm/i915/i915_vma.h | 13 +-
7 files changed, 595 insertions(+), 251 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index c75afc8784e3..fc30856e81fa 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -246,17 +246,25 @@ struct i915_execbuffer {
struct drm_i915_gem_exec_object2 *exec; /** ioctl execobj[] */
struct eb_vma *vma;
- struct intel_engine_cs *engine; /** engine to queue the request to */
+ struct intel_gt *gt; /* gt for the execbuf */
struct intel_context *context; /* logical state for the request */
struct i915_gem_context *gem_context; /** caller's context */
- struct i915_request *request; /** our request to build */
- struct eb_vma *batch; /** identity of the batch obj/vma */
+ /** our requests to build */
+ struct i915_request *requests[MAX_ENGINE_INSTANCE + 1];
+ /** identity of the batch obj/vma */
+ struct eb_vma *batches[MAX_ENGINE_INSTANCE + 1];
struct i915_vma *trampoline; /** trampoline used for chaining */
+ /** used for excl fence in dma_resv objects when > 1 BB submitted */
+ struct dma_fence *composite_fence;
+
/** actual size of execobj[] as we may extend it for the cmdparser */
unsigned int buffer_count;
+ /* number of batches in execbuf IOCTL */
+ unsigned int num_batches;
+
/** list of vma not yet bound during reservation phase */
struct list_head unbound;
@@ -283,7 +291,8 @@ struct i915_execbuffer {
u64 invalid_flags; /** Set of execobj.flags that are invalid */
- u64 batch_len; /** Length of batch within object */
+ /** Length of batch within object */
+ u64 batch_len[MAX_ENGINE_INSTANCE + 1];
u32 batch_start_offset; /** Location within object of batch */
u32 batch_flags; /** Flags composed for emit_bb_start() */
struct intel_gt_buffer_pool_node *batch_pool; /** pool node for batch buffer */
@@ -301,14 +310,13 @@ struct i915_execbuffer {
};
static int eb_parse(struct i915_execbuffer *eb);
-static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb,
- bool throttle);
+static int eb_pin_engine(struct i915_execbuffer *eb, bool throttle);
static void eb_unpin_engine(struct i915_execbuffer *eb);
static inline bool eb_use_cmdparser(const struct i915_execbuffer *eb)
{
- return intel_engine_requires_cmd_parser(eb->engine) ||
- (intel_engine_using_cmd_parser(eb->engine) &&
+ return intel_engine_requires_cmd_parser(eb->context->engine) ||
+ (intel_engine_using_cmd_parser(eb->context->engine) &&
eb->args->batch_len);
}
@@ -546,11 +554,21 @@ eb_validate_vma(struct i915_execbuffer *eb,
return 0;
}
-static void
+static inline bool
+is_batch_buffer(struct i915_execbuffer *eb, unsigned int buffer_idx)
+{
+ return eb->args->flags & I915_EXEC_BATCH_FIRST ?
+ buffer_idx < eb->num_batches :
+ buffer_idx >= eb->args->buffer_count - eb->num_batches;
+}
+
+static int
eb_add_vma(struct i915_execbuffer *eb,
- unsigned int i, unsigned batch_idx,
+ unsigned int *current_batch,
+ unsigned int i,
struct i915_vma *vma)
{
+ struct drm_i915_private *i915 = eb->i915;
struct drm_i915_gem_exec_object2 *entry = &eb->exec[i];
struct eb_vma *ev = &eb->vma[i];
@@ -577,15 +595,43 @@ eb_add_vma(struct i915_execbuffer *eb,
* Note that actual hangs have only been observed on gen7, but for
* paranoia do it everywhere.
*/
- if (i == batch_idx) {
+ if (is_batch_buffer(eb, i)) {
if (entry->relocation_count &&
!(ev->flags & EXEC_OBJECT_PINNED))
ev->flags |= __EXEC_OBJECT_NEEDS_BIAS;
if (eb->reloc_cache.has_fence)
ev->flags |= EXEC_OBJECT_NEEDS_FENCE;
- eb->batch = ev;
+ eb->batches[*current_batch] = ev;
+
+ if (unlikely(ev->flags & EXEC_OBJECT_WRITE)) {
+ drm_dbg(&i915->drm,
+ "Attempting to use self-modifying batch buffer\n");
+ return -EINVAL;
+ }
+
+ if (range_overflows_t(u64,
+ eb->batch_start_offset,
+ eb->args->batch_len,
+ ev->vma->size)) {
+ drm_dbg(&i915->drm, "Attempting to use out-of-bounds batch\n");
+ return -EINVAL;
+ }
+
+ if (eb->args->batch_len == 0)
+ eb->batch_len[*current_batch] = ev->vma->size -
+ eb->batch_start_offset;
+ else
+ eb->batch_len[*current_batch] = eb->args->batch_len;
+ if (unlikely(eb->batch_len[*current_batch] == 0)) { /* impossible! */
+ drm_dbg(&i915->drm, "Invalid batch length\n");
+ return -EINVAL;
+ }
+
+ ++*current_batch;
}
+
+ return 0;
}
static inline int use_cpu_reloc(const struct reloc_cache *cache,
@@ -729,14 +775,6 @@ static int eb_reserve(struct i915_execbuffer *eb)
} while (1);
}
-static unsigned int eb_batch_index(const struct i915_execbuffer *eb)
-{
- if (eb->args->flags & I915_EXEC_BATCH_FIRST)
- return 0;
- else
- return eb->buffer_count - 1;
-}
-
static int eb_select_context(struct i915_execbuffer *eb)
{
struct i915_gem_context *ctx;
@@ -857,9 +895,7 @@ static struct i915_vma *eb_lookup_vma(struct i915_execbuffer *eb, u32 handle)
static int eb_lookup_vmas(struct i915_execbuffer *eb)
{
- struct drm_i915_private *i915 = eb->i915;
- unsigned int batch = eb_batch_index(eb);
- unsigned int i;
+ unsigned int i, current_batch = 0;
int err = 0;
INIT_LIST_HEAD(&eb->relocs);
@@ -879,7 +915,9 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
goto err;
}
- eb_add_vma(eb, i, batch, vma);
+ err = eb_add_vma(eb, ¤t_batch, i, vma);
+ if (err)
+ return err;
if (i915_gem_object_is_userptr(vma->obj)) {
err = i915_gem_object_userptr_submit_init(vma->obj);
@@ -902,26 +940,6 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
}
}
- if (unlikely(eb->batch->flags & EXEC_OBJECT_WRITE)) {
- drm_dbg(&i915->drm,
- "Attempting to use self-modifying batch buffer\n");
- return -EINVAL;
- }
-
- if (range_overflows_t(u64,
- eb->batch_start_offset, eb->batch_len,
- eb->batch->vma->size)) {
- drm_dbg(&i915->drm, "Attempting to use out-of-bounds batch\n");
- return -EINVAL;
- }
-
- if (eb->batch_len == 0)
- eb->batch_len = eb->batch->vma->size - eb->batch_start_offset;
- if (unlikely(eb->batch_len == 0)) { /* impossible! */
- drm_dbg(&i915->drm, "Invalid batch length\n");
- return -EINVAL;
- }
-
return 0;
err:
@@ -1654,8 +1672,7 @@ static int eb_reinit_userptr(struct i915_execbuffer *eb)
return 0;
}
-static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
- struct i915_request *rq)
+static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb)
{
bool have_copy = false;
struct eb_vma *ev;
@@ -1671,21 +1688,6 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
eb_release_vmas(eb, false);
i915_gem_ww_ctx_fini(&eb->ww);
- if (rq) {
- /* nonblocking is always false */
- if (i915_request_wait(rq, I915_WAIT_INTERRUPTIBLE,
- MAX_SCHEDULE_TIMEOUT) < 0) {
- i915_request_put(rq);
- rq = NULL;
-
- err = -EINTR;
- goto err_relock;
- }
-
- i915_request_put(rq);
- rq = NULL;
- }
-
/*
* We take 3 passes through the slowpatch.
*
@@ -1712,28 +1714,21 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
if (!err)
err = eb_reinit_userptr(eb);
-err_relock:
i915_gem_ww_ctx_init(&eb->ww, true);
if (err)
goto out;
/* reacquire the objects */
repeat_validate:
- rq = eb_pin_engine(eb, false);
- if (IS_ERR(rq)) {
- err = PTR_ERR(rq);
- rq = NULL;
+ err = eb_pin_engine(eb, false);
+ if (err)
goto err;
- }
-
- /* We didn't throttle, should be NULL */
- GEM_WARN_ON(rq);
err = eb_validate_vmas(eb);
if (err)
goto err;
- GEM_BUG_ON(!eb->batch);
+ GEM_BUG_ON(!eb->batches[0]);
list_for_each_entry(ev, &eb->relocs, reloc_link) {
if (!have_copy) {
@@ -1797,46 +1792,23 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
}
}
- if (rq)
- i915_request_put(rq);
-
return err;
}
static int eb_relocate_parse(struct i915_execbuffer *eb)
{
int err;
- struct i915_request *rq = NULL;
bool throttle = true;
retry:
- rq = eb_pin_engine(eb, throttle);
- if (IS_ERR(rq)) {
- err = PTR_ERR(rq);
- rq = NULL;
+ err = eb_pin_engine(eb, throttle);
+ if (err) {
if (err != -EDEADLK)
return err;
goto err;
}
- if (rq) {
- bool nonblock = eb->file->filp->f_flags & O_NONBLOCK;
-
- /* Need to drop all locks now for throttling, take slowpath */
- err = i915_request_wait(rq, I915_WAIT_INTERRUPTIBLE, 0);
- if (err == -ETIME) {
- if (nonblock) {
- err = -EWOULDBLOCK;
- i915_request_put(rq);
- goto err;
- }
- goto slow;
- }
- i915_request_put(rq);
- rq = NULL;
- }
-
/* only throttle once, even if we didn't need to throttle */
throttle = false;
@@ -1876,7 +1848,7 @@ static int eb_relocate_parse(struct i915_execbuffer *eb)
return err;
slow:
- err = eb_relocate_parse_slow(eb, rq);
+ err = eb_relocate_parse_slow(eb);
if (err)
/*
* If the user expects the execobject.offset and
@@ -1890,11 +1862,40 @@ static int eb_relocate_parse(struct i915_execbuffer *eb)
return err;
}
+/*
+ * Using two helper loops for the order of which requests / batches are created
+ * and added the to backend. Requests are created in order from the parent to
+ * the last child. Requests are added in the reverse order, from the last child
+ * to parent. This is done for locking reasons as the timeline lock is acquired
+ * during request creation and released when the request is added to the
+ * backend. To make lockdep happy (see intel_context_timeline_lock) this must be
+ * the ordering.
+ */
+#define for_each_batch_create_order(_eb, _i) \
+ for ((_i) = 0; (_i) < (_eb)->num_batches; ++(_i))
+#define for_each_batch_add_order(_eb, _i) \
+ BUILD_BUG_ON(!typecheck(int, _i)); \
+ for ((_i) = (_eb)->num_batches - 1; (_i) >= 0; --(_i))
+
+static struct i915_request *
+eb_find_first_request_added(struct i915_execbuffer *eb)
+{
+ int i;
+
+ for_each_batch_add_order(eb, i)
+ if (eb->requests[i])
+ return eb->requests[i];
+
+ GEM_BUG_ON("Request not found");
+
+ return NULL;
+}
+
static int eb_move_to_gpu(struct i915_execbuffer *eb)
{
const unsigned int count = eb->buffer_count;
unsigned int i = count;
- int err = 0;
+ int err = 0, j;
while (i--) {
struct eb_vma *ev = &eb->vma[i];
@@ -1907,11 +1908,17 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
if (flags & EXEC_OBJECT_CAPTURE) {
struct i915_capture_list *capture;
- capture = kmalloc(sizeof(*capture), GFP_KERNEL);
- if (capture) {
- capture->next = eb->request->capture_list;
- capture->vma = vma;
- eb->request->capture_list = capture;
+ for_each_batch_create_order(eb, j) {
+ if (!eb->requests[j])
+ break;
+
+ capture = kmalloc(sizeof(*capture), GFP_KERNEL);
+ if (capture) {
+ capture->next =
+ eb->requests[j]->capture_list;
+ capture->vma = vma;
+ eb->requests[j]->capture_list = capture;
+ }
}
}
@@ -1932,14 +1939,26 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
flags &= ~EXEC_OBJECT_ASYNC;
}
+ /* We only need to await on the first request */
if (err == 0 && !(flags & EXEC_OBJECT_ASYNC)) {
err = i915_request_await_object
- (eb->request, obj, flags & EXEC_OBJECT_WRITE);
+ (eb_find_first_request_added(eb), obj,
+ flags & EXEC_OBJECT_WRITE);
}
- if (err == 0)
- err = i915_vma_move_to_active(vma, eb->request,
- flags | __EXEC_OBJECT_NO_RESERVE);
+ for_each_batch_add_order(eb, j) {
+ if (err)
+ break;
+ if (!eb->requests[j])
+ continue;
+
+ err = _i915_vma_move_to_active(vma, eb->requests[j],
+ j ? NULL :
+ eb->composite_fence ?
+ eb->composite_fence :
+ &eb->requests[j]->fence,
+ flags | __EXEC_OBJECT_NO_RESERVE);
+ }
}
#ifdef CONFIG_MMU_NOTIFIER
@@ -1970,11 +1989,16 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
goto err_skip;
/* Unconditionally flush any chipset caches (for streaming writes). */
- intel_gt_chipset_flush(eb->engine->gt);
+ intel_gt_chipset_flush(eb->gt);
return 0;
err_skip:
- i915_request_set_error_once(eb->request, err);
+ for_each_batch_create_order(eb, j) {
+ if (!eb->requests[j])
+ break;
+
+ i915_request_set_error_once(eb->requests[j], err);
+ }
return err;
}
@@ -2069,14 +2093,17 @@ static int eb_parse(struct i915_execbuffer *eb)
int err;
if (!eb_use_cmdparser(eb)) {
- batch = eb_dispatch_secure(eb, eb->batch->vma);
+ batch = eb_dispatch_secure(eb, eb->batches[0]->vma);
if (IS_ERR(batch))
return PTR_ERR(batch);
goto secure_batch;
}
- len = eb->batch_len;
+ if (intel_context_is_parallel(eb->context))
+ return -EINVAL;
+
+ len = eb->batch_len[0];
if (!CMDPARSER_USES_GGTT(eb->i915)) {
/*
* ppGTT backed shadow buffers must be mapped RO, to prevent
@@ -2090,11 +2117,11 @@ static int eb_parse(struct i915_execbuffer *eb)
} else {
len += I915_CMD_PARSER_TRAMPOLINE_SIZE;
}
- if (unlikely(len < eb->batch_len)) /* last paranoid check of overflow */
+ if (unlikely(len < eb->batch_len[0])) /* last paranoid check of overflow */
return -EINVAL;
if (!pool) {
- pool = intel_gt_get_buffer_pool(eb->engine->gt, len,
+ pool = intel_gt_get_buffer_pool(eb->gt, len,
I915_MAP_WB);
if (IS_ERR(pool))
return PTR_ERR(pool);
@@ -2119,7 +2146,7 @@ static int eb_parse(struct i915_execbuffer *eb)
trampoline = shadow;
shadow = shadow_batch_pin(eb, pool->obj,
- &eb->engine->gt->ggtt->vm,
+ &eb->gt->ggtt->vm,
PIN_GLOBAL);
if (IS_ERR(shadow)) {
err = PTR_ERR(shadow);
@@ -2141,26 +2168,29 @@ static int eb_parse(struct i915_execbuffer *eb)
if (err)
goto err_trampoline;
- err = intel_engine_cmd_parser(eb->engine,
- eb->batch->vma,
+ err = intel_engine_cmd_parser(eb->context->engine,
+ eb->batches[0]->vma,
eb->batch_start_offset,
- eb->batch_len,
+ eb->batch_len[0],
shadow, trampoline);
if (err)
goto err_unpin_batch;
- eb->batch = &eb->vma[eb->buffer_count++];
- eb->batch->vma = i915_vma_get(shadow);
- eb->batch->flags = __EXEC_OBJECT_HAS_PIN;
+ eb->batches[0] = &eb->vma[eb->buffer_count++];
+ eb->batches[0]->vma = i915_vma_get(shadow);
+ eb->batches[0]->flags = __EXEC_OBJECT_HAS_PIN;
eb->trampoline = trampoline;
eb->batch_start_offset = 0;
secure_batch:
if (batch) {
- eb->batch = &eb->vma[eb->buffer_count++];
- eb->batch->flags = __EXEC_OBJECT_HAS_PIN;
- eb->batch->vma = i915_vma_get(batch);
+ if (intel_context_is_parallel(eb->context))
+ return -EINVAL;
+
+ eb->batches[0] = &eb->vma[eb->buffer_count++];
+ eb->batches[0]->flags = __EXEC_OBJECT_HAS_PIN;
+ eb->batches[0]->vma = i915_vma_get(batch);
}
return 0;
@@ -2176,19 +2206,18 @@ static int eb_parse(struct i915_execbuffer *eb)
return err;
}
-static int eb_submit(struct i915_execbuffer *eb, struct i915_vma *batch)
+static int eb_request_submit(struct i915_execbuffer *eb,
+ struct i915_request *rq,
+ struct i915_vma *batch,
+ u64 batch_len)
{
int err;
- if (intel_context_nopreempt(eb->context))
- __set_bit(I915_FENCE_FLAG_NOPREEMPT, &eb->request->fence.flags);
-
- err = eb_move_to_gpu(eb);
- if (err)
- return err;
+ if (intel_context_nopreempt(rq->context))
+ __set_bit(I915_FENCE_FLAG_NOPREEMPT, &rq->fence.flags);
if (eb->args->flags & I915_EXEC_GEN7_SOL_RESET) {
- err = i915_reset_gen7_sol_offsets(eb->request);
+ err = i915_reset_gen7_sol_offsets(rq);
if (err)
return err;
}
@@ -2199,26 +2228,26 @@ static int eb_submit(struct i915_execbuffer *eb, struct i915_vma *batch)
* allows us to determine if the batch is still waiting on the GPU
* or actually running by checking the breadcrumb.
*/
- if (eb->engine->emit_init_breadcrumb) {
- err = eb->engine->emit_init_breadcrumb(eb->request);
+ if (rq->context->engine->emit_init_breadcrumb) {
+ err = rq->context->engine->emit_init_breadcrumb(rq);
if (err)
return err;
}
- err = eb->engine->emit_bb_start(eb->request,
- batch->node.start +
- eb->batch_start_offset,
- eb->batch_len,
- eb->batch_flags);
+ err = rq->context->engine->emit_bb_start(rq,
+ batch->node.start +
+ eb->batch_start_offset,
+ batch_len,
+ eb->batch_flags);
if (err)
return err;
if (eb->trampoline) {
+ GEM_BUG_ON(intel_context_is_parallel(rq->context));
GEM_BUG_ON(eb->batch_start_offset);
- err = eb->engine->emit_bb_start(eb->request,
- eb->trampoline->node.start +
- eb->batch_len,
- 0, 0);
+ err = rq->context->engine->emit_bb_start(rq,
+ eb->trampoline->node.start +
+ batch_len, 0, 0);
if (err)
return err;
}
@@ -2226,6 +2255,27 @@ static int eb_submit(struct i915_execbuffer *eb, struct i915_vma *batch)
return 0;
}
+static int eb_submit(struct i915_execbuffer *eb)
+{
+ unsigned int i;
+ int err;
+
+ err = eb_move_to_gpu(eb);
+
+ for_each_batch_create_order(eb, i) {
+ if (!eb->requests[i])
+ break;
+
+ trace_i915_request_queue(eb->requests[i], eb->batch_flags);
+ if (!err)
+ err = eb_request_submit(eb, eb->requests[i],
+ eb->batches[i]->vma,
+ eb->batch_len[i]);
+ }
+
+ return err;
+}
+
static int num_vcs_engines(const struct drm_i915_private *i915)
{
return hweight_long(VDBOX_MASK(&i915->gt));
@@ -2291,26 +2341,11 @@ static struct i915_request *eb_throttle(struct i915_execbuffer *eb, struct intel
return i915_request_get(rq);
}
-static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb, bool throttle)
+static int eb_pin_timeline(struct i915_execbuffer *eb, struct intel_context *ce,
+ bool throttle)
{
- struct intel_context *ce = eb->context;
struct intel_timeline *tl;
struct i915_request *rq = NULL;
- int err;
-
- GEM_BUG_ON(eb->args->flags & __EXEC_ENGINE_PINNED);
-
- if (unlikely(intel_context_is_banned(ce)))
- return ERR_PTR(-EIO);
-
- /*
- * Pinning the contexts may generate requests in order to acquire
- * GGTT space, so do this first before we reserve a seqno for
- * ourselves.
- */
- err = intel_context_pin_ww(ce, &eb->ww);
- if (err)
- return ERR_PTR(err);
/*
* Take a local wakeref for preparing to dispatch the execbuf as
@@ -2321,33 +2356,108 @@ static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb, bool throt
* taken on the engine, and the parent device.
*/
tl = intel_context_timeline_lock(ce);
- if (IS_ERR(tl)) {
- intel_context_unpin(ce);
- return ERR_CAST(tl);
- }
+ if (IS_ERR(tl))
+ return PTR_ERR(tl);
intel_context_enter(ce);
if (throttle)
rq = eb_throttle(eb, ce);
intel_context_timeline_unlock(tl);
+ if (rq) {
+ bool nonblock = eb->file->filp->f_flags & O_NONBLOCK;
+ long timeout = nonblock ? 0 : MAX_SCHEDULE_TIMEOUT;
+
+ if (i915_request_wait(rq, I915_WAIT_INTERRUPTIBLE,
+ timeout) < 0) {
+ i915_request_put(rq);
+
+ tl = intel_context_timeline_lock(ce);
+ intel_context_exit(ce);
+ intel_context_timeline_unlock(tl);
+
+ if (nonblock)
+ return -EWOULDBLOCK;
+ else
+ return -EINTR;
+ }
+ i915_request_put(rq);
+ }
+
+ return 0;
+}
+
+static int eb_pin_engine(struct i915_execbuffer *eb, bool throttle)
+{
+ struct intel_context *ce = eb->context, *child;
+ int err;
+ int i = 0, j = 0;
+
+ GEM_BUG_ON(eb->args->flags & __EXEC_ENGINE_PINNED);
+
+ if (unlikely(intel_context_is_banned(ce)))
+ return -EIO;
+
+ /*
+ * Pinning the contexts may generate requests in order to acquire
+ * GGTT space, so do this first before we reserve a seqno for
+ * ourselves.
+ */
+ err = intel_context_pin_ww(ce, &eb->ww);
+ if (err)
+ return err;
+ for_each_child(ce, child) {
+ err = intel_context_pin_ww(child, &eb->ww);
+ GEM_BUG_ON(err); /* perma-pinned should incr a counter */
+ }
+
+ for_each_child(ce, child) {
+ err = eb_pin_timeline(eb, child, throttle);
+ if (err)
+ goto unwind;
+ ++i;
+ }
+ err = eb_pin_timeline(eb, ce, throttle);
+ if (err)
+ goto unwind;
+
eb->args->flags |= __EXEC_ENGINE_PINNED;
- return rq;
+ return 0;
+
+unwind:
+ for_each_child(ce, child) {
+ if (j++ < i) {
+ mutex_lock(&child->timeline->mutex);
+ intel_context_exit(child);
+ mutex_unlock(&child->timeline->mutex);
+ }
+ }
+ for_each_child(ce, child)
+ intel_context_unpin(child);
+ intel_context_unpin(ce);
+ return err;
}
static void eb_unpin_engine(struct i915_execbuffer *eb)
{
- struct intel_context *ce = eb->context;
- struct intel_timeline *tl = ce->timeline;
+ struct intel_context *ce = eb->context, *child;
if (!(eb->args->flags & __EXEC_ENGINE_PINNED))
return;
eb->args->flags &= ~__EXEC_ENGINE_PINNED;
- mutex_lock(&tl->mutex);
+ for_each_child(ce, child) {
+ mutex_lock(&child->timeline->mutex);
+ intel_context_exit(child);
+ mutex_unlock(&child->timeline->mutex);
+
+ intel_context_unpin(child);
+ }
+
+ mutex_lock(&ce->timeline->mutex);
intel_context_exit(ce);
- mutex_unlock(&tl->mutex);
+ mutex_unlock(&ce->timeline->mutex);
intel_context_unpin(ce);
}
@@ -2398,7 +2508,7 @@ eb_select_legacy_ring(struct i915_execbuffer *eb)
static int
eb_select_engine(struct i915_execbuffer *eb)
{
- struct intel_context *ce;
+ struct intel_context *ce, *child;
unsigned int idx;
int err;
@@ -2411,6 +2521,20 @@ eb_select_engine(struct i915_execbuffer *eb)
if (IS_ERR(ce))
return PTR_ERR(ce);
+ if (intel_context_is_parallel(ce)) {
+ if (eb->buffer_count < ce->parallel.number_children + 1) {
+ intel_context_put(ce);
+ return -EINVAL;
+ }
+ if (eb->batch_start_offset || eb->args->batch_len) {
+ intel_context_put(ce);
+ return -EINVAL;
+ }
+ }
+ eb->num_batches = ce->parallel.number_children + 1;
+
+ for_each_child(ce, child)
+ intel_context_get(child);
intel_gt_pm_get(ce->engine->gt);
if (!test_bit(CONTEXT_ALLOC_BIT, &ce->flags)) {
@@ -2418,6 +2542,13 @@ eb_select_engine(struct i915_execbuffer *eb)
if (err)
goto err;
}
+ for_each_child(ce, child) {
+ if (!test_bit(CONTEXT_ALLOC_BIT, &child->flags)) {
+ err = intel_context_alloc_state(child);
+ if (err)
+ goto err;
+ }
+ }
/*
* ABI: Before userspace accesses the GPU (e.g. execbuffer), report
@@ -2428,7 +2559,7 @@ eb_select_engine(struct i915_execbuffer *eb)
goto err;
eb->context = ce;
- eb->engine = ce->engine;
+ eb->gt = ce->engine->gt;
/*
* Make sure engine pool stays alive even if we call intel_context_put
@@ -2439,6 +2570,8 @@ eb_select_engine(struct i915_execbuffer *eb)
err:
intel_gt_pm_put(ce->engine->gt);
+ for_each_child(ce, child)
+ intel_context_put(child);
intel_context_put(ce);
return err;
}
@@ -2446,7 +2579,11 @@ eb_select_engine(struct i915_execbuffer *eb)
static void
eb_put_engine(struct i915_execbuffer *eb)
{
- intel_gt_pm_put(eb->engine->gt);
+ struct intel_context *child;
+
+ intel_gt_pm_put(eb->gt);
+ for_each_child(eb->context, child)
+ intel_context_put(child);
intel_context_put(eb->context);
}
@@ -2669,7 +2806,8 @@ static void put_fence_array(struct eb_fence *fences, int num_fences)
}
static int
-await_fence_array(struct i915_execbuffer *eb)
+await_fence_array(struct i915_execbuffer *eb,
+ struct i915_request *rq)
{
unsigned int n;
int err;
@@ -2683,8 +2821,7 @@ await_fence_array(struct i915_execbuffer *eb)
if (!eb->fences[n].dma_fence)
continue;
- err = i915_request_await_dma_fence(eb->request,
- eb->fences[n].dma_fence);
+ err = i915_request_await_dma_fence(rq, eb->fences[n].dma_fence);
if (err < 0)
return err;
}
@@ -2692,9 +2829,9 @@ await_fence_array(struct i915_execbuffer *eb)
return 0;
}
-static void signal_fence_array(const struct i915_execbuffer *eb)
+static void signal_fence_array(const struct i915_execbuffer *eb,
+ struct dma_fence * const fence)
{
- struct dma_fence * const fence = &eb->request->fence;
unsigned int n;
for (n = 0; n < eb->num_fences; n++) {
@@ -2742,9 +2879,9 @@ static void retire_requests(struct intel_timeline *tl, struct i915_request *end)
break;
}
-static int eb_request_add(struct i915_execbuffer *eb, int err)
+static int eb_request_add(struct i915_execbuffer *eb, struct i915_request *rq,
+ int err, bool last_parallel)
{
- struct i915_request *rq = eb->request;
struct intel_timeline * const tl = i915_request_timeline(rq);
struct i915_sched_attr attr = {};
struct i915_request *prev;
@@ -2766,6 +2903,17 @@ static int eb_request_add(struct i915_execbuffer *eb, int err)
err = -ENOENT; /* override any transient errors */
}
+ if (intel_context_is_parallel(eb->context)) {
+ if (err) {
+ __i915_request_skip(rq);
+ set_bit(I915_FENCE_FLAG_SKIP_PARALLEL,
+ &rq->fence.flags);
+ }
+ if (last_parallel)
+ set_bit(I915_FENCE_FLAG_SUBMIT_PARALLEL,
+ &rq->fence.flags);
+ }
+
__i915_request_queue(rq, &attr);
/* Try to clean up the client's timeline after submitting the request */
@@ -2777,6 +2925,25 @@ static int eb_request_add(struct i915_execbuffer *eb, int err)
return err;
}
+static int eb_requests_add(struct i915_execbuffer *eb, int err)
+{
+ int i;
+
+ /*
+ * We iterate in reverse order of creation to release timeline mutexes in
+ * same order.
+ */
+ for_each_batch_add_order(eb, i) {
+ struct i915_request *rq = eb->requests[i];
+
+ if (!rq)
+ continue;
+ err |= eb_request_add(eb, rq, err, i == 0);
+ }
+
+ return err;
+}
+
static const i915_user_extension_fn execbuf_extensions[] = {
[DRM_I915_GEM_EXECBUFFER_EXT_TIMELINE_FENCES] = parse_timeline_fences,
};
@@ -2803,6 +2970,182 @@ parse_execbuf2_extensions(struct drm_i915_gem_execbuffer2 *args,
eb);
}
+static void eb_requests_get(struct i915_execbuffer *eb)
+{
+ unsigned int i;
+
+ for_each_batch_create_order(eb, i) {
+ if (!eb->requests[i])
+ break;
+
+ i915_request_get(eb->requests[i]);
+ }
+}
+
+static void eb_requests_put(struct i915_execbuffer *eb)
+{
+ unsigned int i;
+
+ for_each_batch_create_order(eb, i) {
+ if (!eb->requests[i])
+ break;
+
+ i915_request_put(eb->requests[i]);
+ }
+}
+
+static struct sync_file *
+eb_composite_fence_create(struct i915_execbuffer *eb, int out_fence_fd)
+{
+ struct sync_file *out_fence = NULL;
+ struct dma_fence_array *fence_array;
+ struct dma_fence **fences;
+ unsigned int i;
+
+ GEM_BUG_ON(!intel_context_is_parent(eb->context));
+
+ fences = kmalloc_array(eb->num_batches, sizeof(*fences), GFP_KERNEL);
+ if (!fences)
+ return ERR_PTR(-ENOMEM);
+
+ for_each_batch_create_order(eb, i)
+ fences[i] = &eb->requests[i]->fence;
+
+ fence_array = dma_fence_array_create(eb->num_batches,
+ fences,
+ eb->context->parallel.fence_context,
+ eb->context->parallel.seqno,
+ false);
+ if (!fence_array) {
+ kfree(fences);
+ return ERR_PTR(-ENOMEM);
+ }
+
+ /* Move ownership to the dma_fence_array created above */
+ for_each_batch_create_order(eb, i)
+ dma_fence_get(fences[i]);
+
+ if (out_fence_fd != -1) {
+ out_fence = sync_file_create(&fence_array->base);
+ /* sync_file now owns fence_arry, drop creation ref */
+ dma_fence_put(&fence_array->base);
+ if (!out_fence)
+ return ERR_PTR(-ENOMEM);
+ }
+
+ eb->composite_fence = &fence_array->base;
+
+ return out_fence;
+}
+
+static struct sync_file *
+eb_fences_add(struct i915_execbuffer *eb, struct i915_request *rq,
+ struct dma_fence *in_fence, int out_fence_fd)
+{
+ struct sync_file *out_fence = NULL;
+ int err;
+
+ if (unlikely(eb->gem_context->syncobj)) {
+ struct dma_fence *fence;
+
+ fence = drm_syncobj_fence_get(eb->gem_context->syncobj);
+ err = i915_request_await_dma_fence(rq, fence);
+ dma_fence_put(fence);
+ if (err)
+ return ERR_PTR(err);
+ }
+
+ if (in_fence) {
+ if (eb->args->flags & I915_EXEC_FENCE_SUBMIT)
+ err = i915_request_await_execution(rq, in_fence);
+ else
+ err = i915_request_await_dma_fence(rq, in_fence);
+ if (err < 0)
+ return ERR_PTR(err);
+ }
+
+ if (eb->fences) {
+ err = await_fence_array(eb, rq);
+ if (err)
+ return ERR_PTR(err);
+ }
+
+ if (intel_context_is_parallel(eb->context)) {
+ out_fence = eb_composite_fence_create(eb, out_fence_fd);
+ if (IS_ERR(out_fence))
+ return ERR_PTR(-ENOMEM);
+ } else if (out_fence_fd != -1) {
+ out_fence = sync_file_create(&rq->fence);
+ if (!out_fence)
+ return ERR_PTR(-ENOMEM);
+ }
+
+ return out_fence;
+}
+
+static struct intel_context *
+eb_find_context(struct i915_execbuffer *eb, unsigned int context_number)
+{
+ struct intel_context *child;
+
+ if (likely(context_number == 0))
+ return eb->context;
+
+ for_each_child(eb->context, child)
+ if (!--context_number)
+ return child;
+
+ GEM_BUG_ON("Context not found");
+
+ return NULL;
+}
+
+static struct sync_file *
+eb_requests_create(struct i915_execbuffer *eb, struct dma_fence *in_fence,
+ int out_fence_fd)
+{
+ struct sync_file *out_fence = NULL;
+ unsigned int i;
+
+ for_each_batch_create_order(eb, i) {
+ /* Allocate a request for this batch buffer nice and early. */
+ eb->requests[i] = i915_request_create(eb_find_context(eb, i));
+ if (IS_ERR(eb->requests[i])) {
+ out_fence = ERR_PTR(PTR_ERR(eb->requests[i]));
+ eb->requests[i] = NULL;
+ return out_fence;
+ }
+
+ /*
+ * Only the first request added (committed to backend) has to
+ * take the in fences into account as all subsequent requests
+ * will have fences inserted inbetween them.
+ */
+ if (i + 1 == eb->num_batches) {
+ out_fence = eb_fences_add(eb, eb->requests[i],
+ in_fence, out_fence_fd);
+ if (IS_ERR(out_fence))
+ return out_fence;
+ }
+
+ /*
+ * Whilst this request exists, batch_obj will be on the
+ * active_list, and so will hold the active reference. Only when
+ * this request is retired will the batch_obj be moved onto
+ * the inactive_list and lose its active reference. Hence we do
+ * not need to explicitly hold another reference here.
+ */
+ eb->requests[i]->batch = eb->batches[i]->vma;
+ if (eb->batch_pool) {
+ GEM_BUG_ON(intel_context_is_parallel(eb->context));
+ intel_gt_buffer_pool_mark_active(eb->batch_pool,
+ eb->requests[i]);
+ }
+ }
+
+ return out_fence;
+}
+
static int
i915_gem_do_execbuffer(struct drm_device *dev,
struct drm_file *file,
@@ -2813,7 +3156,6 @@ i915_gem_do_execbuffer(struct drm_device *dev,
struct i915_execbuffer eb;
struct dma_fence *in_fence = NULL;
struct sync_file *out_fence = NULL;
- struct i915_vma *batch;
int out_fence_fd = -1;
int err;
@@ -2837,12 +3179,15 @@ i915_gem_do_execbuffer(struct drm_device *dev,
eb.buffer_count = args->buffer_count;
eb.batch_start_offset = args->batch_start_offset;
- eb.batch_len = args->batch_len;
eb.trampoline = NULL;
eb.fences = NULL;
eb.num_fences = 0;
+ memset(eb.requests, 0, sizeof(struct i915_request *) *
+ ARRAY_SIZE(eb.requests));
+ eb.composite_fence = NULL;
+
eb.batch_flags = 0;
if (args->flags & I915_EXEC_SECURE) {
if (GRAPHICS_VER(i915) >= 11)
@@ -2926,70 +3271,25 @@ i915_gem_do_execbuffer(struct drm_device *dev,
ww_acquire_done(&eb.ww.ctx);
- batch = eb.batch->vma;
-
- /* Allocate a request for this batch buffer nice and early. */
- eb.request = i915_request_create(eb.context);
- if (IS_ERR(eb.request)) {
- err = PTR_ERR(eb.request);
- goto err_vma;
- }
-
- if (unlikely(eb.gem_context->syncobj)) {
- struct dma_fence *fence;
-
- fence = drm_syncobj_fence_get(eb.gem_context->syncobj);
- err = i915_request_await_dma_fence(eb.request, fence);
- dma_fence_put(fence);
- if (err)
- goto err_ext;
- }
-
- if (in_fence) {
- if (args->flags & I915_EXEC_FENCE_SUBMIT)
- err = i915_request_await_execution(eb.request,
- in_fence);
- else
- err = i915_request_await_dma_fence(eb.request,
- in_fence);
- if (err < 0)
- goto err_request;
- }
-
- if (eb.fences) {
- err = await_fence_array(&eb);
- if (err)
+ out_fence = eb_requests_create(&eb, in_fence, out_fence_fd);
+ if (IS_ERR(out_fence)) {
+ err = PTR_ERR(out_fence);
+ if (eb.requests[0])
goto err_request;
+ else
+ goto err_vma;
}
- if (out_fence_fd != -1) {
- out_fence = sync_file_create(&eb.request->fence);
- if (!out_fence) {
- err = -ENOMEM;
- goto err_request;
- }
- }
-
- /*
- * Whilst this request exists, batch_obj will be on the
- * active_list, and so will hold the active reference. Only when this
- * request is retired will the the batch_obj be moved onto the
- * inactive_list and lose its active reference. Hence we do not need
- * to explicitly hold another reference here.
- */
- eb.request->batch = batch;
- if (eb.batch_pool)
- intel_gt_buffer_pool_mark_active(eb.batch_pool, eb.request);
-
- trace_i915_request_queue(eb.request, eb.batch_flags);
- err = eb_submit(&eb, batch);
+ err = eb_submit(&eb);
err_request:
- i915_request_get(eb.request);
- err = eb_request_add(&eb, err);
+ eb_requests_get(&eb);
+ err = eb_requests_add(&eb, err);
if (eb.fences)
- signal_fence_array(&eb);
+ signal_fence_array(&eb, eb.composite_fence ?
+ eb.composite_fence :
+ &eb.requests[0]->fence);
if (out_fence) {
if (err == 0) {
@@ -3004,10 +3304,15 @@ i915_gem_do_execbuffer(struct drm_device *dev,
if (unlikely(eb.gem_context->syncobj)) {
drm_syncobj_replace_fence(eb.gem_context->syncobj,
- &eb.request->fence);
+ eb.composite_fence ?
+ eb.composite_fence :
+ &eb.requests[0]->fence);
}
- i915_request_put(eb.request);
+ if (!out_fence && eb.composite_fence)
+ dma_fence_put(eb.composite_fence);
+
+ eb_requests_put(&eb);
err_vma:
eb_release_vmas(&eb, true);
diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
index edf12caaade3..246c37d72cd7 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.h
+++ b/drivers/gpu/drm/i915/gt/intel_context.h
@@ -241,7 +241,13 @@ intel_context_timeline_lock(struct intel_context *ce)
struct intel_timeline *tl = ce->timeline;
int err;
- err = mutex_lock_interruptible(&tl->mutex);
+ if (intel_context_is_parent(ce))
+ err = mutex_lock_interruptible_nested(&tl->mutex, 0);
+ else if (intel_context_is_child(ce))
+ err = mutex_lock_interruptible_nested(&tl->mutex,
+ ce->parallel.child_index + 1);
+ else
+ err = mutex_lock_interruptible(&tl->mutex);
if (err)
return ERR_PTR(err);
diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
index 95a5b94b4ece..9e0177dc5484 100644
--- a/drivers/gpu/drm/i915/gt/intel_context_types.h
+++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
@@ -248,6 +248,16 @@ struct intel_context {
* context
*/
struct i915_request *last_rq;
+ /**
+ * @fence_context: fence context composite fence when doing
+ * parallel submission
+ */
+ u64 fence_context;
+ /**
+ * @seqno: seqno for composite fence when doing parallel
+ * submission
+ */
+ u32 seqno;
/** @number_children: number of children if parent */
u8 number_children;
/** @child_index: index into child_list if child */
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 50f0f4eba03b..361fab2cae99 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -3093,6 +3093,8 @@ guc_create_parallel(struct intel_engine_cs **engines,
}
}
+ parent->parallel.fence_context = dma_fence_context_alloc(1);
+
parent->engine->emit_bb_start =
emit_bb_start_parent_no_preempt_mid_batch;
parent->engine->emit_fini_breadcrumb =
diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
index 8950785e55d6..24db8459376b 100644
--- a/drivers/gpu/drm/i915/i915_request.h
+++ b/drivers/gpu/drm/i915/i915_request.h
@@ -147,6 +147,15 @@ enum {
* tail.
*/
I915_FENCE_FLAG_SUBMIT_PARALLEL,
+
+ /*
+ * I915_FENCE_FLAG_SKIP_PARALLEL - request with a context in a
+ * parent-child relationship (parallel submission, multi-lrc) that
+ * hit an error while generating requests in the execbuf IOCTL.
+ * Indicates this request should be skipped as another request in
+ * submission / relationship encoutered an error.
+ */
+ I915_FENCE_FLAG_SKIP_PARALLEL,
};
/**
diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c
index 4b7fc4647e46..90546fa58fc1 100644
--- a/drivers/gpu/drm/i915/i915_vma.c
+++ b/drivers/gpu/drm/i915/i915_vma.c
@@ -1234,9 +1234,10 @@ int __i915_vma_move_to_active(struct i915_vma *vma, struct i915_request *rq)
return i915_active_add_request(&vma->active, rq);
}
-int i915_vma_move_to_active(struct i915_vma *vma,
- struct i915_request *rq,
- unsigned int flags)
+int _i915_vma_move_to_active(struct i915_vma *vma,
+ struct i915_request *rq,
+ struct dma_fence *fence,
+ unsigned int flags)
{
struct drm_i915_gem_object *obj = vma->obj;
int err;
@@ -1257,9 +1258,11 @@ int i915_vma_move_to_active(struct i915_vma *vma,
intel_frontbuffer_put(front);
}
- dma_resv_add_excl_fence(vma->resv, &rq->fence);
- obj->write_domain = I915_GEM_DOMAIN_RENDER;
- obj->read_domains = 0;
+ if (fence) {
+ dma_resv_add_excl_fence(vma->resv, fence);
+ obj->write_domain = I915_GEM_DOMAIN_RENDER;
+ obj->read_domains = 0;
+ }
} else {
if (!(flags & __EXEC_OBJECT_NO_RESERVE)) {
err = dma_resv_reserve_shared(vma->resv, 1);
@@ -1267,8 +1270,10 @@ int i915_vma_move_to_active(struct i915_vma *vma,
return err;
}
- dma_resv_add_shared_fence(vma->resv, &rq->fence);
- obj->write_domain = 0;
+ if (fence) {
+ dma_resv_add_shared_fence(vma->resv, fence);
+ obj->write_domain = 0;
+ }
}
if (flags & EXEC_OBJECT_NEEDS_FENCE && vma->fence)
diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vma.h
index ed69f66c7ab0..648dbe744c96 100644
--- a/drivers/gpu/drm/i915/i915_vma.h
+++ b/drivers/gpu/drm/i915/i915_vma.h
@@ -57,9 +57,16 @@ static inline bool i915_vma_is_active(const struct i915_vma *vma)
int __must_check __i915_vma_move_to_active(struct i915_vma *vma,
struct i915_request *rq);
-int __must_check i915_vma_move_to_active(struct i915_vma *vma,
- struct i915_request *rq,
- unsigned int flags);
+int __must_check _i915_vma_move_to_active(struct i915_vma *vma,
+ struct i915_request *rq,
+ struct dma_fence *fence,
+ unsigned int flags);
+static inline int __must_check
+i915_vma_move_to_active(struct i915_vma *vma, struct i915_request *rq,
+ unsigned int flags)
+{
+ return _i915_vma_move_to_active(vma, rq, &rq->fence, flags);
+}
#define __i915_vma_flags(v) ((unsigned long *)&(v)->flags.counter)
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [Intel-gfx] [PATCH 20/25] drm/i915: Multi-BB execbuf
2021-10-14 17:20 ` Matthew Brost
@ 2021-10-14 18:27 ` John Harrison
-1 siblings, 0 replies; 69+ messages in thread
From: John Harrison @ 2021-10-14 18:27 UTC (permalink / raw)
To: Matthew Brost, intel-gfx, dri-devel
On 10/14/2021 10:20, Matthew Brost wrote:
> Allow multiple batch buffers to be submitted in a single execbuf IOCTL
> after a context has been configured with the 'set_parallel' extension.
> The number batches is implicit based on the contexts configuration.
>
> This is implemented with a series of loops. First a loop is used to find
> all the batches, a loop to pin all the HW contexts, a loop to create all
> the requests, a loop to submit (emit BB start, etc...) all the requests,
> a loop to tie the requests to the VMAs they touch, and finally a loop to
> commit the requests to the backend.
>
> A composite fence is also created for the generated requests to return
> to the user and to stick in dma resv slots.
>
> No behavior from the existing IOCTL should be changed aside from when
> throttling because the ring for a context is full. In this situation,
> i915 will now wait while holding the object locks. This change was done
> because the code is much simpler to wait while holding the locks and we
> believe there isn't a huge benefit of dropping these locks. If this
> proves false we can restructure the code to drop the locks during the
> wait.
>
> IGT: https://patchwork.freedesktop.org/patch/447008/?series=93071&rev=1
> media UMD: https://github.com/intel/media-driver/pull/1252
>
> v2:
> (Matthew Brost)
> - Return proper error value if i915_request_create fails
> v3:
> (John Harrison)
> - Add comment explaining create / add order loops + locking
> - Update commit message explaining different in IOCTL behavior
> - Line wrap some comments
> - eb_add_request returns void
> - Return -EINVAL rather triggering BUG_ON if cmd parser used
> (Checkpatch)
> - Check eb->batch_len[*current_batch]
> v4:
> (CI)
> - Set batch len if passed if via execbuf args
> - Call __i915_request_skip after __i915_request_commit
> (Kernel test robot)
> - Initialize rq to NULL in eb_pin_timeline
> v5:
> (John Harrison)
> - Fix typo in comments near bb order loops
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 783 ++++++++++++------
> drivers/gpu/drm/i915/gt/intel_context.h | 8 +-
> drivers/gpu/drm/i915/gt/intel_context_types.h | 10 +
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +
> drivers/gpu/drm/i915/i915_request.h | 9 +
> drivers/gpu/drm/i915/i915_vma.c | 21 +-
> drivers/gpu/drm/i915/i915_vma.h | 13 +-
> 7 files changed, 595 insertions(+), 251 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
> index c75afc8784e3..fc30856e81fa 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
> @@ -246,17 +246,25 @@ struct i915_execbuffer {
> struct drm_i915_gem_exec_object2 *exec; /** ioctl execobj[] */
> struct eb_vma *vma;
>
> - struct intel_engine_cs *engine; /** engine to queue the request to */
> + struct intel_gt *gt; /* gt for the execbuf */
> struct intel_context *context; /* logical state for the request */
> struct i915_gem_context *gem_context; /** caller's context */
>
> - struct i915_request *request; /** our request to build */
> - struct eb_vma *batch; /** identity of the batch obj/vma */
> + /** our requests to build */
> + struct i915_request *requests[MAX_ENGINE_INSTANCE + 1];
> + /** identity of the batch obj/vma */
> + struct eb_vma *batches[MAX_ENGINE_INSTANCE + 1];
> struct i915_vma *trampoline; /** trampoline used for chaining */
>
> + /** used for excl fence in dma_resv objects when > 1 BB submitted */
> + struct dma_fence *composite_fence;
> +
> /** actual size of execobj[] as we may extend it for the cmdparser */
> unsigned int buffer_count;
>
> + /* number of batches in execbuf IOCTL */
> + unsigned int num_batches;
> +
> /** list of vma not yet bound during reservation phase */
> struct list_head unbound;
>
> @@ -283,7 +291,8 @@ struct i915_execbuffer {
>
> u64 invalid_flags; /** Set of execobj.flags that are invalid */
>
> - u64 batch_len; /** Length of batch within object */
> + /** Length of batch within object */
> + u64 batch_len[MAX_ENGINE_INSTANCE + 1];
> u32 batch_start_offset; /** Location within object of batch */
> u32 batch_flags; /** Flags composed for emit_bb_start() */
> struct intel_gt_buffer_pool_node *batch_pool; /** pool node for batch buffer */
> @@ -301,14 +310,13 @@ struct i915_execbuffer {
> };
>
> static int eb_parse(struct i915_execbuffer *eb);
> -static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb,
> - bool throttle);
> +static int eb_pin_engine(struct i915_execbuffer *eb, bool throttle);
> static void eb_unpin_engine(struct i915_execbuffer *eb);
>
> static inline bool eb_use_cmdparser(const struct i915_execbuffer *eb)
> {
> - return intel_engine_requires_cmd_parser(eb->engine) ||
> - (intel_engine_using_cmd_parser(eb->engine) &&
> + return intel_engine_requires_cmd_parser(eb->context->engine) ||
> + (intel_engine_using_cmd_parser(eb->context->engine) &&
> eb->args->batch_len);
> }
>
> @@ -546,11 +554,21 @@ eb_validate_vma(struct i915_execbuffer *eb,
> return 0;
> }
>
> -static void
> +static inline bool
> +is_batch_buffer(struct i915_execbuffer *eb, unsigned int buffer_idx)
> +{
> + return eb->args->flags & I915_EXEC_BATCH_FIRST ?
> + buffer_idx < eb->num_batches :
> + buffer_idx >= eb->args->buffer_count - eb->num_batches;
> +}
> +
> +static int
> eb_add_vma(struct i915_execbuffer *eb,
> - unsigned int i, unsigned batch_idx,
> + unsigned int *current_batch,
> + unsigned int i,
> struct i915_vma *vma)
> {
> + struct drm_i915_private *i915 = eb->i915;
> struct drm_i915_gem_exec_object2 *entry = &eb->exec[i];
> struct eb_vma *ev = &eb->vma[i];
>
> @@ -577,15 +595,43 @@ eb_add_vma(struct i915_execbuffer *eb,
> * Note that actual hangs have only been observed on gen7, but for
> * paranoia do it everywhere.
> */
> - if (i == batch_idx) {
> + if (is_batch_buffer(eb, i)) {
> if (entry->relocation_count &&
> !(ev->flags & EXEC_OBJECT_PINNED))
> ev->flags |= __EXEC_OBJECT_NEEDS_BIAS;
> if (eb->reloc_cache.has_fence)
> ev->flags |= EXEC_OBJECT_NEEDS_FENCE;
>
> - eb->batch = ev;
> + eb->batches[*current_batch] = ev;
> +
> + if (unlikely(ev->flags & EXEC_OBJECT_WRITE)) {
> + drm_dbg(&i915->drm,
> + "Attempting to use self-modifying batch buffer\n");
> + return -EINVAL;
> + }
> +
> + if (range_overflows_t(u64,
> + eb->batch_start_offset,
> + eb->args->batch_len,
> + ev->vma->size)) {
> + drm_dbg(&i915->drm, "Attempting to use out-of-bounds batch\n");
> + return -EINVAL;
> + }
> +
> + if (eb->args->batch_len == 0)
> + eb->batch_len[*current_batch] = ev->vma->size -
> + eb->batch_start_offset;
> + else
> + eb->batch_len[*current_batch] = eb->args->batch_len;
> + if (unlikely(eb->batch_len[*current_batch] == 0)) { /* impossible! */
> + drm_dbg(&i915->drm, "Invalid batch length\n");
> + return -EINVAL;
> + }
> +
> + ++*current_batch;
> }
> +
> + return 0;
> }
>
> static inline int use_cpu_reloc(const struct reloc_cache *cache,
> @@ -729,14 +775,6 @@ static int eb_reserve(struct i915_execbuffer *eb)
> } while (1);
> }
>
> -static unsigned int eb_batch_index(const struct i915_execbuffer *eb)
> -{
> - if (eb->args->flags & I915_EXEC_BATCH_FIRST)
> - return 0;
> - else
> - return eb->buffer_count - 1;
> -}
> -
> static int eb_select_context(struct i915_execbuffer *eb)
> {
> struct i915_gem_context *ctx;
> @@ -857,9 +895,7 @@ static struct i915_vma *eb_lookup_vma(struct i915_execbuffer *eb, u32 handle)
>
> static int eb_lookup_vmas(struct i915_execbuffer *eb)
> {
> - struct drm_i915_private *i915 = eb->i915;
> - unsigned int batch = eb_batch_index(eb);
> - unsigned int i;
> + unsigned int i, current_batch = 0;
> int err = 0;
>
> INIT_LIST_HEAD(&eb->relocs);
> @@ -879,7 +915,9 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
> goto err;
> }
>
> - eb_add_vma(eb, i, batch, vma);
> + err = eb_add_vma(eb, ¤t_batch, i, vma);
> + if (err)
> + return err;
>
> if (i915_gem_object_is_userptr(vma->obj)) {
> err = i915_gem_object_userptr_submit_init(vma->obj);
> @@ -902,26 +940,6 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
> }
> }
>
> - if (unlikely(eb->batch->flags & EXEC_OBJECT_WRITE)) {
> - drm_dbg(&i915->drm,
> - "Attempting to use self-modifying batch buffer\n");
> - return -EINVAL;
> - }
> -
> - if (range_overflows_t(u64,
> - eb->batch_start_offset, eb->batch_len,
> - eb->batch->vma->size)) {
> - drm_dbg(&i915->drm, "Attempting to use out-of-bounds batch\n");
> - return -EINVAL;
> - }
> -
> - if (eb->batch_len == 0)
> - eb->batch_len = eb->batch->vma->size - eb->batch_start_offset;
> - if (unlikely(eb->batch_len == 0)) { /* impossible! */
> - drm_dbg(&i915->drm, "Invalid batch length\n");
> - return -EINVAL;
> - }
> -
> return 0;
>
> err:
> @@ -1654,8 +1672,7 @@ static int eb_reinit_userptr(struct i915_execbuffer *eb)
> return 0;
> }
>
> -static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
> - struct i915_request *rq)
> +static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb)
> {
> bool have_copy = false;
> struct eb_vma *ev;
> @@ -1671,21 +1688,6 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
> eb_release_vmas(eb, false);
> i915_gem_ww_ctx_fini(&eb->ww);
>
> - if (rq) {
> - /* nonblocking is always false */
> - if (i915_request_wait(rq, I915_WAIT_INTERRUPTIBLE,
> - MAX_SCHEDULE_TIMEOUT) < 0) {
> - i915_request_put(rq);
> - rq = NULL;
> -
> - err = -EINTR;
> - goto err_relock;
> - }
> -
> - i915_request_put(rq);
> - rq = NULL;
> - }
> -
> /*
> * We take 3 passes through the slowpatch.
> *
> @@ -1712,28 +1714,21 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
> if (!err)
> err = eb_reinit_userptr(eb);
>
> -err_relock:
> i915_gem_ww_ctx_init(&eb->ww, true);
> if (err)
> goto out;
>
> /* reacquire the objects */
> repeat_validate:
> - rq = eb_pin_engine(eb, false);
> - if (IS_ERR(rq)) {
> - err = PTR_ERR(rq);
> - rq = NULL;
> + err = eb_pin_engine(eb, false);
> + if (err)
> goto err;
> - }
> -
> - /* We didn't throttle, should be NULL */
> - GEM_WARN_ON(rq);
>
> err = eb_validate_vmas(eb);
> if (err)
> goto err;
>
> - GEM_BUG_ON(!eb->batch);
> + GEM_BUG_ON(!eb->batches[0]);
>
> list_for_each_entry(ev, &eb->relocs, reloc_link) {
> if (!have_copy) {
> @@ -1797,46 +1792,23 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
> }
> }
>
> - if (rq)
> - i915_request_put(rq);
> -
> return err;
> }
>
> static int eb_relocate_parse(struct i915_execbuffer *eb)
> {
> int err;
> - struct i915_request *rq = NULL;
> bool throttle = true;
>
> retry:
> - rq = eb_pin_engine(eb, throttle);
> - if (IS_ERR(rq)) {
> - err = PTR_ERR(rq);
> - rq = NULL;
> + err = eb_pin_engine(eb, throttle);
> + if (err) {
> if (err != -EDEADLK)
> return err;
>
> goto err;
> }
>
> - if (rq) {
> - bool nonblock = eb->file->filp->f_flags & O_NONBLOCK;
> -
> - /* Need to drop all locks now for throttling, take slowpath */
> - err = i915_request_wait(rq, I915_WAIT_INTERRUPTIBLE, 0);
> - if (err == -ETIME) {
> - if (nonblock) {
> - err = -EWOULDBLOCK;
> - i915_request_put(rq);
> - goto err;
> - }
> - goto slow;
> - }
> - i915_request_put(rq);
> - rq = NULL;
> - }
> -
> /* only throttle once, even if we didn't need to throttle */
> throttle = false;
>
> @@ -1876,7 +1848,7 @@ static int eb_relocate_parse(struct i915_execbuffer *eb)
> return err;
>
> slow:
> - err = eb_relocate_parse_slow(eb, rq);
> + err = eb_relocate_parse_slow(eb);
> if (err)
> /*
> * If the user expects the execobject.offset and
> @@ -1890,11 +1862,40 @@ static int eb_relocate_parse(struct i915_execbuffer *eb)
> return err;
> }
>
> +/*
> + * Using two helper loops for the order of which requests / batches are created
> + * and added the to backend. Requests are created in order from the parent to
> + * the last child. Requests are added in the reverse order, from the last child
> + * to parent. This is done for locking reasons as the timeline lock is acquired
> + * during request creation and released when the request is added to the
> + * backend. To make lockdep happy (see intel_context_timeline_lock) this must be
> + * the ordering.
> + */
> +#define for_each_batch_create_order(_eb, _i) \
> + for ((_i) = 0; (_i) < (_eb)->num_batches; ++(_i))
> +#define for_each_batch_add_order(_eb, _i) \
> + BUILD_BUG_ON(!typecheck(int, _i)); \
> + for ((_i) = (_eb)->num_batches - 1; (_i) >= 0; --(_i))
> +
> +static struct i915_request *
> +eb_find_first_request_added(struct i915_execbuffer *eb)
> +{
> + int i;
> +
> + for_each_batch_add_order(eb, i)
> + if (eb->requests[i])
> + return eb->requests[i];
> +
> + GEM_BUG_ON("Request not found");
> +
> + return NULL;
> +}
> +
> static int eb_move_to_gpu(struct i915_execbuffer *eb)
> {
> const unsigned int count = eb->buffer_count;
> unsigned int i = count;
> - int err = 0;
> + int err = 0, j;
>
> while (i--) {
> struct eb_vma *ev = &eb->vma[i];
> @@ -1907,11 +1908,17 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
> if (flags & EXEC_OBJECT_CAPTURE) {
> struct i915_capture_list *capture;
>
> - capture = kmalloc(sizeof(*capture), GFP_KERNEL);
> - if (capture) {
> - capture->next = eb->request->capture_list;
> - capture->vma = vma;
> - eb->request->capture_list = capture;
> + for_each_batch_create_order(eb, j) {
> + if (!eb->requests[j])
> + break;
> +
> + capture = kmalloc(sizeof(*capture), GFP_KERNEL);
> + if (capture) {
> + capture->next =
> + eb->requests[j]->capture_list;
> + capture->vma = vma;
> + eb->requests[j]->capture_list = capture;
> + }
> }
> }
>
> @@ -1932,14 +1939,26 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
> flags &= ~EXEC_OBJECT_ASYNC;
> }
>
> + /* We only need to await on the first request */
> if (err == 0 && !(flags & EXEC_OBJECT_ASYNC)) {
> err = i915_request_await_object
> - (eb->request, obj, flags & EXEC_OBJECT_WRITE);
> + (eb_find_first_request_added(eb), obj,
> + flags & EXEC_OBJECT_WRITE);
> }
>
> - if (err == 0)
> - err = i915_vma_move_to_active(vma, eb->request,
> - flags | __EXEC_OBJECT_NO_RESERVE);
> + for_each_batch_add_order(eb, j) {
> + if (err)
> + break;
> + if (!eb->requests[j])
> + continue;
> +
> + err = _i915_vma_move_to_active(vma, eb->requests[j],
> + j ? NULL :
> + eb->composite_fence ?
> + eb->composite_fence :
> + &eb->requests[j]->fence,
> + flags | __EXEC_OBJECT_NO_RESERVE);
> + }
> }
>
> #ifdef CONFIG_MMU_NOTIFIER
> @@ -1970,11 +1989,16 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
> goto err_skip;
>
> /* Unconditionally flush any chipset caches (for streaming writes). */
> - intel_gt_chipset_flush(eb->engine->gt);
> + intel_gt_chipset_flush(eb->gt);
> return 0;
>
> err_skip:
> - i915_request_set_error_once(eb->request, err);
> + for_each_batch_create_order(eb, j) {
> + if (!eb->requests[j])
> + break;
> +
> + i915_request_set_error_once(eb->requests[j], err);
> + }
> return err;
> }
>
> @@ -2069,14 +2093,17 @@ static int eb_parse(struct i915_execbuffer *eb)
> int err;
>
> if (!eb_use_cmdparser(eb)) {
> - batch = eb_dispatch_secure(eb, eb->batch->vma);
> + batch = eb_dispatch_secure(eb, eb->batches[0]->vma);
> if (IS_ERR(batch))
> return PTR_ERR(batch);
>
> goto secure_batch;
> }
>
> - len = eb->batch_len;
> + if (intel_context_is_parallel(eb->context))
> + return -EINVAL;
> +
> + len = eb->batch_len[0];
> if (!CMDPARSER_USES_GGTT(eb->i915)) {
> /*
> * ppGTT backed shadow buffers must be mapped RO, to prevent
> @@ -2090,11 +2117,11 @@ static int eb_parse(struct i915_execbuffer *eb)
> } else {
> len += I915_CMD_PARSER_TRAMPOLINE_SIZE;
> }
> - if (unlikely(len < eb->batch_len)) /* last paranoid check of overflow */
> + if (unlikely(len < eb->batch_len[0])) /* last paranoid check of overflow */
> return -EINVAL;
>
> if (!pool) {
> - pool = intel_gt_get_buffer_pool(eb->engine->gt, len,
> + pool = intel_gt_get_buffer_pool(eb->gt, len,
> I915_MAP_WB);
> if (IS_ERR(pool))
> return PTR_ERR(pool);
> @@ -2119,7 +2146,7 @@ static int eb_parse(struct i915_execbuffer *eb)
> trampoline = shadow;
>
> shadow = shadow_batch_pin(eb, pool->obj,
> - &eb->engine->gt->ggtt->vm,
> + &eb->gt->ggtt->vm,
> PIN_GLOBAL);
> if (IS_ERR(shadow)) {
> err = PTR_ERR(shadow);
> @@ -2141,26 +2168,29 @@ static int eb_parse(struct i915_execbuffer *eb)
> if (err)
> goto err_trampoline;
>
> - err = intel_engine_cmd_parser(eb->engine,
> - eb->batch->vma,
> + err = intel_engine_cmd_parser(eb->context->engine,
> + eb->batches[0]->vma,
> eb->batch_start_offset,
> - eb->batch_len,
> + eb->batch_len[0],
> shadow, trampoline);
> if (err)
> goto err_unpin_batch;
>
> - eb->batch = &eb->vma[eb->buffer_count++];
> - eb->batch->vma = i915_vma_get(shadow);
> - eb->batch->flags = __EXEC_OBJECT_HAS_PIN;
> + eb->batches[0] = &eb->vma[eb->buffer_count++];
> + eb->batches[0]->vma = i915_vma_get(shadow);
> + eb->batches[0]->flags = __EXEC_OBJECT_HAS_PIN;
>
> eb->trampoline = trampoline;
> eb->batch_start_offset = 0;
>
> secure_batch:
> if (batch) {
> - eb->batch = &eb->vma[eb->buffer_count++];
> - eb->batch->flags = __EXEC_OBJECT_HAS_PIN;
> - eb->batch->vma = i915_vma_get(batch);
> + if (intel_context_is_parallel(eb->context))
> + return -EINVAL;
> +
> + eb->batches[0] = &eb->vma[eb->buffer_count++];
> + eb->batches[0]->flags = __EXEC_OBJECT_HAS_PIN;
> + eb->batches[0]->vma = i915_vma_get(batch);
> }
> return 0;
>
> @@ -2176,19 +2206,18 @@ static int eb_parse(struct i915_execbuffer *eb)
> return err;
> }
>
> -static int eb_submit(struct i915_execbuffer *eb, struct i915_vma *batch)
> +static int eb_request_submit(struct i915_execbuffer *eb,
> + struct i915_request *rq,
> + struct i915_vma *batch,
> + u64 batch_len)
> {
> int err;
>
> - if (intel_context_nopreempt(eb->context))
> - __set_bit(I915_FENCE_FLAG_NOPREEMPT, &eb->request->fence.flags);
> -
> - err = eb_move_to_gpu(eb);
> - if (err)
> - return err;
> + if (intel_context_nopreempt(rq->context))
> + __set_bit(I915_FENCE_FLAG_NOPREEMPT, &rq->fence.flags);
>
> if (eb->args->flags & I915_EXEC_GEN7_SOL_RESET) {
> - err = i915_reset_gen7_sol_offsets(eb->request);
> + err = i915_reset_gen7_sol_offsets(rq);
> if (err)
> return err;
> }
> @@ -2199,26 +2228,26 @@ static int eb_submit(struct i915_execbuffer *eb, struct i915_vma *batch)
> * allows us to determine if the batch is still waiting on the GPU
> * or actually running by checking the breadcrumb.
> */
> - if (eb->engine->emit_init_breadcrumb) {
> - err = eb->engine->emit_init_breadcrumb(eb->request);
> + if (rq->context->engine->emit_init_breadcrumb) {
> + err = rq->context->engine->emit_init_breadcrumb(rq);
> if (err)
> return err;
> }
>
> - err = eb->engine->emit_bb_start(eb->request,
> - batch->node.start +
> - eb->batch_start_offset,
> - eb->batch_len,
> - eb->batch_flags);
> + err = rq->context->engine->emit_bb_start(rq,
> + batch->node.start +
> + eb->batch_start_offset,
> + batch_len,
> + eb->batch_flags);
> if (err)
> return err;
>
> if (eb->trampoline) {
> + GEM_BUG_ON(intel_context_is_parallel(rq->context));
> GEM_BUG_ON(eb->batch_start_offset);
> - err = eb->engine->emit_bb_start(eb->request,
> - eb->trampoline->node.start +
> - eb->batch_len,
> - 0, 0);
> + err = rq->context->engine->emit_bb_start(rq,
> + eb->trampoline->node.start +
> + batch_len, 0, 0);
> if (err)
> return err;
> }
> @@ -2226,6 +2255,27 @@ static int eb_submit(struct i915_execbuffer *eb, struct i915_vma *batch)
> return 0;
> }
>
> +static int eb_submit(struct i915_execbuffer *eb)
> +{
> + unsigned int i;
> + int err;
> +
> + err = eb_move_to_gpu(eb);
> +
> + for_each_batch_create_order(eb, i) {
> + if (!eb->requests[i])
> + break;
> +
> + trace_i915_request_queue(eb->requests[i], eb->batch_flags);
> + if (!err)
> + err = eb_request_submit(eb, eb->requests[i],
> + eb->batches[i]->vma,
> + eb->batch_len[i]);
> + }
> +
> + return err;
> +}
> +
> static int num_vcs_engines(const struct drm_i915_private *i915)
> {
> return hweight_long(VDBOX_MASK(&i915->gt));
> @@ -2291,26 +2341,11 @@ static struct i915_request *eb_throttle(struct i915_execbuffer *eb, struct intel
> return i915_request_get(rq);
> }
>
> -static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb, bool throttle)
> +static int eb_pin_timeline(struct i915_execbuffer *eb, struct intel_context *ce,
> + bool throttle)
> {
> - struct intel_context *ce = eb->context;
> struct intel_timeline *tl;
> struct i915_request *rq = NULL;
> - int err;
> -
> - GEM_BUG_ON(eb->args->flags & __EXEC_ENGINE_PINNED);
> -
> - if (unlikely(intel_context_is_banned(ce)))
> - return ERR_PTR(-EIO);
> -
> - /*
> - * Pinning the contexts may generate requests in order to acquire
> - * GGTT space, so do this first before we reserve a seqno for
> - * ourselves.
> - */
> - err = intel_context_pin_ww(ce, &eb->ww);
> - if (err)
> - return ERR_PTR(err);
>
> /*
> * Take a local wakeref for preparing to dispatch the execbuf as
> @@ -2321,33 +2356,108 @@ static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb, bool throt
> * taken on the engine, and the parent device.
> */
> tl = intel_context_timeline_lock(ce);
> - if (IS_ERR(tl)) {
> - intel_context_unpin(ce);
> - return ERR_CAST(tl);
> - }
> + if (IS_ERR(tl))
> + return PTR_ERR(tl);
>
> intel_context_enter(ce);
> if (throttle)
> rq = eb_throttle(eb, ce);
> intel_context_timeline_unlock(tl);
>
> + if (rq) {
> + bool nonblock = eb->file->filp->f_flags & O_NONBLOCK;
> + long timeout = nonblock ? 0 : MAX_SCHEDULE_TIMEOUT;
> +
> + if (i915_request_wait(rq, I915_WAIT_INTERRUPTIBLE,
> + timeout) < 0) {
> + i915_request_put(rq);
> +
> + tl = intel_context_timeline_lock(ce);
> + intel_context_exit(ce);
> + intel_context_timeline_unlock(tl);
> +
> + if (nonblock)
> + return -EWOULDBLOCK;
> + else
> + return -EINTR;
> + }
> + i915_request_put(rq);
> + }
> +
> + return 0;
> +}
> +
> +static int eb_pin_engine(struct i915_execbuffer *eb, bool throttle)
> +{
> + struct intel_context *ce = eb->context, *child;
> + int err;
> + int i = 0, j = 0;
> +
> + GEM_BUG_ON(eb->args->flags & __EXEC_ENGINE_PINNED);
> +
> + if (unlikely(intel_context_is_banned(ce)))
> + return -EIO;
> +
> + /*
> + * Pinning the contexts may generate requests in order to acquire
> + * GGTT space, so do this first before we reserve a seqno for
> + * ourselves.
> + */
> + err = intel_context_pin_ww(ce, &eb->ww);
> + if (err)
> + return err;
> + for_each_child(ce, child) {
> + err = intel_context_pin_ww(child, &eb->ww);
> + GEM_BUG_ON(err); /* perma-pinned should incr a counter */
> + }
> +
> + for_each_child(ce, child) {
> + err = eb_pin_timeline(eb, child, throttle);
> + if (err)
> + goto unwind;
> + ++i;
> + }
> + err = eb_pin_timeline(eb, ce, throttle);
> + if (err)
> + goto unwind;
> +
> eb->args->flags |= __EXEC_ENGINE_PINNED;
> - return rq;
> + return 0;
> +
> +unwind:
> + for_each_child(ce, child) {
> + if (j++ < i) {
> + mutex_lock(&child->timeline->mutex);
> + intel_context_exit(child);
> + mutex_unlock(&child->timeline->mutex);
> + }
> + }
> + for_each_child(ce, child)
> + intel_context_unpin(child);
> + intel_context_unpin(ce);
> + return err;
> }
>
> static void eb_unpin_engine(struct i915_execbuffer *eb)
> {
> - struct intel_context *ce = eb->context;
> - struct intel_timeline *tl = ce->timeline;
> + struct intel_context *ce = eb->context, *child;
>
> if (!(eb->args->flags & __EXEC_ENGINE_PINNED))
> return;
>
> eb->args->flags &= ~__EXEC_ENGINE_PINNED;
>
> - mutex_lock(&tl->mutex);
> + for_each_child(ce, child) {
> + mutex_lock(&child->timeline->mutex);
> + intel_context_exit(child);
> + mutex_unlock(&child->timeline->mutex);
> +
> + intel_context_unpin(child);
> + }
> +
> + mutex_lock(&ce->timeline->mutex);
> intel_context_exit(ce);
> - mutex_unlock(&tl->mutex);
> + mutex_unlock(&ce->timeline->mutex);
>
> intel_context_unpin(ce);
> }
> @@ -2398,7 +2508,7 @@ eb_select_legacy_ring(struct i915_execbuffer *eb)
> static int
> eb_select_engine(struct i915_execbuffer *eb)
> {
> - struct intel_context *ce;
> + struct intel_context *ce, *child;
> unsigned int idx;
> int err;
>
> @@ -2411,6 +2521,20 @@ eb_select_engine(struct i915_execbuffer *eb)
> if (IS_ERR(ce))
> return PTR_ERR(ce);
>
> + if (intel_context_is_parallel(ce)) {
> + if (eb->buffer_count < ce->parallel.number_children + 1) {
> + intel_context_put(ce);
> + return -EINVAL;
> + }
> + if (eb->batch_start_offset || eb->args->batch_len) {
> + intel_context_put(ce);
> + return -EINVAL;
> + }
> + }
> + eb->num_batches = ce->parallel.number_children + 1;
> +
> + for_each_child(ce, child)
> + intel_context_get(child);
> intel_gt_pm_get(ce->engine->gt);
>
> if (!test_bit(CONTEXT_ALLOC_BIT, &ce->flags)) {
> @@ -2418,6 +2542,13 @@ eb_select_engine(struct i915_execbuffer *eb)
> if (err)
> goto err;
> }
> + for_each_child(ce, child) {
> + if (!test_bit(CONTEXT_ALLOC_BIT, &child->flags)) {
> + err = intel_context_alloc_state(child);
> + if (err)
> + goto err;
> + }
> + }
>
> /*
> * ABI: Before userspace accesses the GPU (e.g. execbuffer), report
> @@ -2428,7 +2559,7 @@ eb_select_engine(struct i915_execbuffer *eb)
> goto err;
>
> eb->context = ce;
> - eb->engine = ce->engine;
> + eb->gt = ce->engine->gt;
>
> /*
> * Make sure engine pool stays alive even if we call intel_context_put
> @@ -2439,6 +2570,8 @@ eb_select_engine(struct i915_execbuffer *eb)
>
> err:
> intel_gt_pm_put(ce->engine->gt);
> + for_each_child(ce, child)
> + intel_context_put(child);
> intel_context_put(ce);
> return err;
> }
> @@ -2446,7 +2579,11 @@ eb_select_engine(struct i915_execbuffer *eb)
> static void
> eb_put_engine(struct i915_execbuffer *eb)
> {
> - intel_gt_pm_put(eb->engine->gt);
> + struct intel_context *child;
> +
> + intel_gt_pm_put(eb->gt);
> + for_each_child(eb->context, child)
> + intel_context_put(child);
> intel_context_put(eb->context);
> }
>
> @@ -2669,7 +2806,8 @@ static void put_fence_array(struct eb_fence *fences, int num_fences)
> }
>
> static int
> -await_fence_array(struct i915_execbuffer *eb)
> +await_fence_array(struct i915_execbuffer *eb,
> + struct i915_request *rq)
> {
> unsigned int n;
> int err;
> @@ -2683,8 +2821,7 @@ await_fence_array(struct i915_execbuffer *eb)
> if (!eb->fences[n].dma_fence)
> continue;
>
> - err = i915_request_await_dma_fence(eb->request,
> - eb->fences[n].dma_fence);
> + err = i915_request_await_dma_fence(rq, eb->fences[n].dma_fence);
> if (err < 0)
> return err;
> }
> @@ -2692,9 +2829,9 @@ await_fence_array(struct i915_execbuffer *eb)
> return 0;
> }
>
> -static void signal_fence_array(const struct i915_execbuffer *eb)
> +static void signal_fence_array(const struct i915_execbuffer *eb,
> + struct dma_fence * const fence)
> {
> - struct dma_fence * const fence = &eb->request->fence;
> unsigned int n;
>
> for (n = 0; n < eb->num_fences; n++) {
> @@ -2742,9 +2879,9 @@ static void retire_requests(struct intel_timeline *tl, struct i915_request *end)
> break;
> }
>
> -static int eb_request_add(struct i915_execbuffer *eb, int err)
> +static int eb_request_add(struct i915_execbuffer *eb, struct i915_request *rq,
> + int err, bool last_parallel)
> {
> - struct i915_request *rq = eb->request;
> struct intel_timeline * const tl = i915_request_timeline(rq);
> struct i915_sched_attr attr = {};
> struct i915_request *prev;
> @@ -2766,6 +2903,17 @@ static int eb_request_add(struct i915_execbuffer *eb, int err)
> err = -ENOENT; /* override any transient errors */
> }
>
> + if (intel_context_is_parallel(eb->context)) {
> + if (err) {
> + __i915_request_skip(rq);
> + set_bit(I915_FENCE_FLAG_SKIP_PARALLEL,
> + &rq->fence.flags);
> + }
> + if (last_parallel)
> + set_bit(I915_FENCE_FLAG_SUBMIT_PARALLEL,
> + &rq->fence.flags);
> + }
> +
> __i915_request_queue(rq, &attr);
>
> /* Try to clean up the client's timeline after submitting the request */
> @@ -2777,6 +2925,25 @@ static int eb_request_add(struct i915_execbuffer *eb, int err)
> return err;
> }
>
> +static int eb_requests_add(struct i915_execbuffer *eb, int err)
> +{
> + int i;
> +
> + /*
> + * We iterate in reverse order of creation to release timeline mutexes in
> + * same order.
> + */
> + for_each_batch_add_order(eb, i) {
> + struct i915_request *rq = eb->requests[i];
> +
> + if (!rq)
> + continue;
> + err |= eb_request_add(eb, rq, err, i == 0);
> + }
> +
> + return err;
> +}
> +
> static const i915_user_extension_fn execbuf_extensions[] = {
> [DRM_I915_GEM_EXECBUFFER_EXT_TIMELINE_FENCES] = parse_timeline_fences,
> };
> @@ -2803,6 +2970,182 @@ parse_execbuf2_extensions(struct drm_i915_gem_execbuffer2 *args,
> eb);
> }
>
> +static void eb_requests_get(struct i915_execbuffer *eb)
> +{
> + unsigned int i;
> +
> + for_each_batch_create_order(eb, i) {
> + if (!eb->requests[i])
> + break;
> +
> + i915_request_get(eb->requests[i]);
> + }
> +}
> +
> +static void eb_requests_put(struct i915_execbuffer *eb)
> +{
> + unsigned int i;
> +
> + for_each_batch_create_order(eb, i) {
> + if (!eb->requests[i])
> + break;
> +
> + i915_request_put(eb->requests[i]);
> + }
> +}
> +
> +static struct sync_file *
> +eb_composite_fence_create(struct i915_execbuffer *eb, int out_fence_fd)
> +{
> + struct sync_file *out_fence = NULL;
> + struct dma_fence_array *fence_array;
> + struct dma_fence **fences;
> + unsigned int i;
> +
> + GEM_BUG_ON(!intel_context_is_parent(eb->context));
> +
> + fences = kmalloc_array(eb->num_batches, sizeof(*fences), GFP_KERNEL);
> + if (!fences)
> + return ERR_PTR(-ENOMEM);
> +
> + for_each_batch_create_order(eb, i)
> + fences[i] = &eb->requests[i]->fence;
> +
> + fence_array = dma_fence_array_create(eb->num_batches,
> + fences,
> + eb->context->parallel.fence_context,
> + eb->context->parallel.seqno,
> + false);
> + if (!fence_array) {
> + kfree(fences);
> + return ERR_PTR(-ENOMEM);
> + }
> +
> + /* Move ownership to the dma_fence_array created above */
> + for_each_batch_create_order(eb, i)
> + dma_fence_get(fences[i]);
> +
> + if (out_fence_fd != -1) {
> + out_fence = sync_file_create(&fence_array->base);
> + /* sync_file now owns fence_arry, drop creation ref */
> + dma_fence_put(&fence_array->base);
> + if (!out_fence)
> + return ERR_PTR(-ENOMEM);
> + }
> +
> + eb->composite_fence = &fence_array->base;
> +
> + return out_fence;
> +}
> +
> +static struct sync_file *
> +eb_fences_add(struct i915_execbuffer *eb, struct i915_request *rq,
> + struct dma_fence *in_fence, int out_fence_fd)
> +{
> + struct sync_file *out_fence = NULL;
> + int err;
> +
> + if (unlikely(eb->gem_context->syncobj)) {
> + struct dma_fence *fence;
> +
> + fence = drm_syncobj_fence_get(eb->gem_context->syncobj);
> + err = i915_request_await_dma_fence(rq, fence);
> + dma_fence_put(fence);
> + if (err)
> + return ERR_PTR(err);
> + }
> +
> + if (in_fence) {
> + if (eb->args->flags & I915_EXEC_FENCE_SUBMIT)
> + err = i915_request_await_execution(rq, in_fence);
> + else
> + err = i915_request_await_dma_fence(rq, in_fence);
> + if (err < 0)
> + return ERR_PTR(err);
> + }
> +
> + if (eb->fences) {
> + err = await_fence_array(eb, rq);
> + if (err)
> + return ERR_PTR(err);
> + }
> +
> + if (intel_context_is_parallel(eb->context)) {
> + out_fence = eb_composite_fence_create(eb, out_fence_fd);
> + if (IS_ERR(out_fence))
> + return ERR_PTR(-ENOMEM);
> + } else if (out_fence_fd != -1) {
> + out_fence = sync_file_create(&rq->fence);
> + if (!out_fence)
> + return ERR_PTR(-ENOMEM);
> + }
> +
> + return out_fence;
> +}
> +
> +static struct intel_context *
> +eb_find_context(struct i915_execbuffer *eb, unsigned int context_number)
> +{
> + struct intel_context *child;
> +
> + if (likely(context_number == 0))
> + return eb->context;
> +
> + for_each_child(eb->context, child)
> + if (!--context_number)
> + return child;
> +
> + GEM_BUG_ON("Context not found");
> +
> + return NULL;
> +}
> +
> +static struct sync_file *
> +eb_requests_create(struct i915_execbuffer *eb, struct dma_fence *in_fence,
> + int out_fence_fd)
> +{
> + struct sync_file *out_fence = NULL;
> + unsigned int i;
> +
> + for_each_batch_create_order(eb, i) {
> + /* Allocate a request for this batch buffer nice and early. */
> + eb->requests[i] = i915_request_create(eb_find_context(eb, i));
> + if (IS_ERR(eb->requests[i])) {
> + out_fence = ERR_PTR(PTR_ERR(eb->requests[i]));
> + eb->requests[i] = NULL;
> + return out_fence;
> + }
> +
> + /*
> + * Only the first request added (committed to backend) has to
> + * take the in fences into account as all subsequent requests
> + * will have fences inserted inbetween them.
> + */
> + if (i + 1 == eb->num_batches) {
> + out_fence = eb_fences_add(eb, eb->requests[i],
> + in_fence, out_fence_fd);
> + if (IS_ERR(out_fence))
> + return out_fence;
> + }
> +
> + /*
> + * Whilst this request exists, batch_obj will be on the
> + * active_list, and so will hold the active reference. Only when
> + * this request is retired will the batch_obj be moved onto
> + * the inactive_list and lose its active reference. Hence we do
> + * not need to explicitly hold another reference here.
> + */
> + eb->requests[i]->batch = eb->batches[i]->vma;
> + if (eb->batch_pool) {
> + GEM_BUG_ON(intel_context_is_parallel(eb->context));
> + intel_gt_buffer_pool_mark_active(eb->batch_pool,
> + eb->requests[i]);
> + }
> + }
> +
> + return out_fence;
> +}
> +
> static int
> i915_gem_do_execbuffer(struct drm_device *dev,
> struct drm_file *file,
> @@ -2813,7 +3156,6 @@ i915_gem_do_execbuffer(struct drm_device *dev,
> struct i915_execbuffer eb;
> struct dma_fence *in_fence = NULL;
> struct sync_file *out_fence = NULL;
> - struct i915_vma *batch;
> int out_fence_fd = -1;
> int err;
>
> @@ -2837,12 +3179,15 @@ i915_gem_do_execbuffer(struct drm_device *dev,
>
> eb.buffer_count = args->buffer_count;
> eb.batch_start_offset = args->batch_start_offset;
> - eb.batch_len = args->batch_len;
> eb.trampoline = NULL;
>
> eb.fences = NULL;
> eb.num_fences = 0;
>
> + memset(eb.requests, 0, sizeof(struct i915_request *) *
> + ARRAY_SIZE(eb.requests));
> + eb.composite_fence = NULL;
> +
> eb.batch_flags = 0;
> if (args->flags & I915_EXEC_SECURE) {
> if (GRAPHICS_VER(i915) >= 11)
> @@ -2926,70 +3271,25 @@ i915_gem_do_execbuffer(struct drm_device *dev,
>
> ww_acquire_done(&eb.ww.ctx);
>
> - batch = eb.batch->vma;
> -
> - /* Allocate a request for this batch buffer nice and early. */
> - eb.request = i915_request_create(eb.context);
> - if (IS_ERR(eb.request)) {
> - err = PTR_ERR(eb.request);
> - goto err_vma;
> - }
> -
> - if (unlikely(eb.gem_context->syncobj)) {
> - struct dma_fence *fence;
> -
> - fence = drm_syncobj_fence_get(eb.gem_context->syncobj);
> - err = i915_request_await_dma_fence(eb.request, fence);
> - dma_fence_put(fence);
> - if (err)
> - goto err_ext;
> - }
> -
> - if (in_fence) {
> - if (args->flags & I915_EXEC_FENCE_SUBMIT)
> - err = i915_request_await_execution(eb.request,
> - in_fence);
> - else
> - err = i915_request_await_dma_fence(eb.request,
> - in_fence);
> - if (err < 0)
> - goto err_request;
> - }
> -
> - if (eb.fences) {
> - err = await_fence_array(&eb);
> - if (err)
> + out_fence = eb_requests_create(&eb, in_fence, out_fence_fd);
> + if (IS_ERR(out_fence)) {
> + err = PTR_ERR(out_fence);
> + if (eb.requests[0])
> goto err_request;
> + else
> + goto err_vma;
> }
>
> - if (out_fence_fd != -1) {
> - out_fence = sync_file_create(&eb.request->fence);
> - if (!out_fence) {
> - err = -ENOMEM;
> - goto err_request;
> - }
> - }
> -
> - /*
> - * Whilst this request exists, batch_obj will be on the
> - * active_list, and so will hold the active reference. Only when this
> - * request is retired will the the batch_obj be moved onto the
> - * inactive_list and lose its active reference. Hence we do not need
> - * to explicitly hold another reference here.
> - */
> - eb.request->batch = batch;
> - if (eb.batch_pool)
> - intel_gt_buffer_pool_mark_active(eb.batch_pool, eb.request);
> -
> - trace_i915_request_queue(eb.request, eb.batch_flags);
> - err = eb_submit(&eb, batch);
> + err = eb_submit(&eb);
>
> err_request:
> - i915_request_get(eb.request);
> - err = eb_request_add(&eb, err);
> + eb_requests_get(&eb);
> + err = eb_requests_add(&eb, err);
>
> if (eb.fences)
> - signal_fence_array(&eb);
> + signal_fence_array(&eb, eb.composite_fence ?
> + eb.composite_fence :
> + &eb.requests[0]->fence);
>
> if (out_fence) {
> if (err == 0) {
> @@ -3004,10 +3304,15 @@ i915_gem_do_execbuffer(struct drm_device *dev,
>
> if (unlikely(eb.gem_context->syncobj)) {
> drm_syncobj_replace_fence(eb.gem_context->syncobj,
> - &eb.request->fence);
> + eb.composite_fence ?
> + eb.composite_fence :
> + &eb.requests[0]->fence);
> }
>
> - i915_request_put(eb.request);
> + if (!out_fence && eb.composite_fence)
> + dma_fence_put(eb.composite_fence);
> +
> + eb_requests_put(&eb);
>
> err_vma:
> eb_release_vmas(&eb, true);
> diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
> index edf12caaade3..246c37d72cd7 100644
> --- a/drivers/gpu/drm/i915/gt/intel_context.h
> +++ b/drivers/gpu/drm/i915/gt/intel_context.h
> @@ -241,7 +241,13 @@ intel_context_timeline_lock(struct intel_context *ce)
> struct intel_timeline *tl = ce->timeline;
> int err;
>
> - err = mutex_lock_interruptible(&tl->mutex);
> + if (intel_context_is_parent(ce))
> + err = mutex_lock_interruptible_nested(&tl->mutex, 0);
> + else if (intel_context_is_child(ce))
> + err = mutex_lock_interruptible_nested(&tl->mutex,
> + ce->parallel.child_index + 1);
> + else
> + err = mutex_lock_interruptible(&tl->mutex);
> if (err)
> return ERR_PTR(err);
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
> index 95a5b94b4ece..9e0177dc5484 100644
> --- a/drivers/gpu/drm/i915/gt/intel_context_types.h
> +++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
> @@ -248,6 +248,16 @@ struct intel_context {
> * context
> */
> struct i915_request *last_rq;
> + /**
> + * @fence_context: fence context composite fence when doing
> + * parallel submission
> + */
> + u64 fence_context;
> + /**
> + * @seqno: seqno for composite fence when doing parallel
> + * submission
> + */
> + u32 seqno;
> /** @number_children: number of children if parent */
> u8 number_children;
> /** @child_index: index into child_list if child */
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 50f0f4eba03b..361fab2cae99 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -3093,6 +3093,8 @@ guc_create_parallel(struct intel_engine_cs **engines,
> }
> }
>
> + parent->parallel.fence_context = dma_fence_context_alloc(1);
> +
> parent->engine->emit_bb_start =
> emit_bb_start_parent_no_preempt_mid_batch;
> parent->engine->emit_fini_breadcrumb =
> diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
> index 8950785e55d6..24db8459376b 100644
> --- a/drivers/gpu/drm/i915/i915_request.h
> +++ b/drivers/gpu/drm/i915/i915_request.h
> @@ -147,6 +147,15 @@ enum {
> * tail.
> */
> I915_FENCE_FLAG_SUBMIT_PARALLEL,
> +
> + /*
> + * I915_FENCE_FLAG_SKIP_PARALLEL - request with a context in a
> + * parent-child relationship (parallel submission, multi-lrc) that
> + * hit an error while generating requests in the execbuf IOCTL.
> + * Indicates this request should be skipped as another request in
> + * submission / relationship encoutered an error.
> + */
> + I915_FENCE_FLAG_SKIP_PARALLEL,
> };
>
> /**
> diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c
> index 4b7fc4647e46..90546fa58fc1 100644
> --- a/drivers/gpu/drm/i915/i915_vma.c
> +++ b/drivers/gpu/drm/i915/i915_vma.c
> @@ -1234,9 +1234,10 @@ int __i915_vma_move_to_active(struct i915_vma *vma, struct i915_request *rq)
> return i915_active_add_request(&vma->active, rq);
> }
>
> -int i915_vma_move_to_active(struct i915_vma *vma,
> - struct i915_request *rq,
> - unsigned int flags)
> +int _i915_vma_move_to_active(struct i915_vma *vma,
> + struct i915_request *rq,
> + struct dma_fence *fence,
> + unsigned int flags)
> {
> struct drm_i915_gem_object *obj = vma->obj;
> int err;
> @@ -1257,9 +1258,11 @@ int i915_vma_move_to_active(struct i915_vma *vma,
> intel_frontbuffer_put(front);
> }
>
> - dma_resv_add_excl_fence(vma->resv, &rq->fence);
> - obj->write_domain = I915_GEM_DOMAIN_RENDER;
> - obj->read_domains = 0;
> + if (fence) {
> + dma_resv_add_excl_fence(vma->resv, fence);
> + obj->write_domain = I915_GEM_DOMAIN_RENDER;
> + obj->read_domains = 0;
> + }
> } else {
> if (!(flags & __EXEC_OBJECT_NO_RESERVE)) {
> err = dma_resv_reserve_shared(vma->resv, 1);
> @@ -1267,8 +1270,10 @@ int i915_vma_move_to_active(struct i915_vma *vma,
> return err;
> }
>
> - dma_resv_add_shared_fence(vma->resv, &rq->fence);
> - obj->write_domain = 0;
> + if (fence) {
> + dma_resv_add_shared_fence(vma->resv, fence);
> + obj->write_domain = 0;
> + }
> }
>
> if (flags & EXEC_OBJECT_NEEDS_FENCE && vma->fence)
> diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vma.h
> index ed69f66c7ab0..648dbe744c96 100644
> --- a/drivers/gpu/drm/i915/i915_vma.h
> +++ b/drivers/gpu/drm/i915/i915_vma.h
> @@ -57,9 +57,16 @@ static inline bool i915_vma_is_active(const struct i915_vma *vma)
>
> int __must_check __i915_vma_move_to_active(struct i915_vma *vma,
> struct i915_request *rq);
> -int __must_check i915_vma_move_to_active(struct i915_vma *vma,
> - struct i915_request *rq,
> - unsigned int flags);
> +int __must_check _i915_vma_move_to_active(struct i915_vma *vma,
> + struct i915_request *rq,
> + struct dma_fence *fence,
> + unsigned int flags);
> +static inline int __must_check
> +i915_vma_move_to_active(struct i915_vma *vma, struct i915_request *rq,
> + unsigned int flags)
> +{
> + return _i915_vma_move_to_active(vma, rq, &rq->fence, flags);
> +}
>
> #define __i915_vma_flags(v) ((unsigned long *)&(v)->flags.counter)
>
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 20/25] drm/i915: Multi-BB execbuf
@ 2021-10-14 18:27 ` John Harrison
0 siblings, 0 replies; 69+ messages in thread
From: John Harrison @ 2021-10-14 18:27 UTC (permalink / raw)
To: Matthew Brost, intel-gfx, dri-devel
On 10/14/2021 10:20, Matthew Brost wrote:
> Allow multiple batch buffers to be submitted in a single execbuf IOCTL
> after a context has been configured with the 'set_parallel' extension.
> The number batches is implicit based on the contexts configuration.
>
> This is implemented with a series of loops. First a loop is used to find
> all the batches, a loop to pin all the HW contexts, a loop to create all
> the requests, a loop to submit (emit BB start, etc...) all the requests,
> a loop to tie the requests to the VMAs they touch, and finally a loop to
> commit the requests to the backend.
>
> A composite fence is also created for the generated requests to return
> to the user and to stick in dma resv slots.
>
> No behavior from the existing IOCTL should be changed aside from when
> throttling because the ring for a context is full. In this situation,
> i915 will now wait while holding the object locks. This change was done
> because the code is much simpler to wait while holding the locks and we
> believe there isn't a huge benefit of dropping these locks. If this
> proves false we can restructure the code to drop the locks during the
> wait.
>
> IGT: https://patchwork.freedesktop.org/patch/447008/?series=93071&rev=1
> media UMD: https://github.com/intel/media-driver/pull/1252
>
> v2:
> (Matthew Brost)
> - Return proper error value if i915_request_create fails
> v3:
> (John Harrison)
> - Add comment explaining create / add order loops + locking
> - Update commit message explaining different in IOCTL behavior
> - Line wrap some comments
> - eb_add_request returns void
> - Return -EINVAL rather triggering BUG_ON if cmd parser used
> (Checkpatch)
> - Check eb->batch_len[*current_batch]
> v4:
> (CI)
> - Set batch len if passed if via execbuf args
> - Call __i915_request_skip after __i915_request_commit
> (Kernel test robot)
> - Initialize rq to NULL in eb_pin_timeline
> v5:
> (John Harrison)
> - Fix typo in comments near bb order loops
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> .../gpu/drm/i915/gem/i915_gem_execbuffer.c | 783 ++++++++++++------
> drivers/gpu/drm/i915/gt/intel_context.h | 8 +-
> drivers/gpu/drm/i915/gt/intel_context_types.h | 10 +
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 +
> drivers/gpu/drm/i915/i915_request.h | 9 +
> drivers/gpu/drm/i915/i915_vma.c | 21 +-
> drivers/gpu/drm/i915/i915_vma.h | 13 +-
> 7 files changed, 595 insertions(+), 251 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
> index c75afc8784e3..fc30856e81fa 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
> @@ -246,17 +246,25 @@ struct i915_execbuffer {
> struct drm_i915_gem_exec_object2 *exec; /** ioctl execobj[] */
> struct eb_vma *vma;
>
> - struct intel_engine_cs *engine; /** engine to queue the request to */
> + struct intel_gt *gt; /* gt for the execbuf */
> struct intel_context *context; /* logical state for the request */
> struct i915_gem_context *gem_context; /** caller's context */
>
> - struct i915_request *request; /** our request to build */
> - struct eb_vma *batch; /** identity of the batch obj/vma */
> + /** our requests to build */
> + struct i915_request *requests[MAX_ENGINE_INSTANCE + 1];
> + /** identity of the batch obj/vma */
> + struct eb_vma *batches[MAX_ENGINE_INSTANCE + 1];
> struct i915_vma *trampoline; /** trampoline used for chaining */
>
> + /** used for excl fence in dma_resv objects when > 1 BB submitted */
> + struct dma_fence *composite_fence;
> +
> /** actual size of execobj[] as we may extend it for the cmdparser */
> unsigned int buffer_count;
>
> + /* number of batches in execbuf IOCTL */
> + unsigned int num_batches;
> +
> /** list of vma not yet bound during reservation phase */
> struct list_head unbound;
>
> @@ -283,7 +291,8 @@ struct i915_execbuffer {
>
> u64 invalid_flags; /** Set of execobj.flags that are invalid */
>
> - u64 batch_len; /** Length of batch within object */
> + /** Length of batch within object */
> + u64 batch_len[MAX_ENGINE_INSTANCE + 1];
> u32 batch_start_offset; /** Location within object of batch */
> u32 batch_flags; /** Flags composed for emit_bb_start() */
> struct intel_gt_buffer_pool_node *batch_pool; /** pool node for batch buffer */
> @@ -301,14 +310,13 @@ struct i915_execbuffer {
> };
>
> static int eb_parse(struct i915_execbuffer *eb);
> -static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb,
> - bool throttle);
> +static int eb_pin_engine(struct i915_execbuffer *eb, bool throttle);
> static void eb_unpin_engine(struct i915_execbuffer *eb);
>
> static inline bool eb_use_cmdparser(const struct i915_execbuffer *eb)
> {
> - return intel_engine_requires_cmd_parser(eb->engine) ||
> - (intel_engine_using_cmd_parser(eb->engine) &&
> + return intel_engine_requires_cmd_parser(eb->context->engine) ||
> + (intel_engine_using_cmd_parser(eb->context->engine) &&
> eb->args->batch_len);
> }
>
> @@ -546,11 +554,21 @@ eb_validate_vma(struct i915_execbuffer *eb,
> return 0;
> }
>
> -static void
> +static inline bool
> +is_batch_buffer(struct i915_execbuffer *eb, unsigned int buffer_idx)
> +{
> + return eb->args->flags & I915_EXEC_BATCH_FIRST ?
> + buffer_idx < eb->num_batches :
> + buffer_idx >= eb->args->buffer_count - eb->num_batches;
> +}
> +
> +static int
> eb_add_vma(struct i915_execbuffer *eb,
> - unsigned int i, unsigned batch_idx,
> + unsigned int *current_batch,
> + unsigned int i,
> struct i915_vma *vma)
> {
> + struct drm_i915_private *i915 = eb->i915;
> struct drm_i915_gem_exec_object2 *entry = &eb->exec[i];
> struct eb_vma *ev = &eb->vma[i];
>
> @@ -577,15 +595,43 @@ eb_add_vma(struct i915_execbuffer *eb,
> * Note that actual hangs have only been observed on gen7, but for
> * paranoia do it everywhere.
> */
> - if (i == batch_idx) {
> + if (is_batch_buffer(eb, i)) {
> if (entry->relocation_count &&
> !(ev->flags & EXEC_OBJECT_PINNED))
> ev->flags |= __EXEC_OBJECT_NEEDS_BIAS;
> if (eb->reloc_cache.has_fence)
> ev->flags |= EXEC_OBJECT_NEEDS_FENCE;
>
> - eb->batch = ev;
> + eb->batches[*current_batch] = ev;
> +
> + if (unlikely(ev->flags & EXEC_OBJECT_WRITE)) {
> + drm_dbg(&i915->drm,
> + "Attempting to use self-modifying batch buffer\n");
> + return -EINVAL;
> + }
> +
> + if (range_overflows_t(u64,
> + eb->batch_start_offset,
> + eb->args->batch_len,
> + ev->vma->size)) {
> + drm_dbg(&i915->drm, "Attempting to use out-of-bounds batch\n");
> + return -EINVAL;
> + }
> +
> + if (eb->args->batch_len == 0)
> + eb->batch_len[*current_batch] = ev->vma->size -
> + eb->batch_start_offset;
> + else
> + eb->batch_len[*current_batch] = eb->args->batch_len;
> + if (unlikely(eb->batch_len[*current_batch] == 0)) { /* impossible! */
> + drm_dbg(&i915->drm, "Invalid batch length\n");
> + return -EINVAL;
> + }
> +
> + ++*current_batch;
> }
> +
> + return 0;
> }
>
> static inline int use_cpu_reloc(const struct reloc_cache *cache,
> @@ -729,14 +775,6 @@ static int eb_reserve(struct i915_execbuffer *eb)
> } while (1);
> }
>
> -static unsigned int eb_batch_index(const struct i915_execbuffer *eb)
> -{
> - if (eb->args->flags & I915_EXEC_BATCH_FIRST)
> - return 0;
> - else
> - return eb->buffer_count - 1;
> -}
> -
> static int eb_select_context(struct i915_execbuffer *eb)
> {
> struct i915_gem_context *ctx;
> @@ -857,9 +895,7 @@ static struct i915_vma *eb_lookup_vma(struct i915_execbuffer *eb, u32 handle)
>
> static int eb_lookup_vmas(struct i915_execbuffer *eb)
> {
> - struct drm_i915_private *i915 = eb->i915;
> - unsigned int batch = eb_batch_index(eb);
> - unsigned int i;
> + unsigned int i, current_batch = 0;
> int err = 0;
>
> INIT_LIST_HEAD(&eb->relocs);
> @@ -879,7 +915,9 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
> goto err;
> }
>
> - eb_add_vma(eb, i, batch, vma);
> + err = eb_add_vma(eb, ¤t_batch, i, vma);
> + if (err)
> + return err;
>
> if (i915_gem_object_is_userptr(vma->obj)) {
> err = i915_gem_object_userptr_submit_init(vma->obj);
> @@ -902,26 +940,6 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
> }
> }
>
> - if (unlikely(eb->batch->flags & EXEC_OBJECT_WRITE)) {
> - drm_dbg(&i915->drm,
> - "Attempting to use self-modifying batch buffer\n");
> - return -EINVAL;
> - }
> -
> - if (range_overflows_t(u64,
> - eb->batch_start_offset, eb->batch_len,
> - eb->batch->vma->size)) {
> - drm_dbg(&i915->drm, "Attempting to use out-of-bounds batch\n");
> - return -EINVAL;
> - }
> -
> - if (eb->batch_len == 0)
> - eb->batch_len = eb->batch->vma->size - eb->batch_start_offset;
> - if (unlikely(eb->batch_len == 0)) { /* impossible! */
> - drm_dbg(&i915->drm, "Invalid batch length\n");
> - return -EINVAL;
> - }
> -
> return 0;
>
> err:
> @@ -1654,8 +1672,7 @@ static int eb_reinit_userptr(struct i915_execbuffer *eb)
> return 0;
> }
>
> -static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
> - struct i915_request *rq)
> +static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb)
> {
> bool have_copy = false;
> struct eb_vma *ev;
> @@ -1671,21 +1688,6 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
> eb_release_vmas(eb, false);
> i915_gem_ww_ctx_fini(&eb->ww);
>
> - if (rq) {
> - /* nonblocking is always false */
> - if (i915_request_wait(rq, I915_WAIT_INTERRUPTIBLE,
> - MAX_SCHEDULE_TIMEOUT) < 0) {
> - i915_request_put(rq);
> - rq = NULL;
> -
> - err = -EINTR;
> - goto err_relock;
> - }
> -
> - i915_request_put(rq);
> - rq = NULL;
> - }
> -
> /*
> * We take 3 passes through the slowpatch.
> *
> @@ -1712,28 +1714,21 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
> if (!err)
> err = eb_reinit_userptr(eb);
>
> -err_relock:
> i915_gem_ww_ctx_init(&eb->ww, true);
> if (err)
> goto out;
>
> /* reacquire the objects */
> repeat_validate:
> - rq = eb_pin_engine(eb, false);
> - if (IS_ERR(rq)) {
> - err = PTR_ERR(rq);
> - rq = NULL;
> + err = eb_pin_engine(eb, false);
> + if (err)
> goto err;
> - }
> -
> - /* We didn't throttle, should be NULL */
> - GEM_WARN_ON(rq);
>
> err = eb_validate_vmas(eb);
> if (err)
> goto err;
>
> - GEM_BUG_ON(!eb->batch);
> + GEM_BUG_ON(!eb->batches[0]);
>
> list_for_each_entry(ev, &eb->relocs, reloc_link) {
> if (!have_copy) {
> @@ -1797,46 +1792,23 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
> }
> }
>
> - if (rq)
> - i915_request_put(rq);
> -
> return err;
> }
>
> static int eb_relocate_parse(struct i915_execbuffer *eb)
> {
> int err;
> - struct i915_request *rq = NULL;
> bool throttle = true;
>
> retry:
> - rq = eb_pin_engine(eb, throttle);
> - if (IS_ERR(rq)) {
> - err = PTR_ERR(rq);
> - rq = NULL;
> + err = eb_pin_engine(eb, throttle);
> + if (err) {
> if (err != -EDEADLK)
> return err;
>
> goto err;
> }
>
> - if (rq) {
> - bool nonblock = eb->file->filp->f_flags & O_NONBLOCK;
> -
> - /* Need to drop all locks now for throttling, take slowpath */
> - err = i915_request_wait(rq, I915_WAIT_INTERRUPTIBLE, 0);
> - if (err == -ETIME) {
> - if (nonblock) {
> - err = -EWOULDBLOCK;
> - i915_request_put(rq);
> - goto err;
> - }
> - goto slow;
> - }
> - i915_request_put(rq);
> - rq = NULL;
> - }
> -
> /* only throttle once, even if we didn't need to throttle */
> throttle = false;
>
> @@ -1876,7 +1848,7 @@ static int eb_relocate_parse(struct i915_execbuffer *eb)
> return err;
>
> slow:
> - err = eb_relocate_parse_slow(eb, rq);
> + err = eb_relocate_parse_slow(eb);
> if (err)
> /*
> * If the user expects the execobject.offset and
> @@ -1890,11 +1862,40 @@ static int eb_relocate_parse(struct i915_execbuffer *eb)
> return err;
> }
>
> +/*
> + * Using two helper loops for the order of which requests / batches are created
> + * and added the to backend. Requests are created in order from the parent to
> + * the last child. Requests are added in the reverse order, from the last child
> + * to parent. This is done for locking reasons as the timeline lock is acquired
> + * during request creation and released when the request is added to the
> + * backend. To make lockdep happy (see intel_context_timeline_lock) this must be
> + * the ordering.
> + */
> +#define for_each_batch_create_order(_eb, _i) \
> + for ((_i) = 0; (_i) < (_eb)->num_batches; ++(_i))
> +#define for_each_batch_add_order(_eb, _i) \
> + BUILD_BUG_ON(!typecheck(int, _i)); \
> + for ((_i) = (_eb)->num_batches - 1; (_i) >= 0; --(_i))
> +
> +static struct i915_request *
> +eb_find_first_request_added(struct i915_execbuffer *eb)
> +{
> + int i;
> +
> + for_each_batch_add_order(eb, i)
> + if (eb->requests[i])
> + return eb->requests[i];
> +
> + GEM_BUG_ON("Request not found");
> +
> + return NULL;
> +}
> +
> static int eb_move_to_gpu(struct i915_execbuffer *eb)
> {
> const unsigned int count = eb->buffer_count;
> unsigned int i = count;
> - int err = 0;
> + int err = 0, j;
>
> while (i--) {
> struct eb_vma *ev = &eb->vma[i];
> @@ -1907,11 +1908,17 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
> if (flags & EXEC_OBJECT_CAPTURE) {
> struct i915_capture_list *capture;
>
> - capture = kmalloc(sizeof(*capture), GFP_KERNEL);
> - if (capture) {
> - capture->next = eb->request->capture_list;
> - capture->vma = vma;
> - eb->request->capture_list = capture;
> + for_each_batch_create_order(eb, j) {
> + if (!eb->requests[j])
> + break;
> +
> + capture = kmalloc(sizeof(*capture), GFP_KERNEL);
> + if (capture) {
> + capture->next =
> + eb->requests[j]->capture_list;
> + capture->vma = vma;
> + eb->requests[j]->capture_list = capture;
> + }
> }
> }
>
> @@ -1932,14 +1939,26 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
> flags &= ~EXEC_OBJECT_ASYNC;
> }
>
> + /* We only need to await on the first request */
> if (err == 0 && !(flags & EXEC_OBJECT_ASYNC)) {
> err = i915_request_await_object
> - (eb->request, obj, flags & EXEC_OBJECT_WRITE);
> + (eb_find_first_request_added(eb), obj,
> + flags & EXEC_OBJECT_WRITE);
> }
>
> - if (err == 0)
> - err = i915_vma_move_to_active(vma, eb->request,
> - flags | __EXEC_OBJECT_NO_RESERVE);
> + for_each_batch_add_order(eb, j) {
> + if (err)
> + break;
> + if (!eb->requests[j])
> + continue;
> +
> + err = _i915_vma_move_to_active(vma, eb->requests[j],
> + j ? NULL :
> + eb->composite_fence ?
> + eb->composite_fence :
> + &eb->requests[j]->fence,
> + flags | __EXEC_OBJECT_NO_RESERVE);
> + }
> }
>
> #ifdef CONFIG_MMU_NOTIFIER
> @@ -1970,11 +1989,16 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
> goto err_skip;
>
> /* Unconditionally flush any chipset caches (for streaming writes). */
> - intel_gt_chipset_flush(eb->engine->gt);
> + intel_gt_chipset_flush(eb->gt);
> return 0;
>
> err_skip:
> - i915_request_set_error_once(eb->request, err);
> + for_each_batch_create_order(eb, j) {
> + if (!eb->requests[j])
> + break;
> +
> + i915_request_set_error_once(eb->requests[j], err);
> + }
> return err;
> }
>
> @@ -2069,14 +2093,17 @@ static int eb_parse(struct i915_execbuffer *eb)
> int err;
>
> if (!eb_use_cmdparser(eb)) {
> - batch = eb_dispatch_secure(eb, eb->batch->vma);
> + batch = eb_dispatch_secure(eb, eb->batches[0]->vma);
> if (IS_ERR(batch))
> return PTR_ERR(batch);
>
> goto secure_batch;
> }
>
> - len = eb->batch_len;
> + if (intel_context_is_parallel(eb->context))
> + return -EINVAL;
> +
> + len = eb->batch_len[0];
> if (!CMDPARSER_USES_GGTT(eb->i915)) {
> /*
> * ppGTT backed shadow buffers must be mapped RO, to prevent
> @@ -2090,11 +2117,11 @@ static int eb_parse(struct i915_execbuffer *eb)
> } else {
> len += I915_CMD_PARSER_TRAMPOLINE_SIZE;
> }
> - if (unlikely(len < eb->batch_len)) /* last paranoid check of overflow */
> + if (unlikely(len < eb->batch_len[0])) /* last paranoid check of overflow */
> return -EINVAL;
>
> if (!pool) {
> - pool = intel_gt_get_buffer_pool(eb->engine->gt, len,
> + pool = intel_gt_get_buffer_pool(eb->gt, len,
> I915_MAP_WB);
> if (IS_ERR(pool))
> return PTR_ERR(pool);
> @@ -2119,7 +2146,7 @@ static int eb_parse(struct i915_execbuffer *eb)
> trampoline = shadow;
>
> shadow = shadow_batch_pin(eb, pool->obj,
> - &eb->engine->gt->ggtt->vm,
> + &eb->gt->ggtt->vm,
> PIN_GLOBAL);
> if (IS_ERR(shadow)) {
> err = PTR_ERR(shadow);
> @@ -2141,26 +2168,29 @@ static int eb_parse(struct i915_execbuffer *eb)
> if (err)
> goto err_trampoline;
>
> - err = intel_engine_cmd_parser(eb->engine,
> - eb->batch->vma,
> + err = intel_engine_cmd_parser(eb->context->engine,
> + eb->batches[0]->vma,
> eb->batch_start_offset,
> - eb->batch_len,
> + eb->batch_len[0],
> shadow, trampoline);
> if (err)
> goto err_unpin_batch;
>
> - eb->batch = &eb->vma[eb->buffer_count++];
> - eb->batch->vma = i915_vma_get(shadow);
> - eb->batch->flags = __EXEC_OBJECT_HAS_PIN;
> + eb->batches[0] = &eb->vma[eb->buffer_count++];
> + eb->batches[0]->vma = i915_vma_get(shadow);
> + eb->batches[0]->flags = __EXEC_OBJECT_HAS_PIN;
>
> eb->trampoline = trampoline;
> eb->batch_start_offset = 0;
>
> secure_batch:
> if (batch) {
> - eb->batch = &eb->vma[eb->buffer_count++];
> - eb->batch->flags = __EXEC_OBJECT_HAS_PIN;
> - eb->batch->vma = i915_vma_get(batch);
> + if (intel_context_is_parallel(eb->context))
> + return -EINVAL;
> +
> + eb->batches[0] = &eb->vma[eb->buffer_count++];
> + eb->batches[0]->flags = __EXEC_OBJECT_HAS_PIN;
> + eb->batches[0]->vma = i915_vma_get(batch);
> }
> return 0;
>
> @@ -2176,19 +2206,18 @@ static int eb_parse(struct i915_execbuffer *eb)
> return err;
> }
>
> -static int eb_submit(struct i915_execbuffer *eb, struct i915_vma *batch)
> +static int eb_request_submit(struct i915_execbuffer *eb,
> + struct i915_request *rq,
> + struct i915_vma *batch,
> + u64 batch_len)
> {
> int err;
>
> - if (intel_context_nopreempt(eb->context))
> - __set_bit(I915_FENCE_FLAG_NOPREEMPT, &eb->request->fence.flags);
> -
> - err = eb_move_to_gpu(eb);
> - if (err)
> - return err;
> + if (intel_context_nopreempt(rq->context))
> + __set_bit(I915_FENCE_FLAG_NOPREEMPT, &rq->fence.flags);
>
> if (eb->args->flags & I915_EXEC_GEN7_SOL_RESET) {
> - err = i915_reset_gen7_sol_offsets(eb->request);
> + err = i915_reset_gen7_sol_offsets(rq);
> if (err)
> return err;
> }
> @@ -2199,26 +2228,26 @@ static int eb_submit(struct i915_execbuffer *eb, struct i915_vma *batch)
> * allows us to determine if the batch is still waiting on the GPU
> * or actually running by checking the breadcrumb.
> */
> - if (eb->engine->emit_init_breadcrumb) {
> - err = eb->engine->emit_init_breadcrumb(eb->request);
> + if (rq->context->engine->emit_init_breadcrumb) {
> + err = rq->context->engine->emit_init_breadcrumb(rq);
> if (err)
> return err;
> }
>
> - err = eb->engine->emit_bb_start(eb->request,
> - batch->node.start +
> - eb->batch_start_offset,
> - eb->batch_len,
> - eb->batch_flags);
> + err = rq->context->engine->emit_bb_start(rq,
> + batch->node.start +
> + eb->batch_start_offset,
> + batch_len,
> + eb->batch_flags);
> if (err)
> return err;
>
> if (eb->trampoline) {
> + GEM_BUG_ON(intel_context_is_parallel(rq->context));
> GEM_BUG_ON(eb->batch_start_offset);
> - err = eb->engine->emit_bb_start(eb->request,
> - eb->trampoline->node.start +
> - eb->batch_len,
> - 0, 0);
> + err = rq->context->engine->emit_bb_start(rq,
> + eb->trampoline->node.start +
> + batch_len, 0, 0);
> if (err)
> return err;
> }
> @@ -2226,6 +2255,27 @@ static int eb_submit(struct i915_execbuffer *eb, struct i915_vma *batch)
> return 0;
> }
>
> +static int eb_submit(struct i915_execbuffer *eb)
> +{
> + unsigned int i;
> + int err;
> +
> + err = eb_move_to_gpu(eb);
> +
> + for_each_batch_create_order(eb, i) {
> + if (!eb->requests[i])
> + break;
> +
> + trace_i915_request_queue(eb->requests[i], eb->batch_flags);
> + if (!err)
> + err = eb_request_submit(eb, eb->requests[i],
> + eb->batches[i]->vma,
> + eb->batch_len[i]);
> + }
> +
> + return err;
> +}
> +
> static int num_vcs_engines(const struct drm_i915_private *i915)
> {
> return hweight_long(VDBOX_MASK(&i915->gt));
> @@ -2291,26 +2341,11 @@ static struct i915_request *eb_throttle(struct i915_execbuffer *eb, struct intel
> return i915_request_get(rq);
> }
>
> -static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb, bool throttle)
> +static int eb_pin_timeline(struct i915_execbuffer *eb, struct intel_context *ce,
> + bool throttle)
> {
> - struct intel_context *ce = eb->context;
> struct intel_timeline *tl;
> struct i915_request *rq = NULL;
> - int err;
> -
> - GEM_BUG_ON(eb->args->flags & __EXEC_ENGINE_PINNED);
> -
> - if (unlikely(intel_context_is_banned(ce)))
> - return ERR_PTR(-EIO);
> -
> - /*
> - * Pinning the contexts may generate requests in order to acquire
> - * GGTT space, so do this first before we reserve a seqno for
> - * ourselves.
> - */
> - err = intel_context_pin_ww(ce, &eb->ww);
> - if (err)
> - return ERR_PTR(err);
>
> /*
> * Take a local wakeref for preparing to dispatch the execbuf as
> @@ -2321,33 +2356,108 @@ static struct i915_request *eb_pin_engine(struct i915_execbuffer *eb, bool throt
> * taken on the engine, and the parent device.
> */
> tl = intel_context_timeline_lock(ce);
> - if (IS_ERR(tl)) {
> - intel_context_unpin(ce);
> - return ERR_CAST(tl);
> - }
> + if (IS_ERR(tl))
> + return PTR_ERR(tl);
>
> intel_context_enter(ce);
> if (throttle)
> rq = eb_throttle(eb, ce);
> intel_context_timeline_unlock(tl);
>
> + if (rq) {
> + bool nonblock = eb->file->filp->f_flags & O_NONBLOCK;
> + long timeout = nonblock ? 0 : MAX_SCHEDULE_TIMEOUT;
> +
> + if (i915_request_wait(rq, I915_WAIT_INTERRUPTIBLE,
> + timeout) < 0) {
> + i915_request_put(rq);
> +
> + tl = intel_context_timeline_lock(ce);
> + intel_context_exit(ce);
> + intel_context_timeline_unlock(tl);
> +
> + if (nonblock)
> + return -EWOULDBLOCK;
> + else
> + return -EINTR;
> + }
> + i915_request_put(rq);
> + }
> +
> + return 0;
> +}
> +
> +static int eb_pin_engine(struct i915_execbuffer *eb, bool throttle)
> +{
> + struct intel_context *ce = eb->context, *child;
> + int err;
> + int i = 0, j = 0;
> +
> + GEM_BUG_ON(eb->args->flags & __EXEC_ENGINE_PINNED);
> +
> + if (unlikely(intel_context_is_banned(ce)))
> + return -EIO;
> +
> + /*
> + * Pinning the contexts may generate requests in order to acquire
> + * GGTT space, so do this first before we reserve a seqno for
> + * ourselves.
> + */
> + err = intel_context_pin_ww(ce, &eb->ww);
> + if (err)
> + return err;
> + for_each_child(ce, child) {
> + err = intel_context_pin_ww(child, &eb->ww);
> + GEM_BUG_ON(err); /* perma-pinned should incr a counter */
> + }
> +
> + for_each_child(ce, child) {
> + err = eb_pin_timeline(eb, child, throttle);
> + if (err)
> + goto unwind;
> + ++i;
> + }
> + err = eb_pin_timeline(eb, ce, throttle);
> + if (err)
> + goto unwind;
> +
> eb->args->flags |= __EXEC_ENGINE_PINNED;
> - return rq;
> + return 0;
> +
> +unwind:
> + for_each_child(ce, child) {
> + if (j++ < i) {
> + mutex_lock(&child->timeline->mutex);
> + intel_context_exit(child);
> + mutex_unlock(&child->timeline->mutex);
> + }
> + }
> + for_each_child(ce, child)
> + intel_context_unpin(child);
> + intel_context_unpin(ce);
> + return err;
> }
>
> static void eb_unpin_engine(struct i915_execbuffer *eb)
> {
> - struct intel_context *ce = eb->context;
> - struct intel_timeline *tl = ce->timeline;
> + struct intel_context *ce = eb->context, *child;
>
> if (!(eb->args->flags & __EXEC_ENGINE_PINNED))
> return;
>
> eb->args->flags &= ~__EXEC_ENGINE_PINNED;
>
> - mutex_lock(&tl->mutex);
> + for_each_child(ce, child) {
> + mutex_lock(&child->timeline->mutex);
> + intel_context_exit(child);
> + mutex_unlock(&child->timeline->mutex);
> +
> + intel_context_unpin(child);
> + }
> +
> + mutex_lock(&ce->timeline->mutex);
> intel_context_exit(ce);
> - mutex_unlock(&tl->mutex);
> + mutex_unlock(&ce->timeline->mutex);
>
> intel_context_unpin(ce);
> }
> @@ -2398,7 +2508,7 @@ eb_select_legacy_ring(struct i915_execbuffer *eb)
> static int
> eb_select_engine(struct i915_execbuffer *eb)
> {
> - struct intel_context *ce;
> + struct intel_context *ce, *child;
> unsigned int idx;
> int err;
>
> @@ -2411,6 +2521,20 @@ eb_select_engine(struct i915_execbuffer *eb)
> if (IS_ERR(ce))
> return PTR_ERR(ce);
>
> + if (intel_context_is_parallel(ce)) {
> + if (eb->buffer_count < ce->parallel.number_children + 1) {
> + intel_context_put(ce);
> + return -EINVAL;
> + }
> + if (eb->batch_start_offset || eb->args->batch_len) {
> + intel_context_put(ce);
> + return -EINVAL;
> + }
> + }
> + eb->num_batches = ce->parallel.number_children + 1;
> +
> + for_each_child(ce, child)
> + intel_context_get(child);
> intel_gt_pm_get(ce->engine->gt);
>
> if (!test_bit(CONTEXT_ALLOC_BIT, &ce->flags)) {
> @@ -2418,6 +2542,13 @@ eb_select_engine(struct i915_execbuffer *eb)
> if (err)
> goto err;
> }
> + for_each_child(ce, child) {
> + if (!test_bit(CONTEXT_ALLOC_BIT, &child->flags)) {
> + err = intel_context_alloc_state(child);
> + if (err)
> + goto err;
> + }
> + }
>
> /*
> * ABI: Before userspace accesses the GPU (e.g. execbuffer), report
> @@ -2428,7 +2559,7 @@ eb_select_engine(struct i915_execbuffer *eb)
> goto err;
>
> eb->context = ce;
> - eb->engine = ce->engine;
> + eb->gt = ce->engine->gt;
>
> /*
> * Make sure engine pool stays alive even if we call intel_context_put
> @@ -2439,6 +2570,8 @@ eb_select_engine(struct i915_execbuffer *eb)
>
> err:
> intel_gt_pm_put(ce->engine->gt);
> + for_each_child(ce, child)
> + intel_context_put(child);
> intel_context_put(ce);
> return err;
> }
> @@ -2446,7 +2579,11 @@ eb_select_engine(struct i915_execbuffer *eb)
> static void
> eb_put_engine(struct i915_execbuffer *eb)
> {
> - intel_gt_pm_put(eb->engine->gt);
> + struct intel_context *child;
> +
> + intel_gt_pm_put(eb->gt);
> + for_each_child(eb->context, child)
> + intel_context_put(child);
> intel_context_put(eb->context);
> }
>
> @@ -2669,7 +2806,8 @@ static void put_fence_array(struct eb_fence *fences, int num_fences)
> }
>
> static int
> -await_fence_array(struct i915_execbuffer *eb)
> +await_fence_array(struct i915_execbuffer *eb,
> + struct i915_request *rq)
> {
> unsigned int n;
> int err;
> @@ -2683,8 +2821,7 @@ await_fence_array(struct i915_execbuffer *eb)
> if (!eb->fences[n].dma_fence)
> continue;
>
> - err = i915_request_await_dma_fence(eb->request,
> - eb->fences[n].dma_fence);
> + err = i915_request_await_dma_fence(rq, eb->fences[n].dma_fence);
> if (err < 0)
> return err;
> }
> @@ -2692,9 +2829,9 @@ await_fence_array(struct i915_execbuffer *eb)
> return 0;
> }
>
> -static void signal_fence_array(const struct i915_execbuffer *eb)
> +static void signal_fence_array(const struct i915_execbuffer *eb,
> + struct dma_fence * const fence)
> {
> - struct dma_fence * const fence = &eb->request->fence;
> unsigned int n;
>
> for (n = 0; n < eb->num_fences; n++) {
> @@ -2742,9 +2879,9 @@ static void retire_requests(struct intel_timeline *tl, struct i915_request *end)
> break;
> }
>
> -static int eb_request_add(struct i915_execbuffer *eb, int err)
> +static int eb_request_add(struct i915_execbuffer *eb, struct i915_request *rq,
> + int err, bool last_parallel)
> {
> - struct i915_request *rq = eb->request;
> struct intel_timeline * const tl = i915_request_timeline(rq);
> struct i915_sched_attr attr = {};
> struct i915_request *prev;
> @@ -2766,6 +2903,17 @@ static int eb_request_add(struct i915_execbuffer *eb, int err)
> err = -ENOENT; /* override any transient errors */
> }
>
> + if (intel_context_is_parallel(eb->context)) {
> + if (err) {
> + __i915_request_skip(rq);
> + set_bit(I915_FENCE_FLAG_SKIP_PARALLEL,
> + &rq->fence.flags);
> + }
> + if (last_parallel)
> + set_bit(I915_FENCE_FLAG_SUBMIT_PARALLEL,
> + &rq->fence.flags);
> + }
> +
> __i915_request_queue(rq, &attr);
>
> /* Try to clean up the client's timeline after submitting the request */
> @@ -2777,6 +2925,25 @@ static int eb_request_add(struct i915_execbuffer *eb, int err)
> return err;
> }
>
> +static int eb_requests_add(struct i915_execbuffer *eb, int err)
> +{
> + int i;
> +
> + /*
> + * We iterate in reverse order of creation to release timeline mutexes in
> + * same order.
> + */
> + for_each_batch_add_order(eb, i) {
> + struct i915_request *rq = eb->requests[i];
> +
> + if (!rq)
> + continue;
> + err |= eb_request_add(eb, rq, err, i == 0);
> + }
> +
> + return err;
> +}
> +
> static const i915_user_extension_fn execbuf_extensions[] = {
> [DRM_I915_GEM_EXECBUFFER_EXT_TIMELINE_FENCES] = parse_timeline_fences,
> };
> @@ -2803,6 +2970,182 @@ parse_execbuf2_extensions(struct drm_i915_gem_execbuffer2 *args,
> eb);
> }
>
> +static void eb_requests_get(struct i915_execbuffer *eb)
> +{
> + unsigned int i;
> +
> + for_each_batch_create_order(eb, i) {
> + if (!eb->requests[i])
> + break;
> +
> + i915_request_get(eb->requests[i]);
> + }
> +}
> +
> +static void eb_requests_put(struct i915_execbuffer *eb)
> +{
> + unsigned int i;
> +
> + for_each_batch_create_order(eb, i) {
> + if (!eb->requests[i])
> + break;
> +
> + i915_request_put(eb->requests[i]);
> + }
> +}
> +
> +static struct sync_file *
> +eb_composite_fence_create(struct i915_execbuffer *eb, int out_fence_fd)
> +{
> + struct sync_file *out_fence = NULL;
> + struct dma_fence_array *fence_array;
> + struct dma_fence **fences;
> + unsigned int i;
> +
> + GEM_BUG_ON(!intel_context_is_parent(eb->context));
> +
> + fences = kmalloc_array(eb->num_batches, sizeof(*fences), GFP_KERNEL);
> + if (!fences)
> + return ERR_PTR(-ENOMEM);
> +
> + for_each_batch_create_order(eb, i)
> + fences[i] = &eb->requests[i]->fence;
> +
> + fence_array = dma_fence_array_create(eb->num_batches,
> + fences,
> + eb->context->parallel.fence_context,
> + eb->context->parallel.seqno,
> + false);
> + if (!fence_array) {
> + kfree(fences);
> + return ERR_PTR(-ENOMEM);
> + }
> +
> + /* Move ownership to the dma_fence_array created above */
> + for_each_batch_create_order(eb, i)
> + dma_fence_get(fences[i]);
> +
> + if (out_fence_fd != -1) {
> + out_fence = sync_file_create(&fence_array->base);
> + /* sync_file now owns fence_arry, drop creation ref */
> + dma_fence_put(&fence_array->base);
> + if (!out_fence)
> + return ERR_PTR(-ENOMEM);
> + }
> +
> + eb->composite_fence = &fence_array->base;
> +
> + return out_fence;
> +}
> +
> +static struct sync_file *
> +eb_fences_add(struct i915_execbuffer *eb, struct i915_request *rq,
> + struct dma_fence *in_fence, int out_fence_fd)
> +{
> + struct sync_file *out_fence = NULL;
> + int err;
> +
> + if (unlikely(eb->gem_context->syncobj)) {
> + struct dma_fence *fence;
> +
> + fence = drm_syncobj_fence_get(eb->gem_context->syncobj);
> + err = i915_request_await_dma_fence(rq, fence);
> + dma_fence_put(fence);
> + if (err)
> + return ERR_PTR(err);
> + }
> +
> + if (in_fence) {
> + if (eb->args->flags & I915_EXEC_FENCE_SUBMIT)
> + err = i915_request_await_execution(rq, in_fence);
> + else
> + err = i915_request_await_dma_fence(rq, in_fence);
> + if (err < 0)
> + return ERR_PTR(err);
> + }
> +
> + if (eb->fences) {
> + err = await_fence_array(eb, rq);
> + if (err)
> + return ERR_PTR(err);
> + }
> +
> + if (intel_context_is_parallel(eb->context)) {
> + out_fence = eb_composite_fence_create(eb, out_fence_fd);
> + if (IS_ERR(out_fence))
> + return ERR_PTR(-ENOMEM);
> + } else if (out_fence_fd != -1) {
> + out_fence = sync_file_create(&rq->fence);
> + if (!out_fence)
> + return ERR_PTR(-ENOMEM);
> + }
> +
> + return out_fence;
> +}
> +
> +static struct intel_context *
> +eb_find_context(struct i915_execbuffer *eb, unsigned int context_number)
> +{
> + struct intel_context *child;
> +
> + if (likely(context_number == 0))
> + return eb->context;
> +
> + for_each_child(eb->context, child)
> + if (!--context_number)
> + return child;
> +
> + GEM_BUG_ON("Context not found");
> +
> + return NULL;
> +}
> +
> +static struct sync_file *
> +eb_requests_create(struct i915_execbuffer *eb, struct dma_fence *in_fence,
> + int out_fence_fd)
> +{
> + struct sync_file *out_fence = NULL;
> + unsigned int i;
> +
> + for_each_batch_create_order(eb, i) {
> + /* Allocate a request for this batch buffer nice and early. */
> + eb->requests[i] = i915_request_create(eb_find_context(eb, i));
> + if (IS_ERR(eb->requests[i])) {
> + out_fence = ERR_PTR(PTR_ERR(eb->requests[i]));
> + eb->requests[i] = NULL;
> + return out_fence;
> + }
> +
> + /*
> + * Only the first request added (committed to backend) has to
> + * take the in fences into account as all subsequent requests
> + * will have fences inserted inbetween them.
> + */
> + if (i + 1 == eb->num_batches) {
> + out_fence = eb_fences_add(eb, eb->requests[i],
> + in_fence, out_fence_fd);
> + if (IS_ERR(out_fence))
> + return out_fence;
> + }
> +
> + /*
> + * Whilst this request exists, batch_obj will be on the
> + * active_list, and so will hold the active reference. Only when
> + * this request is retired will the batch_obj be moved onto
> + * the inactive_list and lose its active reference. Hence we do
> + * not need to explicitly hold another reference here.
> + */
> + eb->requests[i]->batch = eb->batches[i]->vma;
> + if (eb->batch_pool) {
> + GEM_BUG_ON(intel_context_is_parallel(eb->context));
> + intel_gt_buffer_pool_mark_active(eb->batch_pool,
> + eb->requests[i]);
> + }
> + }
> +
> + return out_fence;
> +}
> +
> static int
> i915_gem_do_execbuffer(struct drm_device *dev,
> struct drm_file *file,
> @@ -2813,7 +3156,6 @@ i915_gem_do_execbuffer(struct drm_device *dev,
> struct i915_execbuffer eb;
> struct dma_fence *in_fence = NULL;
> struct sync_file *out_fence = NULL;
> - struct i915_vma *batch;
> int out_fence_fd = -1;
> int err;
>
> @@ -2837,12 +3179,15 @@ i915_gem_do_execbuffer(struct drm_device *dev,
>
> eb.buffer_count = args->buffer_count;
> eb.batch_start_offset = args->batch_start_offset;
> - eb.batch_len = args->batch_len;
> eb.trampoline = NULL;
>
> eb.fences = NULL;
> eb.num_fences = 0;
>
> + memset(eb.requests, 0, sizeof(struct i915_request *) *
> + ARRAY_SIZE(eb.requests));
> + eb.composite_fence = NULL;
> +
> eb.batch_flags = 0;
> if (args->flags & I915_EXEC_SECURE) {
> if (GRAPHICS_VER(i915) >= 11)
> @@ -2926,70 +3271,25 @@ i915_gem_do_execbuffer(struct drm_device *dev,
>
> ww_acquire_done(&eb.ww.ctx);
>
> - batch = eb.batch->vma;
> -
> - /* Allocate a request for this batch buffer nice and early. */
> - eb.request = i915_request_create(eb.context);
> - if (IS_ERR(eb.request)) {
> - err = PTR_ERR(eb.request);
> - goto err_vma;
> - }
> -
> - if (unlikely(eb.gem_context->syncobj)) {
> - struct dma_fence *fence;
> -
> - fence = drm_syncobj_fence_get(eb.gem_context->syncobj);
> - err = i915_request_await_dma_fence(eb.request, fence);
> - dma_fence_put(fence);
> - if (err)
> - goto err_ext;
> - }
> -
> - if (in_fence) {
> - if (args->flags & I915_EXEC_FENCE_SUBMIT)
> - err = i915_request_await_execution(eb.request,
> - in_fence);
> - else
> - err = i915_request_await_dma_fence(eb.request,
> - in_fence);
> - if (err < 0)
> - goto err_request;
> - }
> -
> - if (eb.fences) {
> - err = await_fence_array(&eb);
> - if (err)
> + out_fence = eb_requests_create(&eb, in_fence, out_fence_fd);
> + if (IS_ERR(out_fence)) {
> + err = PTR_ERR(out_fence);
> + if (eb.requests[0])
> goto err_request;
> + else
> + goto err_vma;
> }
>
> - if (out_fence_fd != -1) {
> - out_fence = sync_file_create(&eb.request->fence);
> - if (!out_fence) {
> - err = -ENOMEM;
> - goto err_request;
> - }
> - }
> -
> - /*
> - * Whilst this request exists, batch_obj will be on the
> - * active_list, and so will hold the active reference. Only when this
> - * request is retired will the the batch_obj be moved onto the
> - * inactive_list and lose its active reference. Hence we do not need
> - * to explicitly hold another reference here.
> - */
> - eb.request->batch = batch;
> - if (eb.batch_pool)
> - intel_gt_buffer_pool_mark_active(eb.batch_pool, eb.request);
> -
> - trace_i915_request_queue(eb.request, eb.batch_flags);
> - err = eb_submit(&eb, batch);
> + err = eb_submit(&eb);
>
> err_request:
> - i915_request_get(eb.request);
> - err = eb_request_add(&eb, err);
> + eb_requests_get(&eb);
> + err = eb_requests_add(&eb, err);
>
> if (eb.fences)
> - signal_fence_array(&eb);
> + signal_fence_array(&eb, eb.composite_fence ?
> + eb.composite_fence :
> + &eb.requests[0]->fence);
>
> if (out_fence) {
> if (err == 0) {
> @@ -3004,10 +3304,15 @@ i915_gem_do_execbuffer(struct drm_device *dev,
>
> if (unlikely(eb.gem_context->syncobj)) {
> drm_syncobj_replace_fence(eb.gem_context->syncobj,
> - &eb.request->fence);
> + eb.composite_fence ?
> + eb.composite_fence :
> + &eb.requests[0]->fence);
> }
>
> - i915_request_put(eb.request);
> + if (!out_fence && eb.composite_fence)
> + dma_fence_put(eb.composite_fence);
> +
> + eb_requests_put(&eb);
>
> err_vma:
> eb_release_vmas(&eb, true);
> diff --git a/drivers/gpu/drm/i915/gt/intel_context.h b/drivers/gpu/drm/i915/gt/intel_context.h
> index edf12caaade3..246c37d72cd7 100644
> --- a/drivers/gpu/drm/i915/gt/intel_context.h
> +++ b/drivers/gpu/drm/i915/gt/intel_context.h
> @@ -241,7 +241,13 @@ intel_context_timeline_lock(struct intel_context *ce)
> struct intel_timeline *tl = ce->timeline;
> int err;
>
> - err = mutex_lock_interruptible(&tl->mutex);
> + if (intel_context_is_parent(ce))
> + err = mutex_lock_interruptible_nested(&tl->mutex, 0);
> + else if (intel_context_is_child(ce))
> + err = mutex_lock_interruptible_nested(&tl->mutex,
> + ce->parallel.child_index + 1);
> + else
> + err = mutex_lock_interruptible(&tl->mutex);
> if (err)
> return ERR_PTR(err);
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_context_types.h b/drivers/gpu/drm/i915/gt/intel_context_types.h
> index 95a5b94b4ece..9e0177dc5484 100644
> --- a/drivers/gpu/drm/i915/gt/intel_context_types.h
> +++ b/drivers/gpu/drm/i915/gt/intel_context_types.h
> @@ -248,6 +248,16 @@ struct intel_context {
> * context
> */
> struct i915_request *last_rq;
> + /**
> + * @fence_context: fence context composite fence when doing
> + * parallel submission
> + */
> + u64 fence_context;
> + /**
> + * @seqno: seqno for composite fence when doing parallel
> + * submission
> + */
> + u32 seqno;
> /** @number_children: number of children if parent */
> u8 number_children;
> /** @child_index: index into child_list if child */
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index 50f0f4eba03b..361fab2cae99 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -3093,6 +3093,8 @@ guc_create_parallel(struct intel_engine_cs **engines,
> }
> }
>
> + parent->parallel.fence_context = dma_fence_context_alloc(1);
> +
> parent->engine->emit_bb_start =
> emit_bb_start_parent_no_preempt_mid_batch;
> parent->engine->emit_fini_breadcrumb =
> diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
> index 8950785e55d6..24db8459376b 100644
> --- a/drivers/gpu/drm/i915/i915_request.h
> +++ b/drivers/gpu/drm/i915/i915_request.h
> @@ -147,6 +147,15 @@ enum {
> * tail.
> */
> I915_FENCE_FLAG_SUBMIT_PARALLEL,
> +
> + /*
> + * I915_FENCE_FLAG_SKIP_PARALLEL - request with a context in a
> + * parent-child relationship (parallel submission, multi-lrc) that
> + * hit an error while generating requests in the execbuf IOCTL.
> + * Indicates this request should be skipped as another request in
> + * submission / relationship encoutered an error.
> + */
> + I915_FENCE_FLAG_SKIP_PARALLEL,
> };
>
> /**
> diff --git a/drivers/gpu/drm/i915/i915_vma.c b/drivers/gpu/drm/i915/i915_vma.c
> index 4b7fc4647e46..90546fa58fc1 100644
> --- a/drivers/gpu/drm/i915/i915_vma.c
> +++ b/drivers/gpu/drm/i915/i915_vma.c
> @@ -1234,9 +1234,10 @@ int __i915_vma_move_to_active(struct i915_vma *vma, struct i915_request *rq)
> return i915_active_add_request(&vma->active, rq);
> }
>
> -int i915_vma_move_to_active(struct i915_vma *vma,
> - struct i915_request *rq,
> - unsigned int flags)
> +int _i915_vma_move_to_active(struct i915_vma *vma,
> + struct i915_request *rq,
> + struct dma_fence *fence,
> + unsigned int flags)
> {
> struct drm_i915_gem_object *obj = vma->obj;
> int err;
> @@ -1257,9 +1258,11 @@ int i915_vma_move_to_active(struct i915_vma *vma,
> intel_frontbuffer_put(front);
> }
>
> - dma_resv_add_excl_fence(vma->resv, &rq->fence);
> - obj->write_domain = I915_GEM_DOMAIN_RENDER;
> - obj->read_domains = 0;
> + if (fence) {
> + dma_resv_add_excl_fence(vma->resv, fence);
> + obj->write_domain = I915_GEM_DOMAIN_RENDER;
> + obj->read_domains = 0;
> + }
> } else {
> if (!(flags & __EXEC_OBJECT_NO_RESERVE)) {
> err = dma_resv_reserve_shared(vma->resv, 1);
> @@ -1267,8 +1270,10 @@ int i915_vma_move_to_active(struct i915_vma *vma,
> return err;
> }
>
> - dma_resv_add_shared_fence(vma->resv, &rq->fence);
> - obj->write_domain = 0;
> + if (fence) {
> + dma_resv_add_shared_fence(vma->resv, fence);
> + obj->write_domain = 0;
> + }
> }
>
> if (flags & EXEC_OBJECT_NEEDS_FENCE && vma->fence)
> diff --git a/drivers/gpu/drm/i915/i915_vma.h b/drivers/gpu/drm/i915/i915_vma.h
> index ed69f66c7ab0..648dbe744c96 100644
> --- a/drivers/gpu/drm/i915/i915_vma.h
> +++ b/drivers/gpu/drm/i915/i915_vma.h
> @@ -57,9 +57,16 @@ static inline bool i915_vma_is_active(const struct i915_vma *vma)
>
> int __must_check __i915_vma_move_to_active(struct i915_vma *vma,
> struct i915_request *rq);
> -int __must_check i915_vma_move_to_active(struct i915_vma *vma,
> - struct i915_request *rq,
> - unsigned int flags);
> +int __must_check _i915_vma_move_to_active(struct i915_vma *vma,
> + struct i915_request *rq,
> + struct dma_fence *fence,
> + unsigned int flags);
> +static inline int __must_check
> +i915_vma_move_to_active(struct i915_vma *vma, struct i915_request *rq,
> + unsigned int flags)
> +{
> + return _i915_vma_move_to_active(vma, rq, &rq->fence, flags);
> +}
>
> #define __i915_vma_flags(v) ((unsigned long *)&(v)->flags.counter)
>
^ permalink raw reply [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 21/25] drm/i915/guc: Handle errors in multi-lrc requests
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:20 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:20 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
If an error occurs in the front end when multi-lrc requests are getting
generated we need to skip these in the backend but we still need to
emit the breadcrumbs seqno. An issues arises because with multi-lrc
breadcrumbs there is a handshake between the parent and children to make
forward progress. If all the requests are not present this handshake
doesn't work. To work around this, if multi-lrc request has an error we
skip the handshake but still emit the breadcrumbs seqno.
v2:
(John Harrison)
- Add comment explaining the skipping of the handshake logic
- Fix typos in the commit message
v3:
(John Harrison)
- Fix up some comments about the math to NOP the ring
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 69 ++++++++++++++++++-
1 file changed, 66 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 361fab2cae99..d7710debcd47 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -4070,8 +4070,8 @@ static int emit_bb_start_child_no_preempt_mid_batch(struct i915_request *rq,
}
static u32 *
-emit_fini_breadcrumb_parent_no_preempt_mid_batch(struct i915_request *rq,
- u32 *cs)
+__emit_fini_breadcrumb_parent_no_preempt_mid_batch(struct i915_request *rq,
+ u32 *cs)
{
struct intel_context *ce = rq->context;
u8 i;
@@ -4099,6 +4099,45 @@ emit_fini_breadcrumb_parent_no_preempt_mid_batch(struct i915_request *rq,
get_children_go_addr(ce),
0);
+ return cs;
+}
+
+/*
+ * If this true, a submission of multi-lrc requests had an error and the
+ * requests need to be skipped. The front end (execuf IOCTL) should've called
+ * i915_request_skip which squashes the BB but we still need to emit the fini
+ * breadrcrumbs seqno write. At this point we don't know how many of the
+ * requests in the multi-lrc submission were generated so we can't do the
+ * handshake between the parent and children (e.g. if 4 requests should be
+ * generated but 2nd hit an error only 1 would be seen by the GuC backend).
+ * Simply skip the handshake, but still emit the breadcrumbd seqno, if an error
+ * has occurred on any of the requests in submission / relationship.
+ */
+static inline bool skip_handshake(struct i915_request *rq)
+{
+ return test_bit(I915_FENCE_FLAG_SKIP_PARALLEL, &rq->fence.flags);
+}
+
+static u32 *
+emit_fini_breadcrumb_parent_no_preempt_mid_batch(struct i915_request *rq,
+ u32 *cs)
+{
+ struct intel_context *ce = rq->context;
+
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ if (unlikely(skip_handshake(rq))) {
+ /*
+ * NOP everything in __emit_fini_breadcrumb_parent_no_preempt_mid_batch,
+ * the -6 comes from the length of the emits below.
+ */
+ memset(cs, 0, sizeof(u32) *
+ (ce->engine->emit_fini_breadcrumb_dw - 6));
+ cs += ce->engine->emit_fini_breadcrumb_dw - 6;
+ } else {
+ cs = __emit_fini_breadcrumb_parent_no_preempt_mid_batch(rq, cs);
+ }
+
/* Emit fini breadcrumb */
cs = gen8_emit_ggtt_write(cs,
rq->fence.seqno,
@@ -4115,7 +4154,8 @@ emit_fini_breadcrumb_parent_no_preempt_mid_batch(struct i915_request *rq,
}
static u32 *
-emit_fini_breadcrumb_child_no_preempt_mid_batch(struct i915_request *rq, u32 *cs)
+__emit_fini_breadcrumb_child_no_preempt_mid_batch(struct i915_request *rq,
+ u32 *cs)
{
struct intel_context *ce = rq->context;
struct intel_context *parent = intel_context_to_parent(ce);
@@ -4142,6 +4182,29 @@ emit_fini_breadcrumb_child_no_preempt_mid_batch(struct i915_request *rq, u32 *cs
*cs++ = get_children_go_addr(parent);
*cs++ = 0;
+ return cs;
+}
+
+static u32 *
+emit_fini_breadcrumb_child_no_preempt_mid_batch(struct i915_request *rq,
+ u32 *cs)
+{
+ struct intel_context *ce = rq->context;
+
+ GEM_BUG_ON(!intel_context_is_child(ce));
+
+ if (unlikely(skip_handshake(rq))) {
+ /*
+ * NOP everything in __emit_fini_breadcrumb_child_no_preempt_mid_batch,
+ * the -6 comes from the length of the emits below.
+ */
+ memset(cs, 0, sizeof(u32) *
+ (ce->engine->emit_fini_breadcrumb_dw - 6));
+ cs += ce->engine->emit_fini_breadcrumb_dw - 6;
+ } else {
+ cs = __emit_fini_breadcrumb_child_no_preempt_mid_batch(rq, cs);
+ }
+
/* Emit fini breadcrumb */
cs = gen8_emit_ggtt_write(cs,
rq->fence.seqno,
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 21/25] drm/i915/guc: Handle errors in multi-lrc requests
@ 2021-10-14 17:20 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:20 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
If an error occurs in the front end when multi-lrc requests are getting
generated we need to skip these in the backend but we still need to
emit the breadcrumbs seqno. An issues arises because with multi-lrc
breadcrumbs there is a handshake between the parent and children to make
forward progress. If all the requests are not present this handshake
doesn't work. To work around this, if multi-lrc request has an error we
skip the handshake but still emit the breadcrumbs seqno.
v2:
(John Harrison)
- Add comment explaining the skipping of the handshake logic
- Fix typos in the commit message
v3:
(John Harrison)
- Fix up some comments about the math to NOP the ring
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 69 ++++++++++++++++++-
1 file changed, 66 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index 361fab2cae99..d7710debcd47 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -4070,8 +4070,8 @@ static int emit_bb_start_child_no_preempt_mid_batch(struct i915_request *rq,
}
static u32 *
-emit_fini_breadcrumb_parent_no_preempt_mid_batch(struct i915_request *rq,
- u32 *cs)
+__emit_fini_breadcrumb_parent_no_preempt_mid_batch(struct i915_request *rq,
+ u32 *cs)
{
struct intel_context *ce = rq->context;
u8 i;
@@ -4099,6 +4099,45 @@ emit_fini_breadcrumb_parent_no_preempt_mid_batch(struct i915_request *rq,
get_children_go_addr(ce),
0);
+ return cs;
+}
+
+/*
+ * If this true, a submission of multi-lrc requests had an error and the
+ * requests need to be skipped. The front end (execuf IOCTL) should've called
+ * i915_request_skip which squashes the BB but we still need to emit the fini
+ * breadrcrumbs seqno write. At this point we don't know how many of the
+ * requests in the multi-lrc submission were generated so we can't do the
+ * handshake between the parent and children (e.g. if 4 requests should be
+ * generated but 2nd hit an error only 1 would be seen by the GuC backend).
+ * Simply skip the handshake, but still emit the breadcrumbd seqno, if an error
+ * has occurred on any of the requests in submission / relationship.
+ */
+static inline bool skip_handshake(struct i915_request *rq)
+{
+ return test_bit(I915_FENCE_FLAG_SKIP_PARALLEL, &rq->fence.flags);
+}
+
+static u32 *
+emit_fini_breadcrumb_parent_no_preempt_mid_batch(struct i915_request *rq,
+ u32 *cs)
+{
+ struct intel_context *ce = rq->context;
+
+ GEM_BUG_ON(!intel_context_is_parent(ce));
+
+ if (unlikely(skip_handshake(rq))) {
+ /*
+ * NOP everything in __emit_fini_breadcrumb_parent_no_preempt_mid_batch,
+ * the -6 comes from the length of the emits below.
+ */
+ memset(cs, 0, sizeof(u32) *
+ (ce->engine->emit_fini_breadcrumb_dw - 6));
+ cs += ce->engine->emit_fini_breadcrumb_dw - 6;
+ } else {
+ cs = __emit_fini_breadcrumb_parent_no_preempt_mid_batch(rq, cs);
+ }
+
/* Emit fini breadcrumb */
cs = gen8_emit_ggtt_write(cs,
rq->fence.seqno,
@@ -4115,7 +4154,8 @@ emit_fini_breadcrumb_parent_no_preempt_mid_batch(struct i915_request *rq,
}
static u32 *
-emit_fini_breadcrumb_child_no_preempt_mid_batch(struct i915_request *rq, u32 *cs)
+__emit_fini_breadcrumb_child_no_preempt_mid_batch(struct i915_request *rq,
+ u32 *cs)
{
struct intel_context *ce = rq->context;
struct intel_context *parent = intel_context_to_parent(ce);
@@ -4142,6 +4182,29 @@ emit_fini_breadcrumb_child_no_preempt_mid_batch(struct i915_request *rq, u32 *cs
*cs++ = get_children_go_addr(parent);
*cs++ = 0;
+ return cs;
+}
+
+static u32 *
+emit_fini_breadcrumb_child_no_preempt_mid_batch(struct i915_request *rq,
+ u32 *cs)
+{
+ struct intel_context *ce = rq->context;
+
+ GEM_BUG_ON(!intel_context_is_child(ce));
+
+ if (unlikely(skip_handshake(rq))) {
+ /*
+ * NOP everything in __emit_fini_breadcrumb_child_no_preempt_mid_batch,
+ * the -6 comes from the length of the emits below.
+ */
+ memset(cs, 0, sizeof(u32) *
+ (ce->engine->emit_fini_breadcrumb_dw - 6));
+ cs += ce->engine->emit_fini_breadcrumb_dw - 6;
+ } else {
+ cs = __emit_fini_breadcrumb_child_no_preempt_mid_batch(rq, cs);
+ }
+
/* Emit fini breadcrumb */
cs = gen8_emit_ggtt_write(cs,
rq->fence.seqno,
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 22/25] drm/i915: Make request conflict tracking understand parallel submits
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:20 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:20 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
If an object in the excl or shared slot is a composite fence from a
parallel submit and the current request in the conflict tracking is from
the same parallel context there is no need to enforce ordering as the
ordering is already implicit. Make the request conflict tracking
understand this by comparing a parallel submit's parent context and
skipping conflict insertion if the values match.
v2:
(John Harrison)
- Reword commit message
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_request.c | 43 +++++++++++++++++++----------
1 file changed, 29 insertions(+), 14 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
index 8bdf9f2f9b90..820a1f38b271 100644
--- a/drivers/gpu/drm/i915/i915_request.c
+++ b/drivers/gpu/drm/i915/i915_request.c
@@ -1335,6 +1335,25 @@ i915_request_await_external(struct i915_request *rq, struct dma_fence *fence)
return err;
}
+static inline bool is_parallel_rq(struct i915_request *rq)
+{
+ return intel_context_is_parallel(rq->context);
+}
+
+static inline struct intel_context *request_to_parent(struct i915_request *rq)
+{
+ return intel_context_to_parent(rq->context);
+}
+
+static bool is_same_parallel_context(struct i915_request *to,
+ struct i915_request *from)
+{
+ if (is_parallel_rq(to))
+ return request_to_parent(to) == request_to_parent(from);
+
+ return false;
+}
+
int
i915_request_await_execution(struct i915_request *rq,
struct dma_fence *fence)
@@ -1366,11 +1385,14 @@ i915_request_await_execution(struct i915_request *rq,
* want to run our callback in all cases.
*/
- if (dma_fence_is_i915(fence))
+ if (dma_fence_is_i915(fence)) {
+ if (is_same_parallel_context(rq, to_request(fence)))
+ continue;
ret = __i915_request_await_execution(rq,
to_request(fence));
- else
+ } else {
ret = i915_request_await_external(rq, fence);
+ }
if (ret < 0)
return ret;
} while (--nchild);
@@ -1471,10 +1493,13 @@ i915_request_await_dma_fence(struct i915_request *rq, struct dma_fence *fence)
fence))
continue;
- if (dma_fence_is_i915(fence))
+ if (dma_fence_is_i915(fence)) {
+ if (is_same_parallel_context(rq, to_request(fence)))
+ continue;
ret = i915_request_await_request(rq, to_request(fence));
- else
+ } else {
ret = i915_request_await_external(rq, fence);
+ }
if (ret < 0)
return ret;
@@ -1525,16 +1550,6 @@ i915_request_await_object(struct i915_request *to,
return ret;
}
-static inline bool is_parallel_rq(struct i915_request *rq)
-{
- return intel_context_is_parallel(rq->context);
-}
-
-static inline struct intel_context *request_to_parent(struct i915_request *rq)
-{
- return intel_context_to_parent(rq->context);
-}
-
static struct i915_request *
__i915_request_ensure_parallel_ordering(struct i915_request *rq,
struct intel_timeline *timeline)
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 22/25] drm/i915: Make request conflict tracking understand parallel submits
@ 2021-10-14 17:20 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:20 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
If an object in the excl or shared slot is a composite fence from a
parallel submit and the current request in the conflict tracking is from
the same parallel context there is no need to enforce ordering as the
ordering is already implicit. Make the request conflict tracking
understand this by comparing a parallel submit's parent context and
skipping conflict insertion if the values match.
v2:
(John Harrison)
- Reword commit message
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
---
drivers/gpu/drm/i915/i915_request.c | 43 +++++++++++++++++++----------
1 file changed, 29 insertions(+), 14 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_request.c b/drivers/gpu/drm/i915/i915_request.c
index 8bdf9f2f9b90..820a1f38b271 100644
--- a/drivers/gpu/drm/i915/i915_request.c
+++ b/drivers/gpu/drm/i915/i915_request.c
@@ -1335,6 +1335,25 @@ i915_request_await_external(struct i915_request *rq, struct dma_fence *fence)
return err;
}
+static inline bool is_parallel_rq(struct i915_request *rq)
+{
+ return intel_context_is_parallel(rq->context);
+}
+
+static inline struct intel_context *request_to_parent(struct i915_request *rq)
+{
+ return intel_context_to_parent(rq->context);
+}
+
+static bool is_same_parallel_context(struct i915_request *to,
+ struct i915_request *from)
+{
+ if (is_parallel_rq(to))
+ return request_to_parent(to) == request_to_parent(from);
+
+ return false;
+}
+
int
i915_request_await_execution(struct i915_request *rq,
struct dma_fence *fence)
@@ -1366,11 +1385,14 @@ i915_request_await_execution(struct i915_request *rq,
* want to run our callback in all cases.
*/
- if (dma_fence_is_i915(fence))
+ if (dma_fence_is_i915(fence)) {
+ if (is_same_parallel_context(rq, to_request(fence)))
+ continue;
ret = __i915_request_await_execution(rq,
to_request(fence));
- else
+ } else {
ret = i915_request_await_external(rq, fence);
+ }
if (ret < 0)
return ret;
} while (--nchild);
@@ -1471,10 +1493,13 @@ i915_request_await_dma_fence(struct i915_request *rq, struct dma_fence *fence)
fence))
continue;
- if (dma_fence_is_i915(fence))
+ if (dma_fence_is_i915(fence)) {
+ if (is_same_parallel_context(rq, to_request(fence)))
+ continue;
ret = i915_request_await_request(rq, to_request(fence));
- else
+ } else {
ret = i915_request_await_external(rq, fence);
+ }
if (ret < 0)
return ret;
@@ -1525,16 +1550,6 @@ i915_request_await_object(struct i915_request *to,
return ret;
}
-static inline bool is_parallel_rq(struct i915_request *rq)
-{
- return intel_context_is_parallel(rq->context);
-}
-
-static inline struct intel_context *request_to_parent(struct i915_request *rq)
-{
- return intel_context_to_parent(rq->context);
-}
-
static struct i915_request *
__i915_request_ensure_parallel_ordering(struct i915_request *rq,
struct intel_timeline *timeline)
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 23/25] drm/i915: Update I915_GEM_BUSY IOCTL to understand composite fences
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:20 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:20 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Parallel submission create composite fences (dma_fence_array) for excl /
shared slots in objects. The I915_GEM_BUSY IOCTL checks these slots to
determine the busyness of the object. Prior to patch it only check if
the fence in the slot was a i915_request. Update the check to understand
composite fences and correctly report the busyness.
v2:
(Tvrtko)
- Remove duplicate BUILD_BUG_ON
Reviewed-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gem/i915_gem_busy.c | 57 +++++++++++++++----
.../gpu/drm/i915/gem/i915_gem_execbuffer.c | 5 +-
drivers/gpu/drm/i915/i915_request.h | 6 ++
3 files changed, 55 insertions(+), 13 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_busy.c b/drivers/gpu/drm/i915/gem/i915_gem_busy.c
index 6234e17259c1..7358bebef15c 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_busy.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_busy.c
@@ -4,6 +4,8 @@
* Copyright © 2014-2016 Intel Corporation
*/
+#include <linux/dma-fence-array.h>
+
#include "gt/intel_engine.h"
#include "i915_gem_ioctls.h"
@@ -36,7 +38,7 @@ static __always_inline u32 __busy_write_id(u16 id)
}
static __always_inline unsigned int
-__busy_set_if_active(const struct dma_fence *fence, u32 (*flag)(u16 id))
+__busy_set_if_active(struct dma_fence *fence, u32 (*flag)(u16 id))
{
const struct i915_request *rq;
@@ -46,29 +48,60 @@ __busy_set_if_active(const struct dma_fence *fence, u32 (*flag)(u16 id))
* to eventually flush us, but to minimise latency just ask the
* hardware.
*
- * Note we only report on the status of native fences.
+ * Note we only report on the status of native fences and we currently
+ * have two native fences:
+ *
+ * 1. A composite fence (dma_fence_array) constructed of i915 requests
+ * created during a parallel submission. In this case we deconstruct the
+ * composite fence into individual i915 requests and check the status of
+ * each request.
+ *
+ * 2. A single i915 request.
*/
- if (!dma_fence_is_i915(fence))
+ if (dma_fence_is_array(fence)) {
+ struct dma_fence_array *array = to_dma_fence_array(fence);
+ struct dma_fence **child = array->fences;
+ unsigned int nchild = array->num_fences;
+
+ do {
+ struct dma_fence *current_fence = *child++;
+
+ /* Not an i915 fence, can't be busy per above */
+ if (!dma_fence_is_i915(current_fence) ||
+ !test_bit(I915_FENCE_FLAG_COMPOSITE,
+ ¤t_fence->flags)) {
+ return 0;
+ }
+
+ rq = to_request(current_fence);
+ if (!i915_request_completed(rq))
+ return flag(rq->engine->uabi_class);
+ } while (--nchild);
+
+ /* All requests in array complete, not busy */
return 0;
+ } else {
+ if (!dma_fence_is_i915(fence))
+ return 0;
- /* opencode to_request() in order to avoid const warnings */
- rq = container_of(fence, const struct i915_request, fence);
- if (i915_request_completed(rq))
- return 0;
+ rq = to_request(fence);
+ if (i915_request_completed(rq))
+ return 0;
- /* Beware type-expansion follies! */
- BUILD_BUG_ON(!typecheck(u16, rq->engine->uabi_class));
- return flag(rq->engine->uabi_class);
+ /* Beware type-expansion follies! */
+ BUILD_BUG_ON(!typecheck(u16, rq->engine->uabi_class));
+ return flag(rq->engine->uabi_class);
+ }
}
static __always_inline unsigned int
-busy_check_reader(const struct dma_fence *fence)
+busy_check_reader(struct dma_fence *fence)
{
return __busy_set_if_active(fence, __busy_read_flag);
}
static __always_inline unsigned int
-busy_check_writer(const struct dma_fence *fence)
+busy_check_writer(struct dma_fence *fence)
{
if (!fence)
return 0;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index fc30856e81fa..1231224728e4 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -3008,8 +3008,11 @@ eb_composite_fence_create(struct i915_execbuffer *eb, int out_fence_fd)
if (!fences)
return ERR_PTR(-ENOMEM);
- for_each_batch_create_order(eb, i)
+ for_each_batch_create_order(eb, i) {
fences[i] = &eb->requests[i]->fence;
+ __set_bit(I915_FENCE_FLAG_COMPOSITE,
+ &eb->requests[i]->fence.flags);
+ }
fence_array = dma_fence_array_create(eb->num_batches,
fences,
diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
index 24db8459376b..dc359242d1ae 100644
--- a/drivers/gpu/drm/i915/i915_request.h
+++ b/drivers/gpu/drm/i915/i915_request.h
@@ -156,6 +156,12 @@ enum {
* submission / relationship encoutered an error.
*/
I915_FENCE_FLAG_SKIP_PARALLEL,
+
+ /*
+ * I915_FENCE_FLAG_COMPOSITE - Indicates fence is part of a composite
+ * fence (dma_fence_array) and i915 generated for parallel submission.
+ */
+ I915_FENCE_FLAG_COMPOSITE,
};
/**
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 23/25] drm/i915: Update I915_GEM_BUSY IOCTL to understand composite fences
@ 2021-10-14 17:20 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:20 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Parallel submission create composite fences (dma_fence_array) for excl /
shared slots in objects. The I915_GEM_BUSY IOCTL checks these slots to
determine the busyness of the object. Prior to patch it only check if
the fence in the slot was a i915_request. Update the check to understand
composite fences and correctly report the busyness.
v2:
(Tvrtko)
- Remove duplicate BUILD_BUG_ON
Reviewed-by: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gem/i915_gem_busy.c | 57 +++++++++++++++----
.../gpu/drm/i915/gem/i915_gem_execbuffer.c | 5 +-
drivers/gpu/drm/i915/i915_request.h | 6 ++
3 files changed, 55 insertions(+), 13 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_busy.c b/drivers/gpu/drm/i915/gem/i915_gem_busy.c
index 6234e17259c1..7358bebef15c 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_busy.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_busy.c
@@ -4,6 +4,8 @@
* Copyright © 2014-2016 Intel Corporation
*/
+#include <linux/dma-fence-array.h>
+
#include "gt/intel_engine.h"
#include "i915_gem_ioctls.h"
@@ -36,7 +38,7 @@ static __always_inline u32 __busy_write_id(u16 id)
}
static __always_inline unsigned int
-__busy_set_if_active(const struct dma_fence *fence, u32 (*flag)(u16 id))
+__busy_set_if_active(struct dma_fence *fence, u32 (*flag)(u16 id))
{
const struct i915_request *rq;
@@ -46,29 +48,60 @@ __busy_set_if_active(const struct dma_fence *fence, u32 (*flag)(u16 id))
* to eventually flush us, but to minimise latency just ask the
* hardware.
*
- * Note we only report on the status of native fences.
+ * Note we only report on the status of native fences and we currently
+ * have two native fences:
+ *
+ * 1. A composite fence (dma_fence_array) constructed of i915 requests
+ * created during a parallel submission. In this case we deconstruct the
+ * composite fence into individual i915 requests and check the status of
+ * each request.
+ *
+ * 2. A single i915 request.
*/
- if (!dma_fence_is_i915(fence))
+ if (dma_fence_is_array(fence)) {
+ struct dma_fence_array *array = to_dma_fence_array(fence);
+ struct dma_fence **child = array->fences;
+ unsigned int nchild = array->num_fences;
+
+ do {
+ struct dma_fence *current_fence = *child++;
+
+ /* Not an i915 fence, can't be busy per above */
+ if (!dma_fence_is_i915(current_fence) ||
+ !test_bit(I915_FENCE_FLAG_COMPOSITE,
+ ¤t_fence->flags)) {
+ return 0;
+ }
+
+ rq = to_request(current_fence);
+ if (!i915_request_completed(rq))
+ return flag(rq->engine->uabi_class);
+ } while (--nchild);
+
+ /* All requests in array complete, not busy */
return 0;
+ } else {
+ if (!dma_fence_is_i915(fence))
+ return 0;
- /* opencode to_request() in order to avoid const warnings */
- rq = container_of(fence, const struct i915_request, fence);
- if (i915_request_completed(rq))
- return 0;
+ rq = to_request(fence);
+ if (i915_request_completed(rq))
+ return 0;
- /* Beware type-expansion follies! */
- BUILD_BUG_ON(!typecheck(u16, rq->engine->uabi_class));
- return flag(rq->engine->uabi_class);
+ /* Beware type-expansion follies! */
+ BUILD_BUG_ON(!typecheck(u16, rq->engine->uabi_class));
+ return flag(rq->engine->uabi_class);
+ }
}
static __always_inline unsigned int
-busy_check_reader(const struct dma_fence *fence)
+busy_check_reader(struct dma_fence *fence)
{
return __busy_set_if_active(fence, __busy_read_flag);
}
static __always_inline unsigned int
-busy_check_writer(const struct dma_fence *fence)
+busy_check_writer(struct dma_fence *fence)
{
if (!fence)
return 0;
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
index fc30856e81fa..1231224728e4 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
@@ -3008,8 +3008,11 @@ eb_composite_fence_create(struct i915_execbuffer *eb, int out_fence_fd)
if (!fences)
return ERR_PTR(-ENOMEM);
- for_each_batch_create_order(eb, i)
+ for_each_batch_create_order(eb, i) {
fences[i] = &eb->requests[i]->fence;
+ __set_bit(I915_FENCE_FLAG_COMPOSITE,
+ &eb->requests[i]->fence.flags);
+ }
fence_array = dma_fence_array_create(eb->num_batches,
fences,
diff --git a/drivers/gpu/drm/i915/i915_request.h b/drivers/gpu/drm/i915/i915_request.h
index 24db8459376b..dc359242d1ae 100644
--- a/drivers/gpu/drm/i915/i915_request.h
+++ b/drivers/gpu/drm/i915/i915_request.h
@@ -156,6 +156,12 @@ enum {
* submission / relationship encoutered an error.
*/
I915_FENCE_FLAG_SKIP_PARALLEL,
+
+ /*
+ * I915_FENCE_FLAG_COMPOSITE - Indicates fence is part of a composite
+ * fence (dma_fence_array) and i915 generated for parallel submission.
+ */
+ I915_FENCE_FLAG_COMPOSITE,
};
/**
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 24/25] drm/i915: Enable multi-bb execbuf
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:20 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:20 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Enable multi-bb execbuf by enabling the set_parallel extension.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gem/i915_gem_context.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
index 9a00f11fef46..fb33d0322960 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
@@ -570,9 +570,6 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
struct intel_engine_cs **siblings = NULL;
intel_engine_mask_t prev_mask;
- /* Disabling for now */
- return -ENODEV;
-
/* FIXME: This is NIY for execlists */
if (!(intel_uc_uses_guc_submission(&i915->gt.uc)))
return -ENODEV;
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* [PATCH 24/25] drm/i915: Enable multi-bb execbuf
@ 2021-10-14 17:20 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:20 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
Enable multi-bb execbuf by enabling the set_parallel extension.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gem/i915_gem_context.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
index 9a00f11fef46..fb33d0322960 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
@@ -570,9 +570,6 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
struct intel_engine_cs **siblings = NULL;
intel_engine_mask_t prev_mask;
- /* Disabling for now */
- return -ENODEV;
-
/* FIXME: This is NIY for execlists */
if (!(intel_uc_uses_guc_submission(&i915->gt.uc)))
return -ENODEV;
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread
* Re: [Intel-gfx] [PATCH 24/25] drm/i915: Enable multi-bb execbuf
2021-10-14 17:20 ` Matthew Brost
@ 2021-10-14 18:29 ` John Harrison
-1 siblings, 0 replies; 69+ messages in thread
From: John Harrison @ 2021-10-14 18:29 UTC (permalink / raw)
To: Matthew Brost, intel-gfx, dri-devel
On 10/14/2021 10:20, Matthew Brost wrote:
> Enable multi-bb execbuf by enabling the set_parallel extension.
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/gem/i915_gem_context.c | 3 ---
> 1 file changed, 3 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> index 9a00f11fef46..fb33d0322960 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> @@ -570,9 +570,6 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
> struct intel_engine_cs **siblings = NULL;
> intel_engine_mask_t prev_mask;
>
> - /* Disabling for now */
> - return -ENODEV;
> -
> /* FIXME: This is NIY for execlists */
> if (!(intel_uc_uses_guc_submission(&i915->gt.uc)))
> return -ENODEV;
^ permalink raw reply [flat|nested] 69+ messages in thread
* Re: [PATCH 24/25] drm/i915: Enable multi-bb execbuf
@ 2021-10-14 18:29 ` John Harrison
0 siblings, 0 replies; 69+ messages in thread
From: John Harrison @ 2021-10-14 18:29 UTC (permalink / raw)
To: Matthew Brost, intel-gfx, dri-devel
On 10/14/2021 10:20, Matthew Brost wrote:
> Enable multi-bb execbuf by enabling the set_parallel extension.
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
Reviewed-by: John Harrison <John.C.Harrison@Intel.com>
> ---
> drivers/gpu/drm/i915/gem/i915_gem_context.c | 3 ---
> 1 file changed, 3 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> index 9a00f11fef46..fb33d0322960 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> @@ -570,9 +570,6 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
> struct intel_engine_cs **siblings = NULL;
> intel_engine_mask_t prev_mask;
>
> - /* Disabling for now */
> - return -ENODEV;
> -
> /* FIXME: This is NIY for execlists */
> if (!(intel_uc_uses_guc_submission(&i915->gt.uc)))
> return -ENODEV;
^ permalink raw reply [flat|nested] 69+ messages in thread
* [Intel-gfx] [PATCH 25/25] drm/i915/execlists: Weak parallel submission support for execlists
2021-10-14 17:19 ` Matthew Brost
@ 2021-10-14 17:20 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:20 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
A weak implementation of parallel submission (multi-bb execbuf IOCTL) for
execlists. Doing as little as possible to support this interface for
execlists - basically just passing submit fences between each request
generated and virtual engines are not allowed. This is on par with what
is there for the existing (hopefully soon deprecated) bonding interface.
We perma-pin these execlists contexts to align with GuC implementation.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gem/i915_gem_context.c | 10 ++--
drivers/gpu/drm/i915/gt/intel_context.c | 4 +-
.../drm/i915/gt/intel_execlists_submission.c | 56 ++++++++++++++++++-
drivers/gpu/drm/i915/gt/intel_lrc.c | 2 +
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 -
5 files changed, 64 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
index fb33d0322960..35e87a7d0ea9 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
@@ -570,10 +570,6 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
struct intel_engine_cs **siblings = NULL;
intel_engine_mask_t prev_mask;
- /* FIXME: This is NIY for execlists */
- if (!(intel_uc_uses_guc_submission(&i915->gt.uc)))
- return -ENODEV;
-
if (get_user(slot, &ext->engine_index))
return -EFAULT;
@@ -583,6 +579,12 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
if (get_user(num_siblings, &ext->num_siblings))
return -EFAULT;
+ if (!intel_uc_uses_guc_submission(&i915->gt.uc) && num_siblings != 1) {
+ drm_dbg(&i915->drm, "Only 1 sibling (%d) supported in non-GuC mode\n",
+ num_siblings);
+ return -EINVAL;
+ }
+
if (slot >= set->num_engines) {
drm_dbg(&i915->drm, "Invalid placement value, %d >= %d\n",
slot, set->num_engines);
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index 5634d14052bc..1bec92e1d8e6 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -79,7 +79,8 @@ static int intel_context_active_acquire(struct intel_context *ce)
__i915_active_acquire(&ce->active);
- if (intel_context_is_barrier(ce) || intel_engine_uses_guc(ce->engine))
+ if (intel_context_is_barrier(ce) || intel_engine_uses_guc(ce->engine) ||
+ intel_context_is_parallel(ce))
return 0;
/* Preallocate tracking nodes */
@@ -563,7 +564,6 @@ void intel_context_bind_parent_child(struct intel_context *parent,
* Callers responsibility to validate that this function is used
* correctly but we use GEM_BUG_ON here ensure that they do.
*/
- GEM_BUG_ON(!intel_engine_uses_guc(parent->engine));
GEM_BUG_ON(intel_context_is_pinned(parent));
GEM_BUG_ON(intel_context_is_child(parent));
GEM_BUG_ON(intel_context_is_pinned(child));
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index bedb80057046..8cd986bdf26c 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -927,8 +927,7 @@ static void execlists_submit_ports(struct intel_engine_cs *engine)
static bool ctx_single_port_submission(const struct intel_context *ce)
{
- return (IS_ENABLED(CONFIG_DRM_I915_GVT) &&
- intel_context_force_single_submission(ce));
+ return intel_context_force_single_submission(ce);
}
static bool can_merge_ctx(const struct intel_context *prev,
@@ -2598,6 +2597,58 @@ static void execlists_context_cancel_request(struct intel_context *ce,
current->comm);
}
+static struct intel_context *
+execlists_create_parallel(struct intel_engine_cs **engines,
+ unsigned int num_siblings,
+ unsigned int width)
+{
+ struct intel_engine_cs **siblings = NULL;
+ struct intel_context *parent = NULL, *ce, *err;
+ int i, j;
+
+ GEM_BUG_ON(num_siblings != 1);
+
+ siblings = kmalloc_array(num_siblings,
+ sizeof(*siblings),
+ GFP_KERNEL);
+ if (!siblings)
+ return ERR_PTR(-ENOMEM);
+
+ for (i = 0; i < width; ++i) {
+ for (j = 0; j < num_siblings; ++j)
+ siblings[j] = engines[i * num_siblings + j];
+
+ ce = intel_context_create(siblings[0]);
+ if (!ce) {
+ err = ERR_PTR(-ENOMEM);
+ goto unwind;
+ }
+
+ if (i == 0)
+ parent = ce;
+ else
+ intel_context_bind_parent_child(parent, ce);
+ }
+
+ parent->parallel.fence_context = dma_fence_context_alloc(1);
+
+ intel_context_set_nopreempt(parent);
+ intel_context_set_single_submission(parent);
+ for_each_child(parent, ce) {
+ intel_context_set_nopreempt(ce);
+ intel_context_set_single_submission(ce);
+ }
+
+ kfree(siblings);
+ return parent;
+
+unwind:
+ if (parent)
+ intel_context_put(parent);
+ kfree(siblings);
+ return err;
+}
+
static const struct intel_context_ops execlists_context_ops = {
.flags = COPS_HAS_INFLIGHT,
@@ -2616,6 +2667,7 @@ static const struct intel_context_ops execlists_context_ops = {
.reset = lrc_reset,
.destroy = lrc_destroy,
+ .create_parallel = execlists_create_parallel,
.create_virtual = execlists_create_virtual,
};
diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
index 56156cf18c41..70f4b309522d 100644
--- a/drivers/gpu/drm/i915/gt/intel_lrc.c
+++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
@@ -1065,6 +1065,8 @@ lrc_pin(struct intel_context *ce,
void lrc_unpin(struct intel_context *ce)
{
+ if (unlikely(ce->parallel.last_rq))
+ i915_request_put(ce->parallel.last_rq);
check_redzone((void *)ce->lrc_reg_state - LRC_STATE_OFFSET,
ce->engine);
}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index d7710debcd47..145ffe265305 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -2960,8 +2960,6 @@ static void guc_parent_context_unpin(struct intel_context *ce)
GEM_BUG_ON(!intel_context_is_parent(ce));
GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
- if (ce->parallel.last_rq)
- i915_request_put(ce->parallel.last_rq);
unpin_guc_id(guc, ce);
lrc_unpin(ce);
}
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* [PATCH 25/25] drm/i915/execlists: Weak parallel submission support for execlists
@ 2021-10-14 17:20 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 17:20 UTC (permalink / raw)
To: intel-gfx, dri-devel; +Cc: john.c.harrison
A weak implementation of parallel submission (multi-bb execbuf IOCTL) for
execlists. Doing as little as possible to support this interface for
execlists - basically just passing submit fences between each request
generated and virtual engines are not allowed. This is on par with what
is there for the existing (hopefully soon deprecated) bonding interface.
We perma-pin these execlists contexts to align with GuC implementation.
Signed-off-by: Matthew Brost <matthew.brost@intel.com>
---
drivers/gpu/drm/i915/gem/i915_gem_context.c | 10 ++--
drivers/gpu/drm/i915/gt/intel_context.c | 4 +-
.../drm/i915/gt/intel_execlists_submission.c | 56 ++++++++++++++++++-
drivers/gpu/drm/i915/gt/intel_lrc.c | 2 +
.../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 -
5 files changed, 64 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
index fb33d0322960..35e87a7d0ea9 100644
--- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
@@ -570,10 +570,6 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
struct intel_engine_cs **siblings = NULL;
intel_engine_mask_t prev_mask;
- /* FIXME: This is NIY for execlists */
- if (!(intel_uc_uses_guc_submission(&i915->gt.uc)))
- return -ENODEV;
-
if (get_user(slot, &ext->engine_index))
return -EFAULT;
@@ -583,6 +579,12 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
if (get_user(num_siblings, &ext->num_siblings))
return -EFAULT;
+ if (!intel_uc_uses_guc_submission(&i915->gt.uc) && num_siblings != 1) {
+ drm_dbg(&i915->drm, "Only 1 sibling (%d) supported in non-GuC mode\n",
+ num_siblings);
+ return -EINVAL;
+ }
+
if (slot >= set->num_engines) {
drm_dbg(&i915->drm, "Invalid placement value, %d >= %d\n",
slot, set->num_engines);
diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
index 5634d14052bc..1bec92e1d8e6 100644
--- a/drivers/gpu/drm/i915/gt/intel_context.c
+++ b/drivers/gpu/drm/i915/gt/intel_context.c
@@ -79,7 +79,8 @@ static int intel_context_active_acquire(struct intel_context *ce)
__i915_active_acquire(&ce->active);
- if (intel_context_is_barrier(ce) || intel_engine_uses_guc(ce->engine))
+ if (intel_context_is_barrier(ce) || intel_engine_uses_guc(ce->engine) ||
+ intel_context_is_parallel(ce))
return 0;
/* Preallocate tracking nodes */
@@ -563,7 +564,6 @@ void intel_context_bind_parent_child(struct intel_context *parent,
* Callers responsibility to validate that this function is used
* correctly but we use GEM_BUG_ON here ensure that they do.
*/
- GEM_BUG_ON(!intel_engine_uses_guc(parent->engine));
GEM_BUG_ON(intel_context_is_pinned(parent));
GEM_BUG_ON(intel_context_is_child(parent));
GEM_BUG_ON(intel_context_is_pinned(child));
diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index bedb80057046..8cd986bdf26c 100644
--- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
+++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
@@ -927,8 +927,7 @@ static void execlists_submit_ports(struct intel_engine_cs *engine)
static bool ctx_single_port_submission(const struct intel_context *ce)
{
- return (IS_ENABLED(CONFIG_DRM_I915_GVT) &&
- intel_context_force_single_submission(ce));
+ return intel_context_force_single_submission(ce);
}
static bool can_merge_ctx(const struct intel_context *prev,
@@ -2598,6 +2597,58 @@ static void execlists_context_cancel_request(struct intel_context *ce,
current->comm);
}
+static struct intel_context *
+execlists_create_parallel(struct intel_engine_cs **engines,
+ unsigned int num_siblings,
+ unsigned int width)
+{
+ struct intel_engine_cs **siblings = NULL;
+ struct intel_context *parent = NULL, *ce, *err;
+ int i, j;
+
+ GEM_BUG_ON(num_siblings != 1);
+
+ siblings = kmalloc_array(num_siblings,
+ sizeof(*siblings),
+ GFP_KERNEL);
+ if (!siblings)
+ return ERR_PTR(-ENOMEM);
+
+ for (i = 0; i < width; ++i) {
+ for (j = 0; j < num_siblings; ++j)
+ siblings[j] = engines[i * num_siblings + j];
+
+ ce = intel_context_create(siblings[0]);
+ if (!ce) {
+ err = ERR_PTR(-ENOMEM);
+ goto unwind;
+ }
+
+ if (i == 0)
+ parent = ce;
+ else
+ intel_context_bind_parent_child(parent, ce);
+ }
+
+ parent->parallel.fence_context = dma_fence_context_alloc(1);
+
+ intel_context_set_nopreempt(parent);
+ intel_context_set_single_submission(parent);
+ for_each_child(parent, ce) {
+ intel_context_set_nopreempt(ce);
+ intel_context_set_single_submission(ce);
+ }
+
+ kfree(siblings);
+ return parent;
+
+unwind:
+ if (parent)
+ intel_context_put(parent);
+ kfree(siblings);
+ return err;
+}
+
static const struct intel_context_ops execlists_context_ops = {
.flags = COPS_HAS_INFLIGHT,
@@ -2616,6 +2667,7 @@ static const struct intel_context_ops execlists_context_ops = {
.reset = lrc_reset,
.destroy = lrc_destroy,
+ .create_parallel = execlists_create_parallel,
.create_virtual = execlists_create_virtual,
};
diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
index 56156cf18c41..70f4b309522d 100644
--- a/drivers/gpu/drm/i915/gt/intel_lrc.c
+++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
@@ -1065,6 +1065,8 @@ lrc_pin(struct intel_context *ce,
void lrc_unpin(struct intel_context *ce)
{
+ if (unlikely(ce->parallel.last_rq))
+ i915_request_put(ce->parallel.last_rq);
check_redzone((void *)ce->lrc_reg_state - LRC_STATE_OFFSET,
ce->engine);
}
diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
index d7710debcd47..145ffe265305 100644
--- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
+++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
@@ -2960,8 +2960,6 @@ static void guc_parent_context_unpin(struct intel_context *ce)
GEM_BUG_ON(!intel_context_is_parent(ce));
GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
- if (ce->parallel.last_rq)
- i915_request_put(ce->parallel.last_rq);
unpin_guc_id(guc, ce);
lrc_unpin(ce);
}
--
2.32.0
^ permalink raw reply related [flat|nested] 69+ messages in thread* Re: [Intel-gfx] [PATCH 25/25] drm/i915/execlists: Weak parallel submission support for execlists
2021-10-14 17:20 ` Matthew Brost
@ 2021-10-14 18:42 ` John Harrison
-1 siblings, 0 replies; 69+ messages in thread
From: John Harrison @ 2021-10-14 18:42 UTC (permalink / raw)
To: Matthew Brost, intel-gfx, dri-devel
On 10/14/2021 10:20, Matthew Brost wrote:
> A weak implementation of parallel submission (multi-bb execbuf IOCTL) for
> execlists. Doing as little as possible to support this interface for
> execlists - basically just passing submit fences between each request
> generated and virtual engines are not allowed. This is on par with what
> is there for the existing (hopefully soon deprecated) bonding interface.
>
> We perma-pin these execlists contexts to align with GuC implementation.
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gem/i915_gem_context.c | 10 ++--
> drivers/gpu/drm/i915/gt/intel_context.c | 4 +-
> .../drm/i915/gt/intel_execlists_submission.c | 56 ++++++++++++++++++-
> drivers/gpu/drm/i915/gt/intel_lrc.c | 2 +
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 -
> 5 files changed, 64 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> index fb33d0322960..35e87a7d0ea9 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> @@ -570,10 +570,6 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
> struct intel_engine_cs **siblings = NULL;
> intel_engine_mask_t prev_mask;
>
> - /* FIXME: This is NIY for execlists */
> - if (!(intel_uc_uses_guc_submission(&i915->gt.uc)))
> - return -ENODEV;
> -
> if (get_user(slot, &ext->engine_index))
> return -EFAULT;
>
> @@ -583,6 +579,12 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
> if (get_user(num_siblings, &ext->num_siblings))
> return -EFAULT;
>
> + if (!intel_uc_uses_guc_submission(&i915->gt.uc) && num_siblings != 1) {
> + drm_dbg(&i915->drm, "Only 1 sibling (%d) supported in non-GuC mode\n",
> + num_siblings);
> + return -EINVAL;
> + }
> +
> if (slot >= set->num_engines) {
> drm_dbg(&i915->drm, "Invalid placement value, %d >= %d\n",
> slot, set->num_engines);
> diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
> index 5634d14052bc..1bec92e1d8e6 100644
> --- a/drivers/gpu/drm/i915/gt/intel_context.c
> +++ b/drivers/gpu/drm/i915/gt/intel_context.c
> @@ -79,7 +79,8 @@ static int intel_context_active_acquire(struct intel_context *ce)
>
> __i915_active_acquire(&ce->active);
>
> - if (intel_context_is_barrier(ce) || intel_engine_uses_guc(ce->engine))
> + if (intel_context_is_barrier(ce) || intel_engine_uses_guc(ce->engine) ||
> + intel_context_is_parallel(ce))
> return 0;
>
> /* Preallocate tracking nodes */
> @@ -563,7 +564,6 @@ void intel_context_bind_parent_child(struct intel_context *parent,
> * Callers responsibility to validate that this function is used
> * correctly but we use GEM_BUG_ON here ensure that they do.
> */
> - GEM_BUG_ON(!intel_engine_uses_guc(parent->engine));
> GEM_BUG_ON(intel_context_is_pinned(parent));
> GEM_BUG_ON(intel_context_is_child(parent));
> GEM_BUG_ON(intel_context_is_pinned(child));
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index bedb80057046..8cd986bdf26c 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -927,8 +927,7 @@ static void execlists_submit_ports(struct intel_engine_cs *engine)
>
> static bool ctx_single_port_submission(const struct intel_context *ce)
> {
> - return (IS_ENABLED(CONFIG_DRM_I915_GVT) &&
> - intel_context_force_single_submission(ce));
> + return intel_context_force_single_submission(ce);
Does this change not affect all execlist operation rather than just
parallel submission?
> }
>
> static bool can_merge_ctx(const struct intel_context *prev,
> @@ -2598,6 +2597,58 @@ static void execlists_context_cancel_request(struct intel_context *ce,
> current->comm);
> }
>
> +static struct intel_context *
> +execlists_create_parallel(struct intel_engine_cs **engines,
> + unsigned int num_siblings,
> + unsigned int width)
> +{
> + struct intel_engine_cs **siblings = NULL;
> + struct intel_context *parent = NULL, *ce, *err;
> + int i, j;
> +
> + GEM_BUG_ON(num_siblings != 1);
> +
> + siblings = kmalloc_array(num_siblings,
> + sizeof(*siblings),
> + GFP_KERNEL);
> + if (!siblings)
> + return ERR_PTR(-ENOMEM);
> +
> + for (i = 0; i < width; ++i) {
> + for (j = 0; j < num_siblings; ++j)
> + siblings[j] = engines[i * num_siblings + j];
What is the purpose of this array? The only usage that I can see is
siblings[0] on the line below. The rest of the entries never seem to be
used. So you could just replace that with 'engines[i * num_siblings]'
and drop the siblings array itself completely?
John.
> +
> + ce = intel_context_create(siblings[0]);
> + if (!ce) {
> + err = ERR_PTR(-ENOMEM);
> + goto unwind;
> + }
> +
> + if (i == 0)
> + parent = ce;
> + else
> + intel_context_bind_parent_child(parent, ce);
> + }
> +
> + parent->parallel.fence_context = dma_fence_context_alloc(1);
> +
> + intel_context_set_nopreempt(parent);
> + intel_context_set_single_submission(parent);
> + for_each_child(parent, ce) {
> + intel_context_set_nopreempt(ce);
> + intel_context_set_single_submission(ce);
> + }
> +
> + kfree(siblings);
> + return parent;
> +
> +unwind:
> + if (parent)
> + intel_context_put(parent);
> + kfree(siblings);
> + return err;
> +}
> +
> static const struct intel_context_ops execlists_context_ops = {
> .flags = COPS_HAS_INFLIGHT,
>
> @@ -2616,6 +2667,7 @@ static const struct intel_context_ops execlists_context_ops = {
> .reset = lrc_reset,
> .destroy = lrc_destroy,
>
> + .create_parallel = execlists_create_parallel,
> .create_virtual = execlists_create_virtual,
> };
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
> index 56156cf18c41..70f4b309522d 100644
> --- a/drivers/gpu/drm/i915/gt/intel_lrc.c
> +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
> @@ -1065,6 +1065,8 @@ lrc_pin(struct intel_context *ce,
>
> void lrc_unpin(struct intel_context *ce)
> {
> + if (unlikely(ce->parallel.last_rq))
> + i915_request_put(ce->parallel.last_rq);
> check_redzone((void *)ce->lrc_reg_state - LRC_STATE_OFFSET,
> ce->engine);
> }
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index d7710debcd47..145ffe265305 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -2960,8 +2960,6 @@ static void guc_parent_context_unpin(struct intel_context *ce)
> GEM_BUG_ON(!intel_context_is_parent(ce));
> GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
>
> - if (ce->parallel.last_rq)
> - i915_request_put(ce->parallel.last_rq);
> unpin_guc_id(guc, ce);
> lrc_unpin(ce);
> }
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 25/25] drm/i915/execlists: Weak parallel submission support for execlists
@ 2021-10-14 18:42 ` John Harrison
0 siblings, 0 replies; 69+ messages in thread
From: John Harrison @ 2021-10-14 18:42 UTC (permalink / raw)
To: Matthew Brost, intel-gfx, dri-devel
On 10/14/2021 10:20, Matthew Brost wrote:
> A weak implementation of parallel submission (multi-bb execbuf IOCTL) for
> execlists. Doing as little as possible to support this interface for
> execlists - basically just passing submit fences between each request
> generated and virtual engines are not allowed. This is on par with what
> is there for the existing (hopefully soon deprecated) bonding interface.
>
> We perma-pin these execlists contexts to align with GuC implementation.
>
> Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> ---
> drivers/gpu/drm/i915/gem/i915_gem_context.c | 10 ++--
> drivers/gpu/drm/i915/gt/intel_context.c | 4 +-
> .../drm/i915/gt/intel_execlists_submission.c | 56 ++++++++++++++++++-
> drivers/gpu/drm/i915/gt/intel_lrc.c | 2 +
> .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 -
> 5 files changed, 64 insertions(+), 10 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> index fb33d0322960..35e87a7d0ea9 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> @@ -570,10 +570,6 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
> struct intel_engine_cs **siblings = NULL;
> intel_engine_mask_t prev_mask;
>
> - /* FIXME: This is NIY for execlists */
> - if (!(intel_uc_uses_guc_submission(&i915->gt.uc)))
> - return -ENODEV;
> -
> if (get_user(slot, &ext->engine_index))
> return -EFAULT;
>
> @@ -583,6 +579,12 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
> if (get_user(num_siblings, &ext->num_siblings))
> return -EFAULT;
>
> + if (!intel_uc_uses_guc_submission(&i915->gt.uc) && num_siblings != 1) {
> + drm_dbg(&i915->drm, "Only 1 sibling (%d) supported in non-GuC mode\n",
> + num_siblings);
> + return -EINVAL;
> + }
> +
> if (slot >= set->num_engines) {
> drm_dbg(&i915->drm, "Invalid placement value, %d >= %d\n",
> slot, set->num_engines);
> diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
> index 5634d14052bc..1bec92e1d8e6 100644
> --- a/drivers/gpu/drm/i915/gt/intel_context.c
> +++ b/drivers/gpu/drm/i915/gt/intel_context.c
> @@ -79,7 +79,8 @@ static int intel_context_active_acquire(struct intel_context *ce)
>
> __i915_active_acquire(&ce->active);
>
> - if (intel_context_is_barrier(ce) || intel_engine_uses_guc(ce->engine))
> + if (intel_context_is_barrier(ce) || intel_engine_uses_guc(ce->engine) ||
> + intel_context_is_parallel(ce))
> return 0;
>
> /* Preallocate tracking nodes */
> @@ -563,7 +564,6 @@ void intel_context_bind_parent_child(struct intel_context *parent,
> * Callers responsibility to validate that this function is used
> * correctly but we use GEM_BUG_ON here ensure that they do.
> */
> - GEM_BUG_ON(!intel_engine_uses_guc(parent->engine));
> GEM_BUG_ON(intel_context_is_pinned(parent));
> GEM_BUG_ON(intel_context_is_child(parent));
> GEM_BUG_ON(intel_context_is_pinned(child));
> diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> index bedb80057046..8cd986bdf26c 100644
> --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> @@ -927,8 +927,7 @@ static void execlists_submit_ports(struct intel_engine_cs *engine)
>
> static bool ctx_single_port_submission(const struct intel_context *ce)
> {
> - return (IS_ENABLED(CONFIG_DRM_I915_GVT) &&
> - intel_context_force_single_submission(ce));
> + return intel_context_force_single_submission(ce);
Does this change not affect all execlist operation rather than just
parallel submission?
> }
>
> static bool can_merge_ctx(const struct intel_context *prev,
> @@ -2598,6 +2597,58 @@ static void execlists_context_cancel_request(struct intel_context *ce,
> current->comm);
> }
>
> +static struct intel_context *
> +execlists_create_parallel(struct intel_engine_cs **engines,
> + unsigned int num_siblings,
> + unsigned int width)
> +{
> + struct intel_engine_cs **siblings = NULL;
> + struct intel_context *parent = NULL, *ce, *err;
> + int i, j;
> +
> + GEM_BUG_ON(num_siblings != 1);
> +
> + siblings = kmalloc_array(num_siblings,
> + sizeof(*siblings),
> + GFP_KERNEL);
> + if (!siblings)
> + return ERR_PTR(-ENOMEM);
> +
> + for (i = 0; i < width; ++i) {
> + for (j = 0; j < num_siblings; ++j)
> + siblings[j] = engines[i * num_siblings + j];
What is the purpose of this array? The only usage that I can see is
siblings[0] on the line below. The rest of the entries never seem to be
used. So you could just replace that with 'engines[i * num_siblings]'
and drop the siblings array itself completely?
John.
> +
> + ce = intel_context_create(siblings[0]);
> + if (!ce) {
> + err = ERR_PTR(-ENOMEM);
> + goto unwind;
> + }
> +
> + if (i == 0)
> + parent = ce;
> + else
> + intel_context_bind_parent_child(parent, ce);
> + }
> +
> + parent->parallel.fence_context = dma_fence_context_alloc(1);
> +
> + intel_context_set_nopreempt(parent);
> + intel_context_set_single_submission(parent);
> + for_each_child(parent, ce) {
> + intel_context_set_nopreempt(ce);
> + intel_context_set_single_submission(ce);
> + }
> +
> + kfree(siblings);
> + return parent;
> +
> +unwind:
> + if (parent)
> + intel_context_put(parent);
> + kfree(siblings);
> + return err;
> +}
> +
> static const struct intel_context_ops execlists_context_ops = {
> .flags = COPS_HAS_INFLIGHT,
>
> @@ -2616,6 +2667,7 @@ static const struct intel_context_ops execlists_context_ops = {
> .reset = lrc_reset,
> .destroy = lrc_destroy,
>
> + .create_parallel = execlists_create_parallel,
> .create_virtual = execlists_create_virtual,
> };
>
> diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
> index 56156cf18c41..70f4b309522d 100644
> --- a/drivers/gpu/drm/i915/gt/intel_lrc.c
> +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
> @@ -1065,6 +1065,8 @@ lrc_pin(struct intel_context *ce,
>
> void lrc_unpin(struct intel_context *ce)
> {
> + if (unlikely(ce->parallel.last_rq))
> + i915_request_put(ce->parallel.last_rq);
> check_redzone((void *)ce->lrc_reg_state - LRC_STATE_OFFSET,
> ce->engine);
> }
> diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> index d7710debcd47..145ffe265305 100644
> --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> @@ -2960,8 +2960,6 @@ static void guc_parent_context_unpin(struct intel_context *ce)
> GEM_BUG_ON(!intel_context_is_parent(ce));
> GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
>
> - if (ce->parallel.last_rq)
> - i915_request_put(ce->parallel.last_rq);
> unpin_guc_id(guc, ce);
> lrc_unpin(ce);
> }
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [Intel-gfx] [PATCH 25/25] drm/i915/execlists: Weak parallel submission support for execlists
2021-10-14 18:42 ` John Harrison
@ 2021-10-14 18:55 ` Matthew Brost
-1 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 18:55 UTC (permalink / raw)
To: John Harrison; +Cc: intel-gfx, dri-devel
On Thu, Oct 14, 2021 at 11:42:41AM -0700, John Harrison wrote:
> On 10/14/2021 10:20, Matthew Brost wrote:
> > A weak implementation of parallel submission (multi-bb execbuf IOCTL) for
> > execlists. Doing as little as possible to support this interface for
> > execlists - basically just passing submit fences between each request
> > generated and virtual engines are not allowed. This is on par with what
> > is there for the existing (hopefully soon deprecated) bonding interface.
> >
> > We perma-pin these execlists contexts to align with GuC implementation.
> >
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > drivers/gpu/drm/i915/gem/i915_gem_context.c | 10 ++--
> > drivers/gpu/drm/i915/gt/intel_context.c | 4 +-
> > .../drm/i915/gt/intel_execlists_submission.c | 56 ++++++++++++++++++-
> > drivers/gpu/drm/i915/gt/intel_lrc.c | 2 +
> > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 -
> > 5 files changed, 64 insertions(+), 10 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> > index fb33d0322960..35e87a7d0ea9 100644
> > --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
> > +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> > @@ -570,10 +570,6 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
> > struct intel_engine_cs **siblings = NULL;
> > intel_engine_mask_t prev_mask;
> > - /* FIXME: This is NIY for execlists */
> > - if (!(intel_uc_uses_guc_submission(&i915->gt.uc)))
> > - return -ENODEV;
> > -
> > if (get_user(slot, &ext->engine_index))
> > return -EFAULT;
> > @@ -583,6 +579,12 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
> > if (get_user(num_siblings, &ext->num_siblings))
> > return -EFAULT;
> > + if (!intel_uc_uses_guc_submission(&i915->gt.uc) && num_siblings != 1) {
> > + drm_dbg(&i915->drm, "Only 1 sibling (%d) supported in non-GuC mode\n",
> > + num_siblings);
> > + return -EINVAL;
> > + }
> > +
> > if (slot >= set->num_engines) {
> > drm_dbg(&i915->drm, "Invalid placement value, %d >= %d\n",
> > slot, set->num_engines);
> > diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
> > index 5634d14052bc..1bec92e1d8e6 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_context.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_context.c
> > @@ -79,7 +79,8 @@ static int intel_context_active_acquire(struct intel_context *ce)
> > __i915_active_acquire(&ce->active);
> > - if (intel_context_is_barrier(ce) || intel_engine_uses_guc(ce->engine))
> > + if (intel_context_is_barrier(ce) || intel_engine_uses_guc(ce->engine) ||
> > + intel_context_is_parallel(ce))
> > return 0;
> > /* Preallocate tracking nodes */
> > @@ -563,7 +564,6 @@ void intel_context_bind_parent_child(struct intel_context *parent,
> > * Callers responsibility to validate that this function is used
> > * correctly but we use GEM_BUG_ON here ensure that they do.
> > */
> > - GEM_BUG_ON(!intel_engine_uses_guc(parent->engine));
> > GEM_BUG_ON(intel_context_is_pinned(parent));
> > GEM_BUG_ON(intel_context_is_child(parent));
> > GEM_BUG_ON(intel_context_is_pinned(child));
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index bedb80057046..8cd986bdf26c 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -927,8 +927,7 @@ static void execlists_submit_ports(struct intel_engine_cs *engine)
> > static bool ctx_single_port_submission(const struct intel_context *ce)
> > {
> > - return (IS_ENABLED(CONFIG_DRM_I915_GVT) &&
> > - intel_context_force_single_submission(ce));
> > + return intel_context_force_single_submission(ce);
> Does this change not affect all execlist operation rather than just parallel
> submission?
>
I don't think so. The only place that sets single submission was in the
GVT code. I think was an optimization so this would just compile out if
GVT wasn't built.
> > }
> > static bool can_merge_ctx(const struct intel_context *prev,
> > @@ -2598,6 +2597,58 @@ static void execlists_context_cancel_request(struct intel_context *ce,
> > current->comm);
> > }
> > +static struct intel_context *
> > +execlists_create_parallel(struct intel_engine_cs **engines,
> > + unsigned int num_siblings,
> > + unsigned int width)
> > +{
> > + struct intel_engine_cs **siblings = NULL;
> > + struct intel_context *parent = NULL, *ce, *err;
> > + int i, j;
> > +
> > + GEM_BUG_ON(num_siblings != 1);
> > +
> > + siblings = kmalloc_array(num_siblings,
> > + sizeof(*siblings),
> > + GFP_KERNEL);
> > + if (!siblings)
> > + return ERR_PTR(-ENOMEM);
> > +
> > + for (i = 0; i < width; ++i) {
> > + for (j = 0; j < num_siblings; ++j)
> > + siblings[j] = engines[i * num_siblings + j];
> What is the purpose of this array? The only usage that I can see is
> siblings[0] on the line below. The rest of the entries never seem to be
> used. So you could just replace that with 'engines[i * num_siblings]' and
> drop the siblings array itself completely?
>
Yes, this can be dropped as num_siblings must be 1. I think this was
copied and pasted from the GuC code where num_siblings could be more
than 1. Will fixup in a standalone patch after we merge the GuC parallel
submission.
Matt
> John.
>
>
> > +
> > + ce = intel_context_create(siblings[0]);
> > + if (!ce) {
> > + err = ERR_PTR(-ENOMEM);
> > + goto unwind;
> > + }
> > +
> > + if (i == 0)
> > + parent = ce;
> > + else
> > + intel_context_bind_parent_child(parent, ce);
> > + }
> > +
> > + parent->parallel.fence_context = dma_fence_context_alloc(1);
> > +
> > + intel_context_set_nopreempt(parent);
> > + intel_context_set_single_submission(parent);
> > + for_each_child(parent, ce) {
> > + intel_context_set_nopreempt(ce);
> > + intel_context_set_single_submission(ce);
> > + }
> > +
> > + kfree(siblings);
> > + return parent;
> > +
> > +unwind:
> > + if (parent)
> > + intel_context_put(parent);
> > + kfree(siblings);
> > + return err;
> > +}
> > +
> > static const struct intel_context_ops execlists_context_ops = {
> > .flags = COPS_HAS_INFLIGHT,
> > @@ -2616,6 +2667,7 @@ static const struct intel_context_ops execlists_context_ops = {
> > .reset = lrc_reset,
> > .destroy = lrc_destroy,
> > + .create_parallel = execlists_create_parallel,
> > .create_virtual = execlists_create_virtual,
> > };
> > diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
> > index 56156cf18c41..70f4b309522d 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_lrc.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
> > @@ -1065,6 +1065,8 @@ lrc_pin(struct intel_context *ce,
> > void lrc_unpin(struct intel_context *ce)
> > {
> > + if (unlikely(ce->parallel.last_rq))
> > + i915_request_put(ce->parallel.last_rq);
> > check_redzone((void *)ce->lrc_reg_state - LRC_STATE_OFFSET,
> > ce->engine);
> > }
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > index d7710debcd47..145ffe265305 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > @@ -2960,8 +2960,6 @@ static void guc_parent_context_unpin(struct intel_context *ce)
> > GEM_BUG_ON(!intel_context_is_parent(ce));
> > GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
> > - if (ce->parallel.last_rq)
> > - i915_request_put(ce->parallel.last_rq);
> > unpin_guc_id(guc, ce);
> > lrc_unpin(ce);
> > }
>
^ permalink raw reply [flat|nested] 69+ messages in thread* Re: [PATCH 25/25] drm/i915/execlists: Weak parallel submission support for execlists
@ 2021-10-14 18:55 ` Matthew Brost
0 siblings, 0 replies; 69+ messages in thread
From: Matthew Brost @ 2021-10-14 18:55 UTC (permalink / raw)
To: John Harrison; +Cc: intel-gfx, dri-devel
On Thu, Oct 14, 2021 at 11:42:41AM -0700, John Harrison wrote:
> On 10/14/2021 10:20, Matthew Brost wrote:
> > A weak implementation of parallel submission (multi-bb execbuf IOCTL) for
> > execlists. Doing as little as possible to support this interface for
> > execlists - basically just passing submit fences between each request
> > generated and virtual engines are not allowed. This is on par with what
> > is there for the existing (hopefully soon deprecated) bonding interface.
> >
> > We perma-pin these execlists contexts to align with GuC implementation.
> >
> > Signed-off-by: Matthew Brost <matthew.brost@intel.com>
> > ---
> > drivers/gpu/drm/i915/gem/i915_gem_context.c | 10 ++--
> > drivers/gpu/drm/i915/gt/intel_context.c | 4 +-
> > .../drm/i915/gt/intel_execlists_submission.c | 56 ++++++++++++++++++-
> > drivers/gpu/drm/i915/gt/intel_lrc.c | 2 +
> > .../gpu/drm/i915/gt/uc/intel_guc_submission.c | 2 -
> > 5 files changed, 64 insertions(+), 10 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/i915/gem/i915_gem_context.c b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> > index fb33d0322960..35e87a7d0ea9 100644
> > --- a/drivers/gpu/drm/i915/gem/i915_gem_context.c
> > +++ b/drivers/gpu/drm/i915/gem/i915_gem_context.c
> > @@ -570,10 +570,6 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
> > struct intel_engine_cs **siblings = NULL;
> > intel_engine_mask_t prev_mask;
> > - /* FIXME: This is NIY for execlists */
> > - if (!(intel_uc_uses_guc_submission(&i915->gt.uc)))
> > - return -ENODEV;
> > -
> > if (get_user(slot, &ext->engine_index))
> > return -EFAULT;
> > @@ -583,6 +579,12 @@ set_proto_ctx_engines_parallel_submit(struct i915_user_extension __user *base,
> > if (get_user(num_siblings, &ext->num_siblings))
> > return -EFAULT;
> > + if (!intel_uc_uses_guc_submission(&i915->gt.uc) && num_siblings != 1) {
> > + drm_dbg(&i915->drm, "Only 1 sibling (%d) supported in non-GuC mode\n",
> > + num_siblings);
> > + return -EINVAL;
> > + }
> > +
> > if (slot >= set->num_engines) {
> > drm_dbg(&i915->drm, "Invalid placement value, %d >= %d\n",
> > slot, set->num_engines);
> > diff --git a/drivers/gpu/drm/i915/gt/intel_context.c b/drivers/gpu/drm/i915/gt/intel_context.c
> > index 5634d14052bc..1bec92e1d8e6 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_context.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_context.c
> > @@ -79,7 +79,8 @@ static int intel_context_active_acquire(struct intel_context *ce)
> > __i915_active_acquire(&ce->active);
> > - if (intel_context_is_barrier(ce) || intel_engine_uses_guc(ce->engine))
> > + if (intel_context_is_barrier(ce) || intel_engine_uses_guc(ce->engine) ||
> > + intel_context_is_parallel(ce))
> > return 0;
> > /* Preallocate tracking nodes */
> > @@ -563,7 +564,6 @@ void intel_context_bind_parent_child(struct intel_context *parent,
> > * Callers responsibility to validate that this function is used
> > * correctly but we use GEM_BUG_ON here ensure that they do.
> > */
> > - GEM_BUG_ON(!intel_engine_uses_guc(parent->engine));
> > GEM_BUG_ON(intel_context_is_pinned(parent));
> > GEM_BUG_ON(intel_context_is_child(parent));
> > GEM_BUG_ON(intel_context_is_pinned(child));
> > diff --git a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > index bedb80057046..8cd986bdf26c 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_execlists_submission.c
> > @@ -927,8 +927,7 @@ static void execlists_submit_ports(struct intel_engine_cs *engine)
> > static bool ctx_single_port_submission(const struct intel_context *ce)
> > {
> > - return (IS_ENABLED(CONFIG_DRM_I915_GVT) &&
> > - intel_context_force_single_submission(ce));
> > + return intel_context_force_single_submission(ce);
> Does this change not affect all execlist operation rather than just parallel
> submission?
>
I don't think so. The only place that sets single submission was in the
GVT code. I think was an optimization so this would just compile out if
GVT wasn't built.
> > }
> > static bool can_merge_ctx(const struct intel_context *prev,
> > @@ -2598,6 +2597,58 @@ static void execlists_context_cancel_request(struct intel_context *ce,
> > current->comm);
> > }
> > +static struct intel_context *
> > +execlists_create_parallel(struct intel_engine_cs **engines,
> > + unsigned int num_siblings,
> > + unsigned int width)
> > +{
> > + struct intel_engine_cs **siblings = NULL;
> > + struct intel_context *parent = NULL, *ce, *err;
> > + int i, j;
> > +
> > + GEM_BUG_ON(num_siblings != 1);
> > +
> > + siblings = kmalloc_array(num_siblings,
> > + sizeof(*siblings),
> > + GFP_KERNEL);
> > + if (!siblings)
> > + return ERR_PTR(-ENOMEM);
> > +
> > + for (i = 0; i < width; ++i) {
> > + for (j = 0; j < num_siblings; ++j)
> > + siblings[j] = engines[i * num_siblings + j];
> What is the purpose of this array? The only usage that I can see is
> siblings[0] on the line below. The rest of the entries never seem to be
> used. So you could just replace that with 'engines[i * num_siblings]' and
> drop the siblings array itself completely?
>
Yes, this can be dropped as num_siblings must be 1. I think this was
copied and pasted from the GuC code where num_siblings could be more
than 1. Will fixup in a standalone patch after we merge the GuC parallel
submission.
Matt
> John.
>
>
> > +
> > + ce = intel_context_create(siblings[0]);
> > + if (!ce) {
> > + err = ERR_PTR(-ENOMEM);
> > + goto unwind;
> > + }
> > +
> > + if (i == 0)
> > + parent = ce;
> > + else
> > + intel_context_bind_parent_child(parent, ce);
> > + }
> > +
> > + parent->parallel.fence_context = dma_fence_context_alloc(1);
> > +
> > + intel_context_set_nopreempt(parent);
> > + intel_context_set_single_submission(parent);
> > + for_each_child(parent, ce) {
> > + intel_context_set_nopreempt(ce);
> > + intel_context_set_single_submission(ce);
> > + }
> > +
> > + kfree(siblings);
> > + return parent;
> > +
> > +unwind:
> > + if (parent)
> > + intel_context_put(parent);
> > + kfree(siblings);
> > + return err;
> > +}
> > +
> > static const struct intel_context_ops execlists_context_ops = {
> > .flags = COPS_HAS_INFLIGHT,
> > @@ -2616,6 +2667,7 @@ static const struct intel_context_ops execlists_context_ops = {
> > .reset = lrc_reset,
> > .destroy = lrc_destroy,
> > + .create_parallel = execlists_create_parallel,
> > .create_virtual = execlists_create_virtual,
> > };
> > diff --git a/drivers/gpu/drm/i915/gt/intel_lrc.c b/drivers/gpu/drm/i915/gt/intel_lrc.c
> > index 56156cf18c41..70f4b309522d 100644
> > --- a/drivers/gpu/drm/i915/gt/intel_lrc.c
> > +++ b/drivers/gpu/drm/i915/gt/intel_lrc.c
> > @@ -1065,6 +1065,8 @@ lrc_pin(struct intel_context *ce,
> > void lrc_unpin(struct intel_context *ce)
> > {
> > + if (unlikely(ce->parallel.last_rq))
> > + i915_request_put(ce->parallel.last_rq);
> > check_redzone((void *)ce->lrc_reg_state - LRC_STATE_OFFSET,
> > ce->engine);
> > }
> > diff --git a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > index d7710debcd47..145ffe265305 100644
> > --- a/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > +++ b/drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c
> > @@ -2960,8 +2960,6 @@ static void guc_parent_context_unpin(struct intel_context *ce)
> > GEM_BUG_ON(!intel_context_is_parent(ce));
> > GEM_BUG_ON(!intel_engine_is_virtual(ce->engine));
> > - if (ce->parallel.last_rq)
> > - i915_request_put(ce->parallel.last_rq);
> > unpin_guc_id(guc, ce);
> > lrc_unpin(ce);
> > }
>
^ permalink raw reply [flat|nested] 69+ messages in thread
* [Intel-gfx] ✗ Fi.CI.CHECKPATCH: warning for Parallel submission aka multi-bb execbuf (rev7)
2021-10-14 17:19 ` Matthew Brost
` (25 preceding siblings ...)
(?)
@ 2021-10-14 23:50 ` Patchwork
-1 siblings, 0 replies; 69+ messages in thread
From: Patchwork @ 2021-10-14 23:50 UTC (permalink / raw)
To: Matthew Brost; +Cc: intel-gfx
== Series Details ==
Series: Parallel submission aka multi-bb execbuf (rev7)
URL : https://patchwork.freedesktop.org/series/92789/
State : warning
== Summary ==
$ dim checkpatch origin/drm-tip
84599e51bd29 drm/i915/guc: Move GuC guc_id allocation under submission state sub-struct
8ae907f4ab40 drm/i915/guc: Take GT PM ref when deregistering context
-:83: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'gt' - possible side-effects?
#83: FILE: drivers/gpu/drm/i915/gt/intel_gt_pm.h:44:
+#define with_intel_gt_pm(gt, tmp) \
+ for (tmp = 1, intel_gt_pm_get(gt); tmp; \
+ intel_gt_pm_put(gt), tmp = 0)
-:83: CHECK:MACRO_ARG_REUSE: Macro argument reuse 'tmp' - possible side-effects?
#83: FILE: drivers/gpu/drm/i915/gt/intel_gt_pm.h:44:
+#define with_intel_gt_pm(gt, tmp) \
+ for (tmp = 1, intel_gt_pm_get(gt); tmp; \
+ intel_gt_pm_put(gt), tmp = 0)
total: 0 errors, 0 warnings, 2 checks, 290 lines checked
caa597edafab drm/i915/guc: Take engine PM when a context is pinned with GuC submission
35ecc2c2e66d drm/i915/guc: Don't call switch_to_kernel_context with GuC submission
5913bebc8358 drm/i915: Add logical engine mapping
81882ab40924 drm/i915: Expose logical engine instance to user
a91a24abd8c2 drm/i915/guc: Introduce context parent-child relationship
08c07249af20 drm/i915/guc: Add multi-lrc context registration
fda03257da24 drm/i915/guc: Ensure GuC schedule operations do not operate on child contexts
84e7e2921b9b drm/i915/guc: Assign contexts in parent-child relationship consecutive guc_ids
937b37f379ea drm/i915/guc: Implement parallel context pin / unpin functions
fe0a09e6a3fb drm/i915/guc: Implement multi-lrc submission
-:398: CHECK:SPACING: spaces preferred around that '*' (ctx:ExV)
#398: FILE: drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c:774:
+ *wqi++ = child->ring->tail / sizeof(u64);
^
total: 0 errors, 0 warnings, 1 checks, 597 lines checked
264ff2b6144b drm/i915/guc: Insert submit fences between requests in parent-child relationship
45d246c108f9 drm/i915/guc: Implement multi-lrc reset
-:22: WARNING:TYPO_SPELLING: 'ununsed' may be misspelled - perhaps 'unused'?
#22:
- Delete ununsed is_multi_lrc function
^^^^^^^
total: 0 errors, 1 warnings, 0 checks, 163 lines checked
0f3988587b37 drm/i915/guc: Update debugfs for GuC multi-lrc
b474e9648a8b drm/i915/guc: Connect UAPI to GuC multi-lrc interface
f6db1ca849ee drm/i915/doc: Update parallel submit doc to point to i915_drm.h
-:13: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating?
#13:
deleted file mode 100644
total: 0 errors, 1 warnings, 0 checks, 10 lines checked
4ba62c9f9d88 drm/i915/guc: Add basic GuC multi-lrc selftest
-:22: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating?
#22:
new file mode 100644
total: 0 errors, 1 warnings, 0 checks, 190 lines checked
a79f00b57457 drm/i915/guc: Implement no mid batch preemption for multi-lrc
1d47f6b2307a drm/i915: Multi-BB execbuf
-:385: CHECK:MACRO_ARG_REUSE: Macro argument reuse '_i' - possible side-effects?
#385: FILE: drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c:1874:
+#define for_each_batch_create_order(_eb, _i) \
+ for ((_i) = 0; (_i) < (_eb)->num_batches; ++(_i))
-:387: ERROR:MULTISTATEMENT_MACRO_USE_DO_WHILE: Macros with multiple statements should be enclosed in a do - while loop
#387: FILE: drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c:1876:
+#define for_each_batch_add_order(_eb, _i) \
+ BUILD_BUG_ON(!typecheck(int, _i)); \
+ for ((_i) = (_eb)->num_batches - 1; (_i) >= 0; --(_i))
-:387: CHECK:MACRO_ARG_REUSE: Macro argument reuse '_i' - possible side-effects?
#387: FILE: drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c:1876:
+#define for_each_batch_add_order(_eb, _i) \
+ BUILD_BUG_ON(!typecheck(int, _i)); \
+ for ((_i) = (_eb)->num_batches - 1; (_i) >= 0; --(_i))
total: 1 errors, 0 warnings, 2 checks, 1289 lines checked
39cdfa25bc1e drm/i915/guc: Handle errors in multi-lrc requests
fe5061134b09 drm/i915: Make request conflict tracking understand parallel submits
3b3b377a0bbd drm/i915: Update I915_GEM_BUSY IOCTL to understand composite fences
25451cb4c07c drm/i915: Enable multi-bb execbuf
2bc7c2c9b7ac drm/i915/execlists: Weak parallel submission support for execlists
^ permalink raw reply [flat|nested] 69+ messages in thread* [Intel-gfx] ✗ Fi.CI.SPARSE: warning for Parallel submission aka multi-bb execbuf (rev7)
2021-10-14 17:19 ` Matthew Brost
` (26 preceding siblings ...)
(?)
@ 2021-10-14 23:51 ` Patchwork
-1 siblings, 0 replies; 69+ messages in thread
From: Patchwork @ 2021-10-14 23:51 UTC (permalink / raw)
To: Matthew Brost; +Cc: intel-gfx
== Series Details ==
Series: Parallel submission aka multi-bb execbuf (rev7)
URL : https://patchwork.freedesktop.org/series/92789/
State : warning
== Summary ==
$ dim sparse --fast origin/drm-tip
Sparse version: v0.6.2
Fast mode used, each commit won't be checked separately.
+drivers/gpu/drm/i915/gt/intel_reset.c:1392:5: warning: context imbalance in 'intel_gt_reset_trylock' - different lock contexts for basic block
+drivers/gpu/drm/i915/i915_perf.c:1442:15: warning: memset with byte count of 16777216
+drivers/gpu/drm/i915/i915_perf.c:1496:15: warning: memset with byte count of 16777216
+drivers/gpu/drm/i915/intel_wakeref.c:137:19: warning: context imbalance in 'wakeref_auto_timeout' - unexpected unlock
+drivers/gpu/drm/i915/selftests/i915_syncmap.c:80:54: warning: dubious: x | !y
+./include/asm-generic/bitops/find.h:112:45: warning: shift count is negative (-262080)
+./include/asm-generic/bitops/find.h:32:31: warning: shift count is negative (-262080)
+./include/linux/spinlock.h:418:9: warning: context imbalance in 'fwtable_read16' - different lock contexts for basic block
+./include/linux/spinlock.h:418:9: warning: context imbalance in 'fwtable_read32' - different lock contexts for basic block
+./include/linux/spinlock.h:418:9: warning: context imbalance in 'fwtable_read64' - different lock contexts for basic block
+./include/linux/spinlock.h:418:9: warning: context imbalance in 'fwtable_read8' - different lock contexts for basic block
+./include/linux/spinlock.h:418:9: warning: context imbalance in 'fwtable_write16' - different lock contexts for basic block
+./include/linux/spinlock.h:418:9: warning: context imbalance in 'fwtable_write32' - different lock contexts for basic block
+./include/linux/spinlock.h:418:9: warning: context imbalance in 'fwtable_write8' - different lock contexts for basic block
+./include/linux/spinlock.h:418:9: warning: context imbalance in 'gen6_write16' - different lock contexts for basic block
+./include/linux/spinlock.h:418:9: warning: context imbalance in 'gen6_write32' - different lock contexts for basic block
+./include/linux/spinlock.h:418:9: warning: context imbalance in 'gen6_write8' - different lock contexts for basic block
^ permalink raw reply [flat|nested] 69+ messages in thread* [Intel-gfx] ✓ Fi.CI.BAT: success for Parallel submission aka multi-bb execbuf (rev7)
2021-10-14 17:19 ` Matthew Brost
` (27 preceding siblings ...)
(?)
@ 2021-10-15 0:25 ` Patchwork
-1 siblings, 0 replies; 69+ messages in thread
From: Patchwork @ 2021-10-15 0:25 UTC (permalink / raw)
To: Matthew Brost; +Cc: intel-gfx
[-- Attachment #1: Type: text/plain, Size: 7039 bytes --]
== Series Details ==
Series: Parallel submission aka multi-bb execbuf (rev7)
URL : https://patchwork.freedesktop.org/series/92789/
State : success
== Summary ==
CI Bug Log - changes from CI_DRM_10739 -> Patchwork_21340
====================================================
Summary
-------
**SUCCESS**
No regressions found.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/index.html
New tests
---------
New tests have been introduced between CI_DRM_10739 and Patchwork_21340:
### New IGT tests (1) ###
* igt@i915_selftest@live@guc_multi_lrc:
- Statuses : 30 pass(s)
- Exec time: [0.43, 5.08] s
Known issues
------------
Here are the changes found in Patchwork_21340 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@amdgpu/amd_basic@query-info:
- fi-kbl-soraka: NOTRUN -> [SKIP][1] ([fdo#109271])
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/fi-kbl-soraka/igt@amdgpu/amd_basic@query-info.html
* igt@i915_module_load@reload:
- fi-kbl-soraka: [PASS][2] -> [DMESG-WARN][3] ([i915#1982])
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/fi-kbl-soraka/igt@i915_module_load@reload.html
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/fi-kbl-soraka/igt@i915_module_load@reload.html
* igt@i915_selftest@live@hangcheck:
- fi-ivb-3770: [PASS][4] -> [INCOMPLETE][5] ([i915#3303])
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/fi-ivb-3770/igt@i915_selftest@live@hangcheck.html
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/fi-ivb-3770/igt@i915_selftest@live@hangcheck.html
* igt@kms_chamelium@hdmi-edid-read:
- fi-bdw-samus: NOTRUN -> [SKIP][6] ([fdo#109271] / [fdo#111827]) +8 similar issues
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/fi-bdw-samus/igt@kms_chamelium@hdmi-edid-read.html
* igt@kms_flip@basic-plain-flip@c-dp1:
- fi-cfl-8109u: [PASS][7] -> [FAIL][8] ([i915#4165])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/fi-cfl-8109u/igt@kms_flip@basic-plain-flip@c-dp1.html
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/fi-cfl-8109u/igt@kms_flip@basic-plain-flip@c-dp1.html
* igt@kms_frontbuffer_tracking@basic:
- fi-cml-u2: [PASS][9] -> [DMESG-WARN][10] ([i915#4269])
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/fi-cml-u2/igt@kms_frontbuffer_tracking@basic.html
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/fi-cml-u2/igt@kms_frontbuffer_tracking@basic.html
* igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-b:
- fi-cfl-8109u: [PASS][11] -> [DMESG-WARN][12] ([i915#295]) +14 similar issues
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/fi-cfl-8109u/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-b.html
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/fi-cfl-8109u/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-b.html
* igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d:
- fi-bdw-samus: NOTRUN -> [SKIP][13] ([fdo#109271]) +29 similar issues
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/fi-bdw-samus/igt@kms_pipe_crc_basic@compare-crc-sanitycheck-pipe-d.html
* igt@runner@aborted:
- fi-bdw-5557u: NOTRUN -> [FAIL][14] ([i915#1602] / [i915#2029])
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/fi-bdw-5557u/igt@runner@aborted.html
- fi-bxt-dsi: NOTRUN -> [FAIL][15] ([i915#2426] / [i915#3363])
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/fi-bxt-dsi/igt@runner@aborted.html
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[fdo#111827]: https://bugs.freedesktop.org/show_bug.cgi?id=111827
[i915#1602]: https://gitlab.freedesktop.org/drm/intel/issues/1602
[i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
[i915#2029]: https://gitlab.freedesktop.org/drm/intel/issues/2029
[i915#2426]: https://gitlab.freedesktop.org/drm/intel/issues/2426
[i915#295]: https://gitlab.freedesktop.org/drm/intel/issues/295
[i915#3303]: https://gitlab.freedesktop.org/drm/intel/issues/3303
[i915#3363]: https://gitlab.freedesktop.org/drm/intel/issues/3363
[i915#4165]: https://gitlab.freedesktop.org/drm/intel/issues/4165
[i915#4269]: https://gitlab.freedesktop.org/drm/intel/issues/4269
Participating hosts (39 -> 37)
------------------------------
Additional (1): fi-bdw-samus
Missing (3): fi-bsw-cyan bat-dg1-6 fi-hsw-4200u
Build changes
-------------
* Linux: CI_DRM_10739 -> Patchwork_21340
CI-20190529: 20190529
CI_DRM_10739: 9fedda3a0fa2032ccd125a7bedea74d3ec99d930 @ git://anongit.freedesktop.org/gfx-ci/linux
IGT_6249: 04b156109f1c5128d5ace67420ee2e35e8a24e1d @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
Patchwork_21340: 2bc7c2c9b7acee83c44eade627226f344c6410ad @ git://anongit.freedesktop.org/gfx-ci/linux
== Linux commits ==
2bc7c2c9b7ac drm/i915/execlists: Weak parallel submission support for execlists
25451cb4c07c drm/i915: Enable multi-bb execbuf
3b3b377a0bbd drm/i915: Update I915_GEM_BUSY IOCTL to understand composite fences
fe5061134b09 drm/i915: Make request conflict tracking understand parallel submits
39cdfa25bc1e drm/i915/guc: Handle errors in multi-lrc requests
1d47f6b2307a drm/i915: Multi-BB execbuf
a79f00b57457 drm/i915/guc: Implement no mid batch preemption for multi-lrc
4ba62c9f9d88 drm/i915/guc: Add basic GuC multi-lrc selftest
f6db1ca849ee drm/i915/doc: Update parallel submit doc to point to i915_drm.h
b474e9648a8b drm/i915/guc: Connect UAPI to GuC multi-lrc interface
0f3988587b37 drm/i915/guc: Update debugfs for GuC multi-lrc
45d246c108f9 drm/i915/guc: Implement multi-lrc reset
264ff2b6144b drm/i915/guc: Insert submit fences between requests in parent-child relationship
fe0a09e6a3fb drm/i915/guc: Implement multi-lrc submission
937b37f379ea drm/i915/guc: Implement parallel context pin / unpin functions
84e7e2921b9b drm/i915/guc: Assign contexts in parent-child relationship consecutive guc_ids
fda03257da24 drm/i915/guc: Ensure GuC schedule operations do not operate on child contexts
08c07249af20 drm/i915/guc: Add multi-lrc context registration
a91a24abd8c2 drm/i915/guc: Introduce context parent-child relationship
81882ab40924 drm/i915: Expose logical engine instance to user
5913bebc8358 drm/i915: Add logical engine mapping
35ecc2c2e66d drm/i915/guc: Don't call switch_to_kernel_context with GuC submission
caa597edafab drm/i915/guc: Take engine PM when a context is pinned with GuC submission
8ae907f4ab40 drm/i915/guc: Take GT PM ref when deregistering context
84599e51bd29 drm/i915/guc: Move GuC guc_id allocation under submission state sub-struct
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/index.html
[-- Attachment #2: Type: text/html, Size: 8192 bytes --]
^ permalink raw reply [flat|nested] 69+ messages in thread* [Intel-gfx] ✓ Fi.CI.IGT: success for Parallel submission aka multi-bb execbuf (rev7)
2021-10-14 17:19 ` Matthew Brost
` (28 preceding siblings ...)
(?)
@ 2021-10-15 6:12 ` Patchwork
-1 siblings, 0 replies; 69+ messages in thread
From: Patchwork @ 2021-10-15 6:12 UTC (permalink / raw)
To: Matthew Brost; +Cc: intel-gfx
[-- Attachment #1: Type: text/plain, Size: 30270 bytes --]
== Series Details ==
Series: Parallel submission aka multi-bb execbuf (rev7)
URL : https://patchwork.freedesktop.org/series/92789/
State : success
== Summary ==
CI Bug Log - changes from CI_DRM_10739_full -> Patchwork_21340_full
====================================================
Summary
-------
**SUCCESS**
No regressions found.
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in Patchwork_21340_full:
### IGT changes ###
#### Suppressed ####
The following results come from untrusted machines, tests, or statuses.
They do not affect the overall result.
* {igt@kms_bw@linear-tiling-2-displays-3840x2160p}:
- shard-kbl: NOTRUN -> [DMESG-FAIL][1]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-kbl2/igt@kms_bw@linear-tiling-2-displays-3840x2160p.html
* {igt@kms_bw@linear-tiling-3-displays-2560x1440p}:
- shard-snb: NOTRUN -> [FAIL][2] +1 similar issue
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-snb5/igt@kms_bw@linear-tiling-3-displays-2560x1440p.html
* {igt@kms_bw@linear-tiling-4-displays-1920x1080p}:
- shard-kbl: NOTRUN -> [FAIL][3]
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-kbl2/igt@kms_bw@linear-tiling-4-displays-1920x1080p.html
New tests
---------
New tests have been introduced between CI_DRM_10739_full and Patchwork_21340_full:
### New IGT tests (1) ###
* igt@i915_selftest@live@guc_multi_lrc:
- Statuses : 6 pass(s)
- Exec time: [0.99, 4.97] s
Known issues
------------
Here are the changes found in Patchwork_21340_full that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@gem_create@create-massive:
- shard-kbl: NOTRUN -> [DMESG-WARN][4] ([i915#3002])
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-kbl4/igt@gem_create@create-massive.html
- shard-apl: NOTRUN -> [DMESG-WARN][5] ([i915#3002])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-apl8/igt@gem_create@create-massive.html
* igt@gem_ctx_persistence@process:
- shard-snb: NOTRUN -> [SKIP][6] ([fdo#109271] / [i915#1099]) +3 similar issues
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-snb5/igt@gem_ctx_persistence@process.html
* igt@gem_ctx_sseu@engines:
- shard-tglb: NOTRUN -> [SKIP][7] ([i915#280])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@gem_ctx_sseu@engines.html
* igt@gem_eio@unwedge-stress:
- shard-tglb: [PASS][8] -> [TIMEOUT][9] ([i915#2369] / [i915#3063] / [i915#3648])
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-tglb6/igt@gem_eio@unwedge-stress.html
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb3/igt@gem_eio@unwedge-stress.html
* igt@gem_exec_fair@basic-none-share@rcs0:
- shard-iclb: [PASS][10] -> [FAIL][11] ([i915#2842])
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-iclb3/igt@gem_exec_fair@basic-none-share@rcs0.html
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-iclb6/igt@gem_exec_fair@basic-none-share@rcs0.html
* igt@gem_exec_fair@basic-none-solo@rcs0:
- shard-glk: [PASS][12] -> [FAIL][13] ([i915#2842])
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-glk5/igt@gem_exec_fair@basic-none-solo@rcs0.html
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-glk9/igt@gem_exec_fair@basic-none-solo@rcs0.html
- shard-tglb: NOTRUN -> [FAIL][14] ([i915#2842])
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb1/igt@gem_exec_fair@basic-none-solo@rcs0.html
* igt@gem_exec_fair@basic-none@vcs1:
- shard-iclb: NOTRUN -> [FAIL][15] ([i915#2842])
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-iclb2/igt@gem_exec_fair@basic-none@vcs1.html
* igt@gem_exec_fair@basic-sync@rcs0:
- shard-tglb: NOTRUN -> [SKIP][16] ([i915#2848])
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@gem_exec_fair@basic-sync@rcs0.html
* igt@gem_exec_flush@basic-batch-kernel-default-cmd:
- shard-snb: NOTRUN -> [SKIP][17] ([fdo#109271]) +385 similar issues
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-snb7/igt@gem_exec_flush@basic-batch-kernel-default-cmd.html
* igt@gem_exec_whisper@basic-queues-forked-all:
- shard-glk: [PASS][18] -> [DMESG-WARN][19] ([i915#118])
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-glk9/igt@gem_exec_whisper@basic-queues-forked-all.html
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-glk6/igt@gem_exec_whisper@basic-queues-forked-all.html
* igt@gem_pread@exhaustion:
- shard-apl: NOTRUN -> [WARN][20] ([i915#2658])
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-apl7/igt@gem_pread@exhaustion.html
* igt@gem_pxp@reject-modify-context-protection-on:
- shard-tglb: NOTRUN -> [SKIP][21] ([i915#4270])
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@gem_pxp@reject-modify-context-protection-on.html
* igt@gem_workarounds@suspend-resume-fd:
- shard-kbl: [PASS][22] -> [DMESG-WARN][23] ([i915#180]) +2 similar issues
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-kbl6/igt@gem_workarounds@suspend-resume-fd.html
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-kbl7/igt@gem_workarounds@suspend-resume-fd.html
* igt@gen9_exec_parse@allowed-single:
- shard-skl: NOTRUN -> [DMESG-WARN][24] ([i915#1436] / [i915#1982] / [i915#716])
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-skl3/igt@gen9_exec_parse@allowed-single.html
* igt@gen9_exec_parse@bb-start-far:
- shard-tglb: NOTRUN -> [SKIP][25] ([i915#2856])
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb1/igt@gen9_exec_parse@bb-start-far.html
* igt@i915_suspend@forcewake:
- shard-tglb: [PASS][26] -> [INCOMPLETE][27] ([i915#2411] / [i915#456])
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-tglb1/igt@i915_suspend@forcewake.html
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb7/igt@i915_suspend@forcewake.html
* igt@kms_atomic_transition@plane-all-modeset-transition-fencing:
- shard-tglb: NOTRUN -> [SKIP][28] ([i915#1769])
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@kms_atomic_transition@plane-all-modeset-transition-fencing.html
* igt@kms_big_fb@linear-64bpp-rotate-270:
- shard-tglb: NOTRUN -> [SKIP][29] ([fdo#111614])
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@kms_big_fb@linear-64bpp-rotate-270.html
* igt@kms_big_fb@x-tiled-max-hw-stride-64bpp-rotate-0-hflip:
- shard-apl: NOTRUN -> [SKIP][30] ([fdo#109271] / [i915#3777])
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-apl6/igt@kms_big_fb@x-tiled-max-hw-stride-64bpp-rotate-0-hflip.html
* igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-0-hflip-async-flip:
- shard-tglb: [PASS][31] -> [FAIL][32] ([i915#3722])
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-tglb2/igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-0-hflip-async-flip.html
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-0-hflip-async-flip.html
* igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-180-hflip-async-flip:
- shard-tglb: NOTRUN -> [FAIL][33] ([i915#3722])
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@kms_big_fb@y-tiled-max-hw-stride-64bpp-rotate-180-hflip-async-flip.html
* igt@kms_big_fb@yf-tiled-max-hw-stride-64bpp-rotate-0:
- shard-apl: NOTRUN -> [SKIP][34] ([fdo#109271]) +133 similar issues
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-apl6/igt@kms_big_fb@yf-tiled-max-hw-stride-64bpp-rotate-0.html
* igt@kms_ccs@pipe-a-bad-rotation-90-y_tiled_gen12_mc_ccs:
- shard-apl: NOTRUN -> [SKIP][35] ([fdo#109271] / [i915#3886]) +4 similar issues
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-apl2/igt@kms_ccs@pipe-a-bad-rotation-90-y_tiled_gen12_mc_ccs.html
* igt@kms_ccs@pipe-a-crc-sprite-planes-basic-y_tiled_gen12_mc_ccs:
- shard-kbl: NOTRUN -> [SKIP][36] ([fdo#109271] / [i915#3886]) +4 similar issues
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-kbl3/igt@kms_ccs@pipe-a-crc-sprite-planes-basic-y_tiled_gen12_mc_ccs.html
* igt@kms_ccs@pipe-b-missing-ccs-buffer-y_tiled_gen12_mc_ccs:
- shard-tglb: NOTRUN -> [SKIP][37] ([i915#3689] / [i915#3886]) +3 similar issues
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@kms_ccs@pipe-b-missing-ccs-buffer-y_tiled_gen12_mc_ccs.html
* igt@kms_ccs@pipe-c-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc:
- shard-skl: NOTRUN -> [SKIP][38] ([fdo#109271] / [i915#3886]) +3 similar issues
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-skl3/igt@kms_ccs@pipe-c-missing-ccs-buffer-y_tiled_gen12_rc_ccs_cc.html
* igt@kms_ccs@pipe-d-crc-sprite-planes-basic-y_tiled_ccs:
- shard-tglb: NOTRUN -> [SKIP][39] ([i915#3689]) +1 similar issue
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb1/igt@kms_ccs@pipe-d-crc-sprite-planes-basic-y_tiled_ccs.html
* igt@kms_ccs@pipe-d-missing-ccs-buffer-y_tiled_gen12_rc_ccs:
- shard-kbl: NOTRUN -> [SKIP][40] ([fdo#109271]) +84 similar issues
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-kbl3/igt@kms_ccs@pipe-d-missing-ccs-buffer-y_tiled_gen12_rc_ccs.html
* igt@kms_chamelium@dp-audio-edid:
- shard-tglb: NOTRUN -> [SKIP][41] ([fdo#109284] / [fdo#111827]) +1 similar issue
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@kms_chamelium@dp-audio-edid.html
* igt@kms_chamelium@hdmi-hpd-storm:
- shard-skl: NOTRUN -> [SKIP][42] ([fdo#109271] / [fdo#111827]) +5 similar issues
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-skl4/igt@kms_chamelium@hdmi-hpd-storm.html
* igt@kms_chamelium@hdmi-mode-timings:
- shard-snb: NOTRUN -> [SKIP][43] ([fdo#109271] / [fdo#111827]) +14 similar issues
[43]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-snb5/igt@kms_chamelium@hdmi-mode-timings.html
* igt@kms_color@pipe-a-ctm-0-5:
- shard-skl: [PASS][44] -> [DMESG-WARN][45] ([i915#1982]) +2 similar issues
[44]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-skl2/igt@kms_color@pipe-a-ctm-0-5.html
[45]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-skl5/igt@kms_color@pipe-a-ctm-0-5.html
* igt@kms_color@pipe-d-ctm-negative:
- shard-skl: NOTRUN -> [SKIP][46] ([fdo#109271]) +60 similar issues
[46]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-skl3/igt@kms_color@pipe-d-ctm-negative.html
* igt@kms_color_chamelium@pipe-b-ctm-0-25:
- shard-kbl: NOTRUN -> [SKIP][47] ([fdo#109271] / [fdo#111827]) +7 similar issues
[47]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-kbl3/igt@kms_color_chamelium@pipe-b-ctm-0-25.html
* igt@kms_color_chamelium@pipe-c-ctm-0-25:
- shard-apl: NOTRUN -> [SKIP][48] ([fdo#109271] / [fdo#111827]) +11 similar issues
[48]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-apl6/igt@kms_color_chamelium@pipe-c-ctm-0-25.html
* igt@kms_content_protection@atomic:
- shard-apl: NOTRUN -> [TIMEOUT][49] ([i915#1319])
[49]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-apl6/igt@kms_content_protection@atomic.html
* igt@kms_cursor_crc@pipe-a-cursor-512x512-sliding:
- shard-tglb: NOTRUN -> [SKIP][50] ([fdo#109279] / [i915#3359])
[50]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@kms_cursor_crc@pipe-a-cursor-512x512-sliding.html
* igt@kms_cursor_crc@pipe-c-cursor-32x10-offscreen:
- shard-tglb: NOTRUN -> [SKIP][51] ([i915#3359])
[51]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@kms_cursor_crc@pipe-c-cursor-32x10-offscreen.html
* igt@kms_cursor_legacy@basic-busy-flip-before-cursor-atomic:
- shard-tglb: NOTRUN -> [SKIP][52] ([i915#4103])
[52]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@kms_cursor_legacy@basic-busy-flip-before-cursor-atomic.html
* igt@kms_flip@flip-vs-expired-vblank@a-edp1:
- shard-skl: [PASS][53] -> [FAIL][54] ([i915#79]) +1 similar issue
[53]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-skl8/igt@kms_flip@flip-vs-expired-vblank@a-edp1.html
[54]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-skl4/igt@kms_flip@flip-vs-expired-vblank@a-edp1.html
* igt@kms_flip@flip-vs-expired-vblank@b-hdmi-a2:
- shard-glk: [PASS][55] -> [FAIL][56] ([i915#79]) +1 similar issue
[55]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-glk3/igt@kms_flip@flip-vs-expired-vblank@b-hdmi-a2.html
[56]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-glk5/igt@kms_flip@flip-vs-expired-vblank@b-hdmi-a2.html
* igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-32bpp-ytilegen12rcccs:
- shard-apl: NOTRUN -> [SKIP][57] ([fdo#109271] / [i915#2672])
[57]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-apl2/igt@kms_flip_scaled_crc@flip-32bpp-ytile-to-32bpp-ytilegen12rcccs.html
* igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-pri-indfb-draw-blt:
- shard-tglb: NOTRUN -> [SKIP][58] ([fdo#111825]) +5 similar issues
[58]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-pri-indfb-draw-blt.html
* igt@kms_pipe_crc_basic@suspend-read-crc-pipe-d:
- shard-apl: NOTRUN -> [SKIP][59] ([fdo#109271] / [i915#533])
[59]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-apl6/igt@kms_pipe_crc_basic@suspend-read-crc-pipe-d.html
* igt@kms_plane_alpha_blend@pipe-c-alpha-basic:
- shard-apl: NOTRUN -> [FAIL][60] ([fdo#108145] / [i915#265]) +1 similar issue
[60]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-apl7/igt@kms_plane_alpha_blend@pipe-c-alpha-basic.html
- shard-skl: NOTRUN -> [FAIL][61] ([fdo#108145] / [i915#265])
[61]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-skl4/igt@kms_plane_alpha_blend@pipe-c-alpha-basic.html
* igt@kms_plane_cursor@pipe-d-viewport-size-256:
- shard-tglb: NOTRUN -> [DMESG-WARN][62] ([i915#1982])
[62]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@kms_plane_cursor@pipe-d-viewport-size-256.html
* igt@kms_plane_lowres@pipe-c-tiling-x:
- shard-tglb: NOTRUN -> [SKIP][63] ([i915#3536])
[63]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@kms_plane_lowres@pipe-c-tiling-x.html
* igt@kms_psr2_sf@overlay-plane-update-sf-dmg-area-5:
- shard-skl: NOTRUN -> [SKIP][64] ([fdo#109271] / [i915#658])
[64]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-skl3/igt@kms_psr2_sf@overlay-plane-update-sf-dmg-area-5.html
- shard-kbl: NOTRUN -> [SKIP][65] ([fdo#109271] / [i915#658])
[65]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-kbl3/igt@kms_psr2_sf@overlay-plane-update-sf-dmg-area-5.html
* igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-5:
- shard-apl: NOTRUN -> [SKIP][66] ([fdo#109271] / [i915#658]) +1 similar issue
[66]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-apl6/igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-5.html
* igt@kms_psr2_su@frontbuffer:
- shard-iclb: [PASS][67] -> [SKIP][68] ([fdo#109642] / [fdo#111068] / [i915#658])
[67]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-iclb2/igt@kms_psr2_su@frontbuffer.html
[68]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-iclb3/igt@kms_psr2_su@frontbuffer.html
* igt@kms_psr2_su@page_flip:
- shard-tglb: NOTRUN -> [SKIP][69] ([i915#1911])
[69]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb1/igt@kms_psr2_su@page_flip.html
* igt@kms_psr@psr2_suspend:
- shard-iclb: [PASS][70] -> [SKIP][71] ([fdo#109441])
[70]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-iclb2/igt@kms_psr@psr2_suspend.html
[71]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-iclb3/igt@kms_psr@psr2_suspend.html
* igt@kms_rotation_crc@primary-yf-tiled-reflect-x-90:
- shard-tglb: NOTRUN -> [SKIP][72] ([fdo#111615]) +2 similar issues
[72]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@kms_rotation_crc@primary-yf-tiled-reflect-x-90.html
* igt@nouveau_crc@pipe-d-source-outp-complete:
- shard-tglb: NOTRUN -> [SKIP][73] ([i915#2530])
[73]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@nouveau_crc@pipe-d-source-outp-complete.html
* igt@perf@polling-small-buf:
- shard-skl: [PASS][74] -> [FAIL][75] ([i915#1722])
[74]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-skl8/igt@perf@polling-small-buf.html
[75]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-skl4/igt@perf@polling-small-buf.html
* igt@prime_nv_test@i915_import_gtt_mmap:
- shard-tglb: NOTRUN -> [SKIP][76] ([fdo#109291]) +1 similar issue
[76]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@prime_nv_test@i915_import_gtt_mmap.html
* igt@prime_vgem@coherency-gtt:
- shard-tglb: NOTRUN -> [SKIP][77] ([fdo#111656])
[77]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@prime_vgem@coherency-gtt.html
* igt@sysfs_clients@recycle:
- shard-apl: NOTRUN -> [SKIP][78] ([fdo#109271] / [i915#2994])
[78]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-apl2/igt@sysfs_clients@recycle.html
* igt@sysfs_clients@sema-10:
- shard-skl: NOTRUN -> [SKIP][79] ([fdo#109271] / [i915#2994])
[79]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-skl3/igt@sysfs_clients@sema-10.html
- shard-kbl: NOTRUN -> [SKIP][80] ([fdo#109271] / [i915#2994]) +1 similar issue
[80]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-kbl3/igt@sysfs_clients@sema-10.html
#### Possible fixes ####
* igt@feature_discovery@psr2:
- shard-iclb: [SKIP][81] ([i915#658]) -> [PASS][82]
[81]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-iclb8/igt@feature_discovery@psr2.html
[82]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-iclb2/igt@feature_discovery@psr2.html
* igt@gem_ctx_persistence@hostile:
- shard-apl: [FAIL][83] ([i915#2410]) -> [PASS][84]
[83]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-apl3/igt@gem_ctx_persistence@hostile.html
[84]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-apl3/igt@gem_ctx_persistence@hostile.html
* igt@gem_eio@in-flight-contexts-1us:
- shard-iclb: [TIMEOUT][85] ([i915#3070]) -> [PASS][86]
[85]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-iclb8/igt@gem_eio@in-flight-contexts-1us.html
[86]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-iclb2/igt@gem_eio@in-flight-contexts-1us.html
* igt@gem_exec_fair@basic-none@rcs0:
- shard-kbl: [FAIL][87] ([i915#2842]) -> [PASS][88] +2 similar issues
[87]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-kbl6/igt@gem_exec_fair@basic-none@rcs0.html
[88]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-kbl6/igt@gem_exec_fair@basic-none@rcs0.html
* igt@gem_exec_fair@basic-pace-solo@rcs0:
- shard-glk: [FAIL][89] ([i915#2842]) -> [PASS][90] +1 similar issue
[89]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-glk5/igt@gem_exec_fair@basic-pace-solo@rcs0.html
[90]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-glk3/igt@gem_exec_fair@basic-pace-solo@rcs0.html
* igt@gem_exec_fair@basic-pace@vcs0:
- shard-tglb: [FAIL][91] ([i915#2842]) -> [PASS][92]
[91]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-tglb6/igt@gem_exec_fair@basic-pace@vcs0.html
[92]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb3/igt@gem_exec_fair@basic-pace@vcs0.html
* igt@gem_exec_fair@basic-throttle@rcs0:
- shard-iclb: [FAIL][93] ([i915#2849]) -> [PASS][94]
[93]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-iclb4/igt@gem_exec_fair@basic-throttle@rcs0.html
[94]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-iclb6/igt@gem_exec_fair@basic-throttle@rcs0.html
* igt@gem_huc_copy@huc-copy:
- shard-tglb: [SKIP][95] ([i915#2190]) -> [PASS][96]
[95]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-tglb7/igt@gem_huc_copy@huc-copy.html
[96]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb1/igt@gem_huc_copy@huc-copy.html
* igt@gem_workarounds@suspend-resume:
- shard-apl: [DMESG-WARN][97] ([i915#180]) -> [PASS][98]
[97]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-apl6/igt@gem_workarounds@suspend-resume.html
[98]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-apl7/igt@gem_workarounds@suspend-resume.html
* igt@i915_suspend@fence-restore-untiled:
- shard-skl: [INCOMPLETE][99] ([i915#146] / [i915#198]) -> [PASS][100]
[99]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-skl5/igt@i915_suspend@fence-restore-untiled.html
[100]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-skl4/igt@i915_suspend@fence-restore-untiled.html
* igt@kms_big_fb@yf-tiled-16bpp-rotate-0:
- shard-glk: [DMESG-WARN][101] ([i915#118]) -> [PASS][102]
[101]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-glk4/igt@kms_big_fb@yf-tiled-16bpp-rotate-0.html
[102]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-glk2/igt@kms_big_fb@yf-tiled-16bpp-rotate-0.html
* igt@kms_cursor_crc@pipe-a-cursor-suspend:
- shard-kbl: [DMESG-WARN][103] ([i915#180]) -> [PASS][104] +3 similar issues
[103]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-kbl7/igt@kms_cursor_crc@pipe-a-cursor-suspend.html
[104]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-kbl4/igt@kms_cursor_crc@pipe-a-cursor-suspend.html
* igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size:
- shard-skl: [FAIL][105] ([i915#2346] / [i915#533]) -> [PASS][106]
[105]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-skl1/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html
[106]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-skl1/igt@kms_cursor_legacy@flip-vs-cursor-atomic-transitions-varying-size.html
* igt@kms_flip@flip-vs-expired-vblank-interruptible@c-dp1:
- shard-apl: [FAIL][107] ([i915#79]) -> [PASS][108]
[107]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-apl2/igt@kms_flip@flip-vs-expired-vblank-interruptible@c-dp1.html
[108]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-apl2/igt@kms_flip@flip-vs-expired-vblank-interruptible@c-dp1.html
* igt@kms_flip@flip-vs-expired-vblank-interruptible@c-hdmi-a2:
- shard-glk: [FAIL][109] ([i915#79]) -> [PASS][110]
[109]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-glk7/igt@kms_flip@flip-vs-expired-vblank-interruptible@c-hdmi-a2.html
[110]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-glk6/igt@kms_flip@flip-vs-expired-vblank-interruptible@c-hdmi-a2.html
* igt@kms_flip@flip-vs-expired-vblank@a-dp1:
- shard-kbl: [FAIL][111] ([i915#79]) -> [PASS][112]
[111]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-kbl4/igt@kms_flip@flip-vs-expired-vblank@a-dp1.html
[112]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-kbl1/igt@kms_flip@flip-vs-expired-vblank@a-dp1.html
* igt@kms_flip@flip-vs-suspend-interruptible@a-edp1:
- shard-tglb: [INCOMPLETE][113] ([i915#2411] / [i915#456]) -> [PASS][114]
[113]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-tglb7/igt@kms_flip@flip-vs-suspend-interruptible@a-edp1.html
[114]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb8/igt@kms_flip@flip-vs-suspend-interruptible@a-edp1.html
* igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile:
- shard-iclb: [SKIP][115] ([i915#3701]) -> [PASS][116]
[115]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-iclb2/igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile.html
[116]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-iclb8/igt@kms_flip_scaled_crc@flip-64bpp-ytile-to-16bpp-ytile.html
* igt@kms_frontbuffer_tracking@fbc-2p-primscrn-cur-indfb-draw-mmap-wc:
- shard-glk: [FAIL][117] ([i915#2546]) -> [PASS][118]
[117]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-glk6/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-cur-indfb-draw-mmap-wc.html
[118]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-glk7/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-cur-indfb-draw-mmap-wc.html
* igt@kms_hdr@bpc-switch-suspend:
- shard-skl: [FAIL][119] ([i915#1188]) -> [PASS][120]
[119]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-skl9/igt@kms_hdr@bpc-switch-suspend.html
[120]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-skl8/igt@kms_hdr@bpc-switch-suspend.html
* igt@kms_plane@plane-panning-bottom-right-suspend@pipe-a-planes:
- shard-skl: [INCOMPLETE][121] ([i915#198]) -> [PASS][122]
[121]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-skl4/igt@kms_plane@plane-panning-bottom-right-suspend@pipe-a-planes.html
[122]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-skl3/igt@kms_plane@plane-panning-bottom-right-suspend@pipe-a-planes.html
* igt@kms_plane_alpha_blend@pipe-b-constant-alpha-min:
- shard-skl: [FAIL][123] ([fdo#108145] / [i915#265]) -> [PASS][124]
[123]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-skl7/igt@kms_plane_alpha_blend@pipe-b-constant-alpha-min.html
[124]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-skl3/igt@kms_plane_alpha_blend@pipe-b-constant-alpha-min.html
* igt@kms_psr@psr2_primary_mmap_cpu:
- shard-iclb: [SKIP][125] ([fdo#109441]) -> [PASS][126]
[125]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-iclb8/igt@kms_psr@psr2_primary_mmap_cpu.html
[126]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-iclb2/igt@kms_psr@psr2_primary_mmap_cpu.html
* igt@kms_vblank@pipe-c-ts-continuation-suspend:
- shard-tglb: [INCOMPLETE][127] ([i915#2828] / [i915#456]) -> [PASS][128]
[127]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-tglb7/igt@kms_vblank@pipe-c-ts-continuation-suspend.html
[128]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-tglb1/igt@kms_vblank@pipe-c-ts-continuation-suspend.html
#### Warnings ####
* igt@i915_pm_rc6_residency@rc6-idle:
- shard-iclb: [FAIL][129] ([i915#2680]) -> [WARN][130] ([i915#1804] / [i915#2684])
[129]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-iclb3/igt@i915_pm_rc6_residency@rc6-idle.html
[130]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-iclb6/igt@i915_pm_rc6_residency@rc6-idle.html
* igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-3:
- shard-iclb: [SKIP][131] ([i915#658]) -> [SKIP][132] ([i915#2920]) +1 similar issue
[131]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-iclb3/igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-3.html
[132]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-iclb2/igt@kms_psr2_sf@overlay-primary-update-sf-dmg-area-3.html
* igt@kms_psr2_su@page_flip:
- shard-iclb: [FAIL][133] ([i915#4148]) -> [SKIP][134] ([fdo#109642] / [fdo#111068] / [i915#658])
[133]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-iclb2/igt@kms_psr2_su@page_flip.html
[134]: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/shard-iclb8/igt@kms_psr2_su@page_flip.html
* igt@runner@aborted:
- shard-kbl: ([FAIL][135], [FAIL][136], [FAIL][137], [FAIL][138], [FAIL][139], [FAIL][140], [FAIL][141], [FAIL][142]) ([i915#180] / [i915#1814] / [i915#3363]) -> ([FAIL][143], [FAIL][144], [FAIL][145], [FAIL][146], [FAIL][147], [FAIL][148], [FAIL][149], [FAIL][150], [FAIL][151], [FAIL][152]) ([i915#1436] / [i915#180] / [i915#1814] / [i915#3002] / [i915#3363])
[135]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-kbl4/igt@runner@aborted.html
[136]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-kbl6/igt@runner@aborted.html
[137]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-kbl3/igt@runner@aborted.html
[138]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-kbl7/igt@runner@aborted.html
[139]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-kbl3/igt@runner@aborted.html
[140]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_10739/shard-kbl4/igt@runner@aborted.html
[141]: https://i
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/Patchwork_21340/index.html
[-- Attachment #2: Type: text/html, Size: 33474 bytes --]
^ permalink raw reply [flat|nested] 69+ messages in thread