All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2 0/9] TTM shrinker helpers and xe buffer object shrinker
@ 2024-04-16 10:07 Thomas Hellström
  2024-04-16 10:07 ` [PATCH v2 1/9] drm/ttm: Allow TTM LRU list nodes of different types Thomas Hellström
                   ` (17 more replies)
  0 siblings, 18 replies; 29+ messages in thread
From: Thomas Hellström @ 2024-04-16 10:07 UTC (permalink / raw)
  To: intel-xe
  Cc: Thomas Hellström, Somalapuram Amaranath,
	Christian König, dri-devel, Matthew Brost

This series implements TTM shrinker / eviction helpers and an xe bo
shrinker. It builds on two previous series, *and obsoletes these*. First

https://www.mail-archive.com/dri-devel@lists.freedesktop.org/msg484425.html

for patch 1-4, which IMO still could be reviewed and pushed as a
separate series.

Second the previous TTM shrinker series

https://lore.kernel.org/linux-mm/b7491378-defd-4f1c-31e2-29e4c77e2d67@amd.com/T/

Where the comment about layering
https://lore.kernel.org/linux-mm/b7491378-defd-4f1c-31e2-29e4c77e2d67@amd.com/T/#ma918844aa8a6efe8768fdcda0c6590d5c93850c9

now addressed, and this version also implements shmem objects for backup
rather than direct swap-cache insertions, which was used in the previuos
series. It turns out that with per-page backup / shrinking, shmem objects
appears to work just as well as direct swap-cache insertions with the
added benefit that was introduced in the previous TTM shrinker series to
avoid running out of swap entries isn't really needed.

In any case, patch 1-4 are better described in their separate series.
(RFC is removed for those).

Patch 5 could in theory be skipped but introduces a possibility to easily
add or test multiple backup backends, like the direct swap-cache
insertion or even files into fast dedicated nvme storage for for example.

Patch 6 introduces helpers in the ttm_pool code for page-by-page shrinking
and recovery. It avoids having to temporarily allocate a huge amount of
memory to be able to shrink a buffer object. It also introduces the
possibility to immediately write-back pages if needed, since that tends
to be a bit delayed when left to kswapd.

Patch 7 Adds a simple error injection to the above code to help increase
test coverage.

Patch 8 introduces a LRU walk helper for eviction and shrinking. It's
currently xe-only but not xe-specific and can easily be moved to TTM when
used by more than one driver or when eviction is implemented using it.

Patch 9 introduces a helper callback for shrinking (Also ready to be
moved to TTM) and an xe-specific shrinker implementation. It also
adds a kunit test to test the shrinker functionality by trying to
allocate twice the available amount of RAM as buffer objects. If there
is no swap-space available, the buffer objects are marked
purgeable.

v2:
- Squash obsolete revision history in the patch commit messages.
- Fix a couple of review comments by Christian
- Don't store the mem_type in the TTM managers but in the
  resource cursor.
- Rename introduced TTM *back_up* function names to *backup*
- Add ttm pool recovery fault injection.
- Shrinker xe kunit test
- Various bugfixes

Cc: Somalapuram Amaranath <Amaranath.Somalapuram@amd.com>
Cc: Christian König <christian.koenig@amd.com>
Cc: <dri-devel@lists.freedesktop.org>

Thomas Hellström (8):
  drm/ttm: Allow TTM LRU list nodes of different types
  drm/ttm: Use LRU hitches
  drm/ttm, drm/amdgpu, drm/xe: Consider hitch moves within bulk sublist
    moves
  drm/ttm: Allow continued swapout after -ENOSPC falure
  drm/ttm: Add a virtual base class for graphics memory backup
  drm/ttm/pool: Provide a helper to shrink pages.
  drm/xe, drm/ttm: Provide a generic LRU walker helper
  drm/xe: Add a shrinker for xe bos

 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c |   4 +
 drivers/gpu/drm/ttm/Makefile           |   2 +-
 drivers/gpu/drm/ttm/ttm_backup_shmem.c | 137 +++++++++
 drivers/gpu/drm/ttm/ttm_bo.c           |   1 +
 drivers/gpu/drm/ttm/ttm_device.c       |  33 ++-
 drivers/gpu/drm/ttm/ttm_pool.c         | 391 ++++++++++++++++++++++++-
 drivers/gpu/drm/ttm/ttm_resource.c     | 231 ++++++++++++---
 drivers/gpu/drm/ttm/ttm_tt.c           |  34 +++
 drivers/gpu/drm/xe/Makefile            |   2 +
 drivers/gpu/drm/xe/xe_bo.c             | 123 ++++++--
 drivers/gpu/drm/xe/xe_bo.h             |   3 +
 drivers/gpu/drm/xe/xe_device.c         |   8 +
 drivers/gpu/drm/xe/xe_device_types.h   |   2 +
 drivers/gpu/drm/xe/xe_shrinker.c       | 237 +++++++++++++++
 drivers/gpu/drm/xe/xe_shrinker.h       |  18 ++
 drivers/gpu/drm/xe/xe_ttm_helpers.c    | 224 ++++++++++++++
 drivers/gpu/drm/xe/xe_ttm_helpers.h    |  63 ++++
 drivers/gpu/drm/xe/xe_vm.c             |   4 +
 include/drm/ttm/ttm_backup.h           | 136 +++++++++
 include/drm/ttm/ttm_device.h           |   2 +
 include/drm/ttm/ttm_pool.h             |   4 +
 include/drm/ttm/ttm_resource.h         |  96 +++++-
 include/drm/ttm/ttm_tt.h               |  19 ++
 23 files changed, 1683 insertions(+), 91 deletions(-)
 create mode 100644 drivers/gpu/drm/ttm/ttm_backup_shmem.c
 create mode 100644 drivers/gpu/drm/xe/xe_shrinker.c
 create mode 100644 drivers/gpu/drm/xe/xe_shrinker.h
 create mode 100644 drivers/gpu/drm/xe/xe_ttm_helpers.c
 create mode 100644 drivers/gpu/drm/xe/xe_ttm_helpers.h
 create mode 100644 include/drm/ttm/ttm_backup.h

-- 
2.44.0



Thomas Hellström (9):
  drm/ttm: Allow TTM LRU list nodes of different types
  drm/ttm: Use LRU hitches
  drm/ttm, drm/amdgpu, drm/xe: Consider hitch moves within bulk sublist
    moves
  drm/ttm: Allow continued swapout after -ENOSPC falure
  drm/ttm: Add a virtual base class for graphics memory backup
  drm/ttm/pool: Provide a helper to shrink pages.
  drm/ttm: Use fault-injection to test error paths
  drm/xe, drm/ttm: Provide a generic LRU walker helper
  drm/xe: Add a shrinker for xe bos

 drivers/gpu/drm/Kconfig                |  10 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c |   4 +
 drivers/gpu/drm/ttm/Makefile           |   2 +-
 drivers/gpu/drm/ttm/ttm_backup_shmem.c | 137 ++++++++
 drivers/gpu/drm/ttm/ttm_bo.c           |   1 +
 drivers/gpu/drm/ttm/ttm_device.c       |  33 +-
 drivers/gpu/drm/ttm/ttm_pool.c         | 412 ++++++++++++++++++++++++-
 drivers/gpu/drm/ttm/ttm_resource.c     | 229 +++++++++++---
 drivers/gpu/drm/ttm/ttm_tt.c           |  37 +++
 drivers/gpu/drm/xe/Makefile            |   2 +
 drivers/gpu/drm/xe/tests/xe_bo.c       | 118 +++++++
 drivers/gpu/drm/xe/tests/xe_bo_test.c  |   1 +
 drivers/gpu/drm/xe/tests/xe_bo_test.h  |   1 +
 drivers/gpu/drm/xe/xe_bo.c             | 145 ++++++++-
 drivers/gpu/drm/xe/xe_bo.h             |   4 +
 drivers/gpu/drm/xe/xe_device.c         |   8 +
 drivers/gpu/drm/xe/xe_device_types.h   |   2 +
 drivers/gpu/drm/xe/xe_shrinker.c       | 226 ++++++++++++++
 drivers/gpu/drm/xe/xe_shrinker.h       |  18 ++
 drivers/gpu/drm/xe/xe_ttm_helpers.c    | 224 ++++++++++++++
 drivers/gpu/drm/xe/xe_ttm_helpers.h    |  50 +++
 drivers/gpu/drm/xe/xe_vm.c             |   4 +
 include/drm/ttm/ttm_backup.h           | 136 ++++++++
 include/drm/ttm/ttm_pool.h             |   5 +
 include/drm/ttm/ttm_resource.h         |  99 +++++-
 include/drm/ttm/ttm_tt.h               |  20 ++
 26 files changed, 1839 insertions(+), 89 deletions(-)
 create mode 100644 drivers/gpu/drm/ttm/ttm_backup_shmem.c
 create mode 100644 drivers/gpu/drm/xe/xe_shrinker.c
 create mode 100644 drivers/gpu/drm/xe/xe_shrinker.h
 create mode 100644 drivers/gpu/drm/xe/xe_ttm_helpers.c
 create mode 100644 drivers/gpu/drm/xe/xe_ttm_helpers.h
 create mode 100644 include/drm/ttm/ttm_backup.h

-- 
2.44.0


^ permalink raw reply	[flat|nested] 29+ messages in thread

end of thread, other threads:[~2024-05-02 11:41 UTC | newest]

Thread overview: 29+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-04-16 10:07 [PATCH v2 0/9] TTM shrinker helpers and xe buffer object shrinker Thomas Hellström
2024-04-16 10:07 ` [PATCH v2 1/9] drm/ttm: Allow TTM LRU list nodes of different types Thomas Hellström
2024-04-17  1:15   ` Matthew Brost
2024-04-17  6:09     ` Christian König
2024-05-02 11:41       ` Thomas Hellström
2024-04-16 10:07 ` [PATCH v2 2/9] drm/ttm: Use LRU hitches Thomas Hellström
2024-04-26  1:23   ` Matthew Brost
2024-04-16 10:07 ` [PATCH v2 3/9] drm/ttm, drm/amdgpu, drm/xe: Consider hitch moves within bulk sublist moves Thomas Hellström
2024-04-26  1:20   ` Matthew Brost
2024-04-16 10:07 ` [PATCH v2 4/9] drm/ttm: Allow continued swapout after -ENOSPC falure Thomas Hellström
2024-04-17  1:44   ` Matthew Brost
2024-04-16 10:07 ` [PATCH v2 5/9] drm/ttm: Add a virtual base class for graphics memory backup Thomas Hellström
2024-04-16 10:07 ` [PATCH v2 6/9] drm/ttm/pool: Provide a helper to shrink pages Thomas Hellström
2024-04-16 10:07 ` [PATCH v2 7/9] drm/ttm: Use fault-injection to test error paths Thomas Hellström
2024-04-16 10:07 ` [PATCH v2 8/9] drm/xe, drm/ttm: Provide a generic LRU walker helper Thomas Hellström
2024-04-16 10:07 ` [PATCH v2 9/9] drm/xe: Add a shrinker for xe bos Thomas Hellström
2024-04-16 21:32   ` kernel test robot
2024-04-17 13:45   ` kernel test robot
2024-04-16 10:33 ` ✓ CI.Patch_applied: success for TTM shrinker helpers and xe buffer object shrinker (rev2) Patchwork
2024-04-16 10:33 ` ✗ CI.checkpatch: warning " Patchwork
2024-04-16 10:35 ` ✓ CI.KUnit: success " Patchwork
2024-04-16 10:46 ` ✓ CI.Build: " Patchwork
2024-04-16 10:49 ` ✓ CI.Hooks: " Patchwork
2024-04-16 10:50 ` ✗ CI.checksparse: warning " Patchwork
2024-04-16 11:37 ` ✓ CI.BAT: success " Patchwork
2024-04-16 11:55 ` [PATCH v2 0/9] TTM shrinker helpers and xe buffer object shrinker Christian König
2024-04-16 13:08   ` Thomas Hellström
2024-04-16 13:24     ` Christian König
2024-04-17  8:37 ` ✗ CI.FULL: failure for TTM shrinker helpers and xe buffer object shrinker (rev2) Patchwork

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.