* [PATCH v15 02/17] arm64: untag user pointers in access_ok and __uaccess_mask_ptr
From: Andrey Konovalov @ 2019-05-06 16:30 UTC (permalink / raw)
To: linux-arm-kernel, linux-mm, linux-kernel, amd-gfx, dri-devel,
linux-rdma, linux-media, kvm, linux-kselftest
Cc: Catalin Marinas, Vincenzo Frascino, Will Deacon, Mark Rutland,
Andrew Morton, Greg Kroah-Hartman, Kees Cook, Yishai Hadas,
Felix Kuehling, Alexander Deucher, Christian Koenig,
Mauro Carvalho Chehab, Jens Wiklander, Alex Williamson,
Leon Romanovsky, Dmitry Vyukov, Kostya Serebryany,
Evgeniy Stepanov
In-Reply-To: <cover.1557160186.git.andreyknvl@google.com>
This patch is a part of a series that extends arm64 kernel ABI to allow to
pass tagged user pointers (with the top byte set to something else other
than 0x00) as syscall arguments.
copy_from_user (and a few other similar functions) are used to copy data
from user memory into the kernel memory or vice versa. Since a user can
provided a tagged pointer to one of the syscalls that use copy_from_user,
we need to correctly handle such pointers.
Do this by untagging user pointers in access_ok and in __uaccess_mask_ptr,
before performing access validity checks.
Note, that this patch only temporarily untags the pointers to perform the
checks, but then passes them as is into the kernel internals.
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
---
arch/arm64/include/asm/uaccess.h | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h
index e5d5f31c6d36..9164ecb5feca 100644
--- a/arch/arm64/include/asm/uaccess.h
+++ b/arch/arm64/include/asm/uaccess.h
@@ -94,7 +94,7 @@ static inline unsigned long __range_ok(const void __user *addr, unsigned long si
return ret;
}
-#define access_ok(addr, size) __range_ok(addr, size)
+#define access_ok(addr, size) __range_ok(untagged_addr(addr), size)
#define user_addr_max get_fs
#define _ASM_EXTABLE(from, to) \
@@ -226,7 +226,8 @@ static inline void uaccess_enable_not_uao(void)
/*
* Sanitise a uaccess pointer such that it becomes NULL if above the
- * current addr_limit.
+ * current addr_limit. In case the pointer is tagged (has the top byte set),
+ * untag the pointer before checking.
*/
#define uaccess_mask_ptr(ptr) (__typeof__(ptr))__uaccess_mask_ptr(ptr)
static inline void __user *__uaccess_mask_ptr(const void __user *ptr)
@@ -234,10 +235,11 @@ static inline void __user *__uaccess_mask_ptr(const void __user *ptr)
void __user *safe_ptr;
asm volatile(
- " bics xzr, %1, %2\n"
+ " bics xzr, %3, %2\n"
" csel %0, %1, xzr, eq\n"
: "=&r" (safe_ptr)
- : "r" (ptr), "r" (current_thread_info()->addr_limit)
+ : "r" (ptr), "r" (current_thread_info()->addr_limit),
+ "r" (untagged_addr(ptr))
: "cc");
csdb();
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related
* [PATCH v15 01/17] uaccess: add untagged_addr definition for other arches
From: Andrey Konovalov @ 2019-05-06 16:30 UTC (permalink / raw)
To: linux-arm-kernel, linux-mm, linux-kernel, amd-gfx, dri-devel,
linux-rdma, linux-media, kvm, linux-kselftest
Cc: Catalin Marinas, Vincenzo Frascino, Will Deacon, Mark Rutland,
Andrew Morton, Greg Kroah-Hartman, Kees Cook, Yishai Hadas,
Felix Kuehling, Alexander Deucher, Christian Koenig,
Mauro Carvalho Chehab, Jens Wiklander, Alex Williamson,
Leon Romanovsky, Dmitry Vyukov, Kostya Serebryany,
Evgeniy Stepanov
In-Reply-To: <cover.1557160186.git.andreyknvl@google.com>
To allow arm64 syscalls to accept tagged pointers from userspace, we must
untag them when they are passed to the kernel. Since untagging is done in
generic parts of the kernel, the untagged_addr macro needs to be defined
for all architectures.
Define it as a noop for architectures other than arm64.
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
---
include/linux/mm.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 6b10c21630f5..44041df804a6 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -99,6 +99,10 @@ extern int mmap_rnd_compat_bits __read_mostly;
#include <asm/pgtable.h>
#include <asm/processor.h>
+#ifndef untagged_addr
+#define untagged_addr(addr) (addr)
+#endif
+
#ifndef __pa_symbol
#define __pa_symbol(x) __pa(RELOC_HIDE((unsigned long)(x), 0))
#endif
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply related
* [PATCH v15 00/17] arm64: untag user pointers passed to the kernel
From: Andrey Konovalov @ 2019-05-06 16:30 UTC (permalink / raw)
To: linux-arm-kernel, linux-mm, linux-kernel, amd-gfx, dri-devel,
linux-rdma, linux-media, kvm, linux-kselftest
Cc: Catalin Marinas, Vincenzo Frascino, Will Deacon, Mark Rutland,
Andrew Morton, Greg Kroah-Hartman, Kees Cook, Yishai Hadas,
Felix Kuehling, Alexander Deucher, Christian Koenig,
Mauro Carvalho Chehab, Jens Wiklander, Alex Williamson,
Leon Romanovsky, Dmitry Vyukov, Kostya Serebryany,
Evgeniy Stepanov
=== Overview
arm64 has a feature called Top Byte Ignore, which allows to embed pointer
tags into the top byte of each pointer. Userspace programs (such as
HWASan, a memory debugging tool [1]) might use this feature and pass
tagged user pointers to the kernel through syscalls or other interfaces.
Right now the kernel is already able to handle user faults with tagged
pointers, due to these patches:
1. 81cddd65 ("arm64: traps: fix userspace cache maintenance emulation on a
tagged pointer")
2. 7dcd9dd8 ("arm64: hw_breakpoint: fix watchpoint matching for tagged
pointers")
3. 276e9327 ("arm64: entry: improve data abort handling of tagged
pointers")
This patchset extends tagged pointer support to syscall arguments.
As per the proposed ABI change [3], tagged pointers are only allowed to be
passed to syscalls when they point to memory ranges obtained by anonymous
mmap() or sbrk() (see the patchset [3] for more details).
For non-memory syscalls this is done by untaging user pointers when the
kernel performs pointer checking to find out whether the pointer comes
from userspace (most notably in access_ok). The untagging is done only
when the pointer is being checked, the tag is preserved as the pointer
makes its way through the kernel and stays tagged when the kernel
dereferences the pointer when perfoming user memory accesses.
Memory syscalls (mmap, mprotect, etc.) don't do user memory accesses but
rather deal with memory ranges, and untagged pointers are better suited to
describe memory ranges internally. Thus for memory syscalls we untag
pointers completely when they enter the kernel.
=== Other approaches
One of the alternative approaches to untagging that was considered is to
completely strip the pointer tag as the pointer enters the kernel with
some kind of a syscall wrapper, but that won't work with the countless
number of different ioctl calls. With this approach we would need a custom
wrapper for each ioctl variation, which doesn't seem practical.
An alternative approach to untagging pointers in memory syscalls prologues
is to inspead allow tagged pointers to be passed to find_vma() (and other
vma related functions) and untag them there. Unfortunately, a lot of
find_vma() callers then compare or subtract the returned vma start and end
fields against the pointer that was being searched. Thus this approach
would still require changing all find_vma() callers.
=== Testing
The following testing approaches has been taken to find potential issues
with user pointer untagging:
1. Static testing (with sparse [2] and separately with a custom static
analyzer based on Clang) to track casts of __user pointers to integer
types to find places where untagging needs to be done.
2. Static testing with grep to find parts of the kernel that call
find_vma() (and other similar functions) or directly compare against
vm_start/vm_end fields of vma.
3. Static testing with grep to find parts of the kernel that compare
user pointers with TASK_SIZE or other similar consts and macros.
4. Dynamic testing: adding BUG_ON(has_tag(addr)) to find_vma() and running
a modified syzkaller version that passes tagged pointers to the kernel.
Based on the results of the testing the requried patches have been added
to the patchset.
=== Notes
This patchset is meant to be merged together with "arm64 relaxed ABI" [3].
This patchset is a prerequisite for ARM's memory tagging hardware feature
support [4].
This patchset has been merged into the Pixel 2 & 3 kernel trees and is
now being used to enable testing of Pixel phones with HWASan.
Thanks!
[1] http://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html
[2] https://github.com/lucvoo/sparse-dev/commit/5f960cb10f56ec2017c128ef9d16060e0145f292
[3] https://lkml.org/lkml/2019/3/18/819
[4] https://community.arm.com/processors/b/blog/posts/arm-a-profile-architecture-2018-developments-armv85a
Changes in v15:
- Removed unnecessary untagging from radeon_ttm_tt_set_userptr().
- Removed unnecessary untagging from amdgpu_ttm_tt_set_userptr().
- Moved untagging to validate_range() in userfaultfd code.
- Moved untagging to ib_uverbs_(re)reg_mr() from mlx4_get_umem_mr().
- Rebased onto 5.1.
Changes in v14:
- Moved untagging for most memory syscalls to an arm64 specific
implementation, instead of doing that in the common code.
- Dropped "net, arm64: untag user pointers in tcp_zerocopy_receive", since
the provided user pointers don't come from an anonymous map and thus are
not covered by this ABI relaxation.
- Dropped "kernel, arm64: untag user pointers in prctl_set_mm*".
- Moved untagging from __check_mem_type() to tee_shm_register().
- Updated untagging for the amdgpu and radeon drivers to cover the MMU
notifier, as suggested by Felix.
- Since this ABI relaxation doesn't actually allow tagged instruction
pointers, dropped the following patches:
- Dropped "tracing, arm64: untag user pointers in seq_print_user_ip".
- Dropped "uprobes, arm64: untag user pointers in find_active_uprobe".
- Dropped "bpf, arm64: untag user pointers in stack_map_get_build_id_offset".
- Rebased onto 5.1-rc7 (37624b58).
Changes in v13:
- Simplified untagging in tcp_zerocopy_receive().
- Looked at find_vma() callers in drivers/, which allowed to identify a
few other places where untagging is needed.
- Added patch "mm, arm64: untag user pointers in get_vaddr_frames".
- Added patch "drm/amdgpu, arm64: untag user pointers in
amdgpu_ttm_tt_get_user_pages".
- Added patch "drm/radeon, arm64: untag user pointers in
radeon_ttm_tt_pin_userptr".
- Added patch "IB/mlx4, arm64: untag user pointers in mlx4_get_umem_mr".
- Added patch "media/v4l2-core, arm64: untag user pointers in
videobuf_dma_contig_user_get".
- Added patch "tee/optee, arm64: untag user pointers in check_mem_type".
- Added patch "vfio/type1, arm64: untag user pointers".
Changes in v12:
- Changed untagging in tcp_zerocopy_receive() to also untag zc->address.
- Fixed untagging in prctl_set_mm* to only untag pointers for vma lookups
and validity checks, but leave them as is for actual user space accesses.
- Updated the link to the v2 of the "arm64 relaxed ABI" patchset [3].
- Dropped the documentation patch, as the "arm64 relaxed ABI" patchset [3]
handles that.
Changes in v11:
- Added "uprobes, arm64: untag user pointers in find_active_uprobe" patch.
- Added "bpf, arm64: untag user pointers in stack_map_get_build_id_offset"
patch.
- Fixed "tracing, arm64: untag user pointers in seq_print_user_ip" to
correctly perform subtration with a tagged addr.
- Moved untagged_addr() from SYSCALL_DEFINE3(mprotect) and
SYSCALL_DEFINE4(pkey_mprotect) to do_mprotect_pkey().
- Moved untagged_addr() definition for other arches from
include/linux/memory.h to include/linux/mm.h.
- Changed untagging in strn*_user() to perform userspace accesses through
tagged pointers.
- Updated the documentation to mention that passing tagged pointers to
memory syscalls is allowed.
- Updated the test to use malloc'ed memory instead of stack memory.
Changes in v10:
- Added "mm, arm64: untag user pointers passed to memory syscalls" back.
- New patch "fs, arm64: untag user pointers in fs/userfaultfd.c".
- New patch "net, arm64: untag user pointers in tcp_zerocopy_receive".
- New patch "kernel, arm64: untag user pointers in prctl_set_mm*".
- New patch "tracing, arm64: untag user pointers in seq_print_user_ip".
Changes in v9:
- Rebased onto 4.20-rc6.
- Used u64 instead of __u64 in type casts in the untagged_addr macro for
arm64.
- Added braces around (addr) in the untagged_addr macro for other arches.
Changes in v8:
- Rebased onto 65102238 (4.20-rc1).
- Added a note to the cover letter on why syscall wrappers/shims that untag
user pointers won't work.
- Added a note to the cover letter that this patchset has been merged into
the Pixel 2 kernel tree.
- Documentation fixes, in particular added a list of syscalls that don't
support tagged user pointers.
Changes in v7:
- Rebased onto 17b57b18 (4.19-rc6).
- Dropped the "arm64: untag user address in __do_user_fault" patch, since
the existing patches already handle user faults properly.
- Dropped the "usb, arm64: untag user addresses in devio" patch, since the
passed pointer must come from a vma and therefore be untagged.
- Dropped the "arm64: annotate user pointers casts detected by sparse"
patch (see the discussion to the replies of the v6 of this patchset).
- Added more context to the cover letter.
- Updated Documentation/arm64/tagged-pointers.txt.
Changes in v6:
- Added annotations for user pointer casts found by sparse.
- Rebased onto 050cdc6c (4.19-rc1+).
Changes in v5:
- Added 3 new patches that add untagging to places found with static
analysis.
- Rebased onto 44c929e1 (4.18-rc8).
Changes in v4:
- Added a selftest for checking that passing tagged pointers to the
kernel succeeds.
- Rebased onto 81e97f013 (4.18-rc1+).
Changes in v3:
- Rebased onto e5c51f30 (4.17-rc6+).
- Added linux-arch@ to the list of recipients.
Changes in v2:
- Rebased onto 2d618bdf (4.17-rc3+).
- Removed excessive untagging in gup.c.
- Removed untagging pointers returned from __uaccess_mask_ptr.
Changes in v1:
- Rebased onto 4.17-rc1.
Changes in RFC v2:
- Added "#ifndef untagged_addr..." fallback in linux/uaccess.h instead of
defining it for each arch individually.
- Updated Documentation/arm64/tagged-pointers.txt.
- Dropped "mm, arm64: untag user addresses in memory syscalls".
- Rebased onto 3eb2ce82 (4.16-rc7).
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Andrey Konovalov (17):
uaccess: add untagged_addr definition for other arches
arm64: untag user pointers in access_ok and __uaccess_mask_ptr
lib, arm64: untag user pointers in strn*_user
mm: add ksys_ wrappers to memory syscalls
arms64: untag user pointers passed to memory syscalls
mm: untag user pointers in do_pages_move
mm, arm64: untag user pointers in mm/gup.c
mm, arm64: untag user pointers in get_vaddr_frames
fs, arm64: untag user pointers in copy_mount_options
fs, arm64: untag user pointers in fs/userfaultfd.c
drm/amdgpu, arm64: untag user pointers
drm/radeon, arm64: untag user pointers in radeon_gem_userptr_ioctl
IB, arm64: untag user pointers in ib_uverbs_(re)reg_mr()
media/v4l2-core, arm64: untag user pointers in
videobuf_dma_contig_user_get
tee, arm64: untag user pointers in tee_shm_register
vfio/type1, arm64: untag user pointers in vaddr_get_pfn
selftests, arm64: add a selftest for passing tagged pointers to kernel
arch/arm64/include/asm/uaccess.h | 10 +-
arch/arm64/kernel/sys.c | 128 ++++++++++++++++-
.../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 2 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 2 +
drivers/gpu/drm/radeon/radeon_gem.c | 2 +
drivers/infiniband/core/uverbs_cmd.c | 4 +
drivers/media/v4l2-core/videobuf-dma-contig.c | 9 +-
drivers/tee/tee_shm.c | 1 +
drivers/vfio/vfio_iommu_type1.c | 2 +
fs/namespace.c | 2 +-
fs/userfaultfd.c | 22 +--
include/linux/mm.h | 4 +
include/linux/syscalls.h | 22 +++
ipc/shm.c | 7 +-
lib/strncpy_from_user.c | 3 +-
lib/strnlen_user.c | 3 +-
mm/frame_vector.c | 2 +
mm/gup.c | 4 +
mm/madvise.c | 129 +++++++++---------
mm/mempolicy.c | 21 ++-
mm/migrate.c | 1 +
mm/mincore.c | 57 ++++----
mm/mlock.c | 20 ++-
mm/mmap.c | 30 +++-
mm/mprotect.c | 6 +-
mm/mremap.c | 27 ++--
mm/msync.c | 35 +++--
tools/testing/selftests/arm64/.gitignore | 1 +
tools/testing/selftests/arm64/Makefile | 11 ++
.../testing/selftests/arm64/run_tags_test.sh | 12 ++
tools/testing/selftests/arm64/tags_test.c | 21 +++
31 files changed, 436 insertions(+), 164 deletions(-)
create mode 100644 tools/testing/selftests/arm64/.gitignore
create mode 100644 tools/testing/selftests/arm64/Makefile
create mode 100755 tools/testing/selftests/arm64/run_tags_test.sh
create mode 100644 tools/testing/selftests/arm64/tags_test.c
--
2.21.0.1020.gf2820cf01a-goog
^ permalink raw reply
* Re: [PATCH v14 13/17] IB/mlx4, arm64: untag user pointers in mlx4_get_umem_mr
From: Andrey Konovalov @ 2019-05-06 16:21 UTC (permalink / raw)
To: Catalin Marinas
Cc: Linux ARM, Linux Memory Management List, LKML, amd-gfx, dri-devel,
linux-rdma, linux-media, kvm, open list:KERNEL SELFTEST FRAMEWORK,
Vincenzo Frascino, Will Deacon, Mark Rutland, Andrew Morton,
Greg Kroah-Hartman, Kees Cook, Yishai Hadas, Kuehling, Felix,
Deucher, Alexander, Koenig
In-Reply-To: <20190503170310.GL55449@arrakis.emea.arm.com>
On Fri, May 3, 2019 at 7:03 PM Catalin Marinas <catalin.marinas@arm.com> wrote:
>
> On Tue, Apr 30, 2019 at 03:25:09PM +0200, Andrey Konovalov wrote:
> > This patch is a part of a series that extends arm64 kernel ABI to allow to
> > pass tagged user pointers (with the top byte set to something else other
> > than 0x00) as syscall arguments.
> >
> > mlx4_get_umem_mr() uses provided user pointers for vma lookups, which can
> > only by done with untagged pointers.
> >
> > Untag user pointers in this function.
> >
> > Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> > Reviewed-by: Leon Romanovsky <leonro@mellanox.com>
> > ---
> > drivers/infiniband/hw/mlx4/mr.c | 7 ++++---
> > 1 file changed, 4 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/infiniband/hw/mlx4/mr.c b/drivers/infiniband/hw/mlx4/mr.c
> > index 395379a480cb..9a35ed2c6a6f 100644
> > --- a/drivers/infiniband/hw/mlx4/mr.c
> > +++ b/drivers/infiniband/hw/mlx4/mr.c
> > @@ -378,6 +378,7 @@ static struct ib_umem *mlx4_get_umem_mr(struct ib_udata *udata, u64 start,
> > * again
> > */
> > if (!ib_access_writable(access_flags)) {
> > + unsigned long untagged_start = untagged_addr(start);
> > struct vm_area_struct *vma;
> >
> > down_read(¤t->mm->mmap_sem);
> > @@ -386,9 +387,9 @@ static struct ib_umem *mlx4_get_umem_mr(struct ib_udata *udata, u64 start,
> > * cover the memory, but for now it requires a single vma to
> > * entirely cover the MR to support RO mappings.
> > */
> > - vma = find_vma(current->mm, start);
> > - if (vma && vma->vm_end >= start + length &&
> > - vma->vm_start <= start) {
> > + vma = find_vma(current->mm, untagged_start);
> > + if (vma && vma->vm_end >= untagged_start + length &&
> > + vma->vm_start <= untagged_start) {
> > if (vma->vm_flags & VM_WRITE)
> > access_flags |= IB_ACCESS_LOCAL_WRITE;
> > } else {
>
> Discussion ongoing on the previous version of the patch but I'm more
> inclined to do this in ib_uverbs_(re)reg_mr() on cmd.start.
OK, I want to publish v15 sooner to fix the issue with emails
addresses, so I'll implement this approach there for now.
>
> --
> Catalin
^ permalink raw reply
* Re: [PATCH v14 10/17] fs, arm64: untag user pointers in fs/userfaultfd.c
From: Andrey Konovalov @ 2019-05-06 14:15 UTC (permalink / raw)
To: Catalin Marinas
Cc: Linux ARM, Linux Memory Management List, LKML, amd-gfx, dri-devel,
linux-rdma, linux-media, kvm, open list:KERNEL SELFTEST FRAMEWORK,
Vincenzo Frascino, Will Deacon, Mark Rutland, Andrew Morton,
Greg Kroah-Hartman, Kees Cook, Yishai Hadas, Kuehling, Felix,
Deucher, Alexander, Koenig
In-Reply-To: <20190503165646.GK55449@arrakis.emea.arm.com>
On Fri, May 3, 2019 at 6:56 PM Catalin Marinas <catalin.marinas@arm.com> wrote:
>
> On Tue, Apr 30, 2019 at 03:25:06PM +0200, Andrey Konovalov wrote:
> > This patch is a part of a series that extends arm64 kernel ABI to allow to
> > pass tagged user pointers (with the top byte set to something else other
> > than 0x00) as syscall arguments.
> >
> > userfaultfd_register() and userfaultfd_unregister() use provided user
> > pointers for vma lookups, which can only by done with untagged pointers.
> >
> > Untag user pointers in these functions.
> >
> > Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> > ---
> > fs/userfaultfd.c | 5 +++++
> > 1 file changed, 5 insertions(+)
> >
> > diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
> > index f5de1e726356..fdee0db0e847 100644
> > --- a/fs/userfaultfd.c
> > +++ b/fs/userfaultfd.c
> > @@ -1325,6 +1325,9 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx,
> > goto out;
> > }
> >
> > + uffdio_register.range.start =
> > + untagged_addr(uffdio_register.range.start);
> > +
> > ret = validate_range(mm, uffdio_register.range.start,
> > uffdio_register.range.len);
> > if (ret)
> > @@ -1514,6 +1517,8 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx,
> > if (copy_from_user(&uffdio_unregister, buf, sizeof(uffdio_unregister)))
> > goto out;
> >
> > + uffdio_unregister.start = untagged_addr(uffdio_unregister.start);
> > +
> > ret = validate_range(mm, uffdio_unregister.start,
> > uffdio_unregister.len);
> > if (ret)
>
> Wouldn't it be easier to do this in validate_range()? There are a few
> more calls in this file, though I didn't check whether a tagged address
> would cause issues.
Yes, I think it makes more sense, will do in v15, thanks!
>
> --
> Catalin
^ permalink raw reply
* Re: [PATCH v14 08/17] mm, arm64: untag user pointers in get_vaddr_frames
From: Andrey Konovalov @ 2019-05-06 13:53 UTC (permalink / raw)
To: Catalin Marinas
Cc: Linux ARM, Linux Memory Management List, LKML, amd-gfx, dri-devel,
linux-rdma, linux-media, kvm, open list:KERNEL SELFTEST FRAMEWORK,
Vincenzo Frascino, Will Deacon, Mark Rutland, Andrew Morton,
Greg Kroah-Hartman, Kees Cook, Yishai Hadas, Kuehling, Felix,
Deucher, Alexander, Koenig
In-Reply-To: <20190503165113.GJ55449@arrakis.emea.arm.com>
On Fri, May 3, 2019 at 6:51 PM Catalin Marinas <catalin.marinas@arm.com> wrote:
>
> On Tue, Apr 30, 2019 at 03:25:04PM +0200, Andrey Konovalov wrote:
> > This patch is a part of a series that extends arm64 kernel ABI to allow to
> > pass tagged user pointers (with the top byte set to something else other
> > than 0x00) as syscall arguments.
> >
> > get_vaddr_frames uses provided user pointers for vma lookups, which can
> > only by done with untagged pointers. Instead of locating and changing
> > all callers of this function, perform untagging in it.
> >
> > Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> > ---
> > mm/frame_vector.c | 2 ++
> > 1 file changed, 2 insertions(+)
> >
> > diff --git a/mm/frame_vector.c b/mm/frame_vector.c
> > index c64dca6e27c2..c431ca81dad5 100644
> > --- a/mm/frame_vector.c
> > +++ b/mm/frame_vector.c
> > @@ -46,6 +46,8 @@ int get_vaddr_frames(unsigned long start, unsigned int nr_frames,
> > if (WARN_ON_ONCE(nr_frames > vec->nr_allocated))
> > nr_frames = vec->nr_allocated;
> >
> > + start = untagged_addr(start);
> > +
> > down_read(&mm->mmap_sem);
> > locked = 1;
> > vma = find_vma_intersection(mm, start, start + 1);
>
> Is this some buffer that the user may have malloc'ed? I got lost when
> trying to track down the provenience of this buffer.
The caller that I found when I was looking at this:
drivers/gpu/drm/exynos/exynos_drm_g2d.c:482
exynos_g2d_set_cmdlist_ioctl()->g2d_map_cmdlist_gem()->g2d_userptr_get_dma_addr()->get_vaddr_frames()
>
> --
> Catalin
^ permalink raw reply
* Re: [PATCH v14 11/17] drm/amdgpu, arm64: untag user pointers
From: Andrey Konovalov @ 2019-05-06 13:50 UTC (permalink / raw)
To: Kuehling, Felix
Cc: linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org,
linux-kernel@vger.kernel.org, amd-gfx@lists.freedesktop.org,
dri-devel@lists.freedesktop.org, linux-rdma@vger.kernel.org,
linux-media@vger.kernel.org, kvm@vger.kernel.org,
linux-kselftest@vger.kernel.org, Catalin Marinas,
Vincenzo Frascino, Will Deacon, Mark Rutland, Andrew Morton, Greg
In-Reply-To: <3108d33e-8e18-a73e-5e1a-f0db64f02ab3@amd.com>
On Tue, Apr 30, 2019 at 8:03 PM Kuehling, Felix <Felix.Kuehling@amd.com> wrote:
>
> On 2019-04-30 9:25 a.m., Andrey Konovalov wrote:
> > [CAUTION: External Email]
> >
> > This patch is a part of a series that extends arm64 kernel ABI to allow to
> > pass tagged user pointers (with the top byte set to something else other
> > than 0x00) as syscall arguments.
> >
> > amdgpu_ttm_tt_get_user_pages() uses provided user pointers for vma
> > lookups, which can only by done with untagged pointers. This patch
> > untag user pointers when they are being set in
> > amdgpu_ttm_tt_set_userptr().
> >
> > In amdgpu_gem_userptr_ioctl() and amdgpu_amdkfd_gpuvm.c/init_user_pages()
> > an MMU notifier is set up with a (tagged) userspace pointer. The untagged
> > address should be used so that MMU notifiers for the untagged address get
> > correctly matched up with the right BO. This patch untag user pointers in
> > amdgpu_gem_userptr_ioctl() for the GEM case and in
> > amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu() for the KFD case.
> >
> > Suggested-by: Kuehling, Felix <Felix.Kuehling@amd.com>
> > Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> > ---
> > drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 2 +-
> > drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 2 ++
> > drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 2 +-
> > 3 files changed, 4 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> > index 1921dec3df7a..20cac44ed449 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
> > @@ -1121,7 +1121,7 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
> > alloc_flags = 0;
> > if (!offset || !*offset)
> > return -EINVAL;
> > - user_addr = *offset;
> > + user_addr = untagged_addr(*offset);
> > } else if (flags & ALLOC_MEM_FLAGS_DOORBELL) {
> > domain = AMDGPU_GEM_DOMAIN_GTT;
> > alloc_domain = AMDGPU_GEM_DOMAIN_CPU;
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> > index d21dd2f369da..985cb82b2aa6 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> > @@ -286,6 +286,8 @@ int amdgpu_gem_userptr_ioctl(struct drm_device *dev, void *data,
> > uint32_t handle;
> > int r;
> >
> > + args->addr = untagged_addr(args->addr);
> > +
> > if (offset_in_page(args->addr | args->size))
> > return -EINVAL;
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> > index 73e71e61dc99..1d30e97ac2c4 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> > @@ -1248,7 +1248,7 @@ int amdgpu_ttm_tt_set_userptr(struct ttm_tt *ttm, uint64_t addr,
> > if (gtt == NULL)
> > return -EINVAL;
> >
> > - gtt->userptr = addr;
> > + gtt->userptr = untagged_addr(addr);
>
> Doing this here seems unnecessary. You already untagged the address in
> both callers of this function. Untagging in the two callers ensures that
> the userptr and MMU notifier are in sync, using the same untagged
> address. Doing it again here is redundant.
Will fix in v15, thanks!
>
> Regards,
> Felix
>
>
> > gtt->userflags = flags;
> >
> > if (gtt->usertask)
> > --
> > 2.21.0.593.g511ec345e18-goog
> >
^ permalink raw reply
* Re: [PATCH v14 12/17] drm/radeon, arm64: untag user pointers
From: Andrey Konovalov @ 2019-05-06 13:44 UTC (permalink / raw)
To: Kuehling, Felix
Cc: Mark Rutland, Koenig@google.com, kvm@vger.kernel.org,
Szabolcs Nagy, Catalin Marinas, Will Deacon,
dri-devel@lists.freedesktop.org, linux-mm@kvack.org,
linux-kselftest@vger.kernel.org, Chintan Pandya,
Vincenzo Frascino, Jacob Bramley, Leon Romanovsky,
linux-rdma@vger.kernel.org, amd-gfx@lists.freedesktop.org,
linux-arm-kernel@lists.infradead.org, Dave Martin,
Evgeniy Stepanov, linux-media
In-Reply-To: <bfe5e11e-6dc4-352f-57eb-d527f965a2ef@amd.com>
On Tue, Apr 30, 2019 at 7:57 PM Kuehling, Felix <Felix.Kuehling@amd.com> wrote:
>
> On 2019-04-30 9:25 a.m., Andrey Konovalov wrote:
> > [CAUTION: External Email]
> >
> > This patch is a part of a series that extends arm64 kernel ABI to allow to
> > pass tagged user pointers (with the top byte set to something else other
> > than 0x00) as syscall arguments.
> >
> > radeon_ttm_tt_pin_userptr() uses provided user pointers for vma
> > lookups, which can only by done with untagged pointers. This patch
> > untags user pointers when they are being set in
> > radeon_ttm_tt_pin_userptr().
> >
> > In amdgpu_gem_userptr_ioctl() an MMU notifier is set up with a (tagged)
> > userspace pointer. The untagged address should be used so that MMU
> > notifiers for the untagged address get correctly matched up with the right
> > BO. This patch untags user pointers in radeon_gem_userptr_ioctl().
> >
> > Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> > ---
> > drivers/gpu/drm/radeon/radeon_gem.c | 2 ++
> > drivers/gpu/drm/radeon/radeon_ttm.c | 2 +-
> > 2 files changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/gpu/drm/radeon/radeon_gem.c b/drivers/gpu/drm/radeon/radeon_gem.c
> > index 44617dec8183..90eb78fb5eb2 100644
> > --- a/drivers/gpu/drm/radeon/radeon_gem.c
> > +++ b/drivers/gpu/drm/radeon/radeon_gem.c
> > @@ -291,6 +291,8 @@ int radeon_gem_userptr_ioctl(struct drm_device *dev, void *data,
> > uint32_t handle;
> > int r;
> >
> > + args->addr = untagged_addr(args->addr);
> > +
> > if (offset_in_page(args->addr | args->size))
> > return -EINVAL;
> >
> > diff --git a/drivers/gpu/drm/radeon/radeon_ttm.c b/drivers/gpu/drm/radeon/radeon_ttm.c
> > index 9920a6fc11bf..dce722c494c1 100644
> > --- a/drivers/gpu/drm/radeon/radeon_ttm.c
> > +++ b/drivers/gpu/drm/radeon/radeon_ttm.c
> > @@ -742,7 +742,7 @@ int radeon_ttm_tt_set_userptr(struct ttm_tt *ttm, uint64_t addr,
> > if (gtt == NULL)
> > return -EINVAL;
> >
> > - gtt->userptr = addr;
> > + gtt->userptr = untagged_addr(addr);
>
> Doing this here seems unnecessary, because you already untagged the
> address in the only caller of this function in radeon_gem_userptr_ioctl.
> The change there will affect both the userptr and MMU notifier setup and
> makes sure that both are in sync, using the same untagged address.
Will fix in v15, thanks!
>
> Regards,
> Felix
>
>
> > gtt->usermm = current->mm;
> > gtt->userflags = flags;
> > return 0;
> > --
> > 2.21.0.593.g511ec345e18-goog
> >
^ permalink raw reply
* Re: [PATCH v3 00/26] compat_ioctl: cleanups
From: Andy Shevchenko @ 2019-05-06 9:03 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Linux NVMe Mailinglist, linux-iio, linux-remoteproc, linux-fbdev,
dri-devel, Platform Driver, linux-ide,
open list:MEMORY TECHNOLOGY..., sparclinux, linux1394-devel,
devel, linux-s390, linux-scsi, linux-bluetooth, y2038, qat-linux,
amd-gfx, linux-input, Marcel Holtmann, Linux Media Mailing List,
open list:REAL TIME CLOCK (RTC) SUBSYSTEM,
ALSA Development Mailing List
In-Reply-To: <20190416202013.4034148-1-arnd@arndb.de>
On Tue, Apr 16, 2019 at 11:23 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> Hi Al,
>
> It took me way longer than I had hoped to revisit this series, see
> https://lore.kernel.org/lkml/20180912150142.157913-1-arnd@arndb.de/
> for the previously posted version.
>
> I've come to the point where all conversion handlers and most
> COMPATIBLE_IOCTL() entries are gone from this file, but for
> now, this series only has the parts that have either been reviewed
> previously, or that are simple enough to include.
>
> The main missing piece is the SG_IO/SG_GET_REQUEST_TABLE conversion.
> I'll post the patches I made for that later, as they need more
> testing and review from the scsi maintainers.
>
> I hope you can still take these for the coming merge window, unless
> new problems come up.
> drivers/platform/x86/wmi.c | 2 +-
Acked-by: Andy Shevchenko <andy.shevchenko@gmail.com>
--
With Best Regards,
Andy Shevchenko
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply
* Re: [PATCH] net: rds: fix spelling mistake "syctl" -> "sysctl"
From: David Miller @ 2019-05-05 17:19 UTC (permalink / raw)
To: colin.king
Cc: santosh.shilimkar, netdev, linux-rdma, rds-devel, kernel-janitors,
linux-kernel
In-Reply-To: <20190503121017.5227-1-colin.king@canonical.com>
From: Colin King <colin.king@canonical.com>
Date: Fri, 3 May 2019 13:10:17 +0100
> From: Colin Ian King <colin.king@canonical.com>
>
> There is a spelling mistake in a pr_warn warning. Fix it.
>
> Signed-off-by: Colin Ian King <colin.king@canonical.com>
Applied.
^ permalink raw reply
* Re: [RFC PATCH 0/5] cgroup support for GPU devices
From: Leon Romanovsky @ 2019-05-05 16:55 UTC (permalink / raw)
To: Kenny Ho
Cc: Parav Pandit, David Airlie, kenny.ho, intel-gfx, Welty, Brian,
cgroups, dri-devel, Michal Hocko, Johannes Weiner, linux-mm,
J??r??me Glisse, Li Zefan, Vladimir Davydov, Alex Deucher,
Tejun Heo, Christian K??nig, RDMA mailing list
In-Reply-To: <CAOWid-cCq+yB9m-u8YpHFuhUZ+C7EpbT2OD27iszJVrruAtqKg@mail.gmail.com>
On Sun, May 05, 2019 at 12:34:16PM -0400, Kenny Ho wrote:
> (sent again. Not sure why my previous email was just a reply instead
> of reply-all.)
>
> On Sun, May 5, 2019 at 12:05 PM Leon Romanovsky <leon@kernel.org> wrote:
> > We are talking about two different access patterns for this device
> > memory (DM). One is to use this device memory (DM) and second to configure/limit.
> > Usually those actions will be performed by different groups.
> >
> > First group (programmers) is using special API [1] through libibverbs [2]
> > without any notion of cgroups or any limitations. Second group (sysadmins)
> > is less interested in application specifics and for them "device memory" means
> > "memory" and not "rdma, nic specific, internal memory".
> Um... I am not sure that answered it, especially in the context of
> cgroup (this is just for my curiosity btw, I don't know much about
> rdma.) You said sysadmins are less interested in application
> specifics but then how would they make the judgement call on how much
> "device memory" is provisioned to one application/container over
> another (let say you have 5 cgroup sharing an rdma device)? What are
> the consequences of under provisioning "device memory" to an
> application? And if they are all just memory, can a sysadmin
> provision more system memory in place of device memory (like, are they
> interchangeable)? I guess I am confused because if device memory is
> just memory (not rdma, nic specific) to sysadmins how would they know
> to set the right amount?
One of the immediate usages of this DM that come to my mind is very
fast spinlocks for MPI applications. In such case, the amount of DM
will be property of network topology in given MPI cluster.
In this scenario, precise amount of memory will ensure that all jobs
will continue to give maximal performance despite any programmer's
error in DM allocation.
For under provisioning scenario and if application is written correctly,
users will experience more latency and less performance, due to the PCI
accesses.
Slide 3 in Liran's presentation gives brief overview about motivation.
Thanks
>
> Regards,
> Kenny
>
> > [1] ibv_alloc_dm()
> > http://man7.org/linux/man-pages/man3/ibv_alloc_dm.3.html
> > https://www.openfabrics.org/images/2018workshop/presentations/304_LLiss_OnDeviceMemory.pdf
> > [2] https://github.com/linux-rdma/rdma-core/blob/master/libibverbs/
> >
> > >
> > > I think we need to be careful about drawing the line between
> > > duplication and over couplings between subsystems. I have other
> > > thoughts and concerns and I will try to organize them into a response
> > > in the next few days.
> > >
> > > Regards,
> > > Kenny
> > >
> > >
> > > > >
> > > > > Is AMD interested in collaborating to help shape this framework?
> > > > > It is intended to be device-neutral, so could be leveraged by various
> > > > > types of devices.
> > > > > If you have an alternative solution well underway, then maybe
> > > > > we can work together to merge our efforts into one.
> > > > > In the end, the DRM community is best served with common solution.
> > > > >
> > > > >
> > > > > >
> > > > > >>> and with future work, we could extend to:
> > > > > >>> * track and control share of GPU time (reuse of cpu/cpuacct)
> > > > > >>> * apply mask of allowed execution engines (reuse of cpusets)
> > > > > >>>
> > > > > >>> Instead of introducing a new cgroup subsystem for GPU devices, a new
> > > > > >>> framework is proposed to allow devices to register with existing cgroup
> > > > > >>> controllers, which creates per-device cgroup_subsys_state within the
> > > > > >>> cgroup. This gives device drivers their own private cgroup controls
> > > > > >>> (such as memory limits or other parameters) to be applied to device
> > > > > >>> resources instead of host system resources.
> > > > > >>> Device drivers (GPU or other) are then able to reuse the existing cgroup
> > > > > >>> controls, instead of inventing similar ones.
> > > > > >>>
> > > > > >>> Per-device controls would be exposed in cgroup filesystem as:
> > > > > >>> mount/<cgroup_name>/<subsys_name>.devices/<dev_name>/<subsys_files>
> > > > > >>> such as (for example):
> > > > > >>> mount/<cgroup_name>/memory.devices/<dev_name>/memory.max
> > > > > >>> mount/<cgroup_name>/memory.devices/<dev_name>/memory.current
> > > > > >>> mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.stat
> > > > > >>> mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.weight
> > > > > >>>
> > > > > >>> The drm/i915 patch in this series is based on top of other RFC work [1]
> > > > > >>> for i915 device memory support.
> > > > > >>>
> > > > > >>> AMD [2] and Intel [3] have proposed related work in this area within the
> > > > > >>> last few years, listed below as reference. This new RFC reuses existing
> > > > > >>> cgroup controllers and takes a different approach than prior work.
> > > > > >>>
> > > > > >>> Finally, some potential discussion points for this series:
> > > > > >>> * merge proposed <subsys_name>.devices into a single devices directory?
> > > > > >>> * allow devices to have multiple registrations for subsets of resources?
> > > > > >>> * document a 'common charging policy' for device drivers to follow?
> > > > > >>>
> > > > > >>> [1] https://patchwork.freedesktop.org/series/56683/
> > > > > >>> [2] https://lists.freedesktop.org/archives/dri-devel/2018-November/197106.html
> > > > > >>> [3] https://lists.freedesktop.org/archives/intel-gfx/2018-January/153156.html
> > > > > >>>
> > > > > >>>
> > > > > >>> Brian Welty (5):
> > > > > >>> cgroup: Add cgroup_subsys per-device registration framework
> > > > > >>> cgroup: Change kernfs_node for directories to store
> > > > > >>> cgroup_subsys_state
> > > > > >>> memcg: Add per-device support to memory cgroup subsystem
> > > > > >>> drm: Add memory cgroup registration and DRIVER_CGROUPS feature bit
> > > > > >>> drm/i915: Use memory cgroup for enforcing device memory limit
> > > > > >>>
> > > > > >>> drivers/gpu/drm/drm_drv.c | 12 +
> > > > > >>> drivers/gpu/drm/drm_gem.c | 7 +
> > > > > >>> drivers/gpu/drm/i915/i915_drv.c | 2 +-
> > > > > >>> drivers/gpu/drm/i915/intel_memory_region.c | 24 +-
> > > > > >>> include/drm/drm_device.h | 3 +
> > > > > >>> include/drm/drm_drv.h | 8 +
> > > > > >>> include/drm/drm_gem.h | 11 +
> > > > > >>> include/linux/cgroup-defs.h | 28 ++
> > > > > >>> include/linux/cgroup.h | 3 +
> > > > > >>> include/linux/memcontrol.h | 10 +
> > > > > >>> kernel/cgroup/cgroup-v1.c | 10 +-
> > > > > >>> kernel/cgroup/cgroup.c | 310 ++++++++++++++++++---
> > > > > >>> mm/memcontrol.c | 183 +++++++++++-
> > > > > >>> 13 files changed, 552 insertions(+), 59 deletions(-)
> > > > > >>>
> > > > > >>> --
> > > > > >>> 2.21.0
> > > > > >>>
> > > > > >> _______________________________________________
> > > > > >> dri-devel mailing list
> > > > > >> dri-devel@lists.freedesktop.org
> > > > > >> https://lists.freedesktop.org/mailman/listinfo/dri-devel
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply
* Re: [RFC PATCH 0/5] cgroup support for GPU devices
From: Chris Down @ 2019-05-05 16:46 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Parav Pandit, David Airlie, kenny.ho, intel-gfx, Welty, Brian,
Harish.Kasiviswanathan, cgroups, dri-devel, Michal Hocko,
Johannes Weiner, linux-mm, J??r??me Glisse, Li Zefan, Kenny Ho,
Vladimir Davydov, Rodrigo Vivi, Alex Deucher, Tejun Heo,
Christian K??nig, RDMA mailing list
In-Reply-To: <20190505160506.GF6938@mtr-leonro.mtl.com>
Leon Romanovsky writes:
>First group (programmers) is using special API [1] through libibverbs [2]
>without any notion of cgroups or any limitations. Second group (sysadmins)
>is less interested in application specifics and for them "device memory" means
>"memory" and not "rdma, nic specific, internal memory".
I'd suggest otherwise, based on historic precedent -- sysadmins are typically
very opinionated about operation of the memory subsystem (hence the endless
discussions about swap, caching behaviour, etc).
Especially in this case, these types of memory operate fundamentally
differently and have significantly different performance and availability
characteristics. That's not something that can be trivially abstracted over
without non-trivial drawbacks.
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply
* Re: [RFC PATCH 0/5] cgroup support for GPU devices
From: Kenny Ho @ 2019-05-05 16:34 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Parav Pandit, David Airlie, kenny.ho, intel-gfx, Welty, Brian,
Harish.Kasiviswanathan, cgroups, dri-devel, Michal Hocko,
Johannes Weiner, linux-mm, J??r??me Glisse, Li Zefan,
Vladimir Davydov, Rodrigo Vivi, Alex Deucher, Tejun Heo,
Christian K??nig, RDMA mailing list
In-Reply-To: <20190505160506.GF6938@mtr-leonro.mtl.com>
(sent again. Not sure why my previous email was just a reply instead
of reply-all.)
On Sun, May 5, 2019 at 12:05 PM Leon Romanovsky <leon@kernel.org> wrote:
> We are talking about two different access patterns for this device
> memory (DM). One is to use this device memory (DM) and second to configure/limit.
> Usually those actions will be performed by different groups.
>
> First group (programmers) is using special API [1] through libibverbs [2]
> without any notion of cgroups or any limitations. Second group (sysadmins)
> is less interested in application specifics and for them "device memory" means
> "memory" and not "rdma, nic specific, internal memory".
Um... I am not sure that answered it, especially in the context of
cgroup (this is just for my curiosity btw, I don't know much about
rdma.) You said sysadmins are less interested in application
specifics but then how would they make the judgement call on how much
"device memory" is provisioned to one application/container over
another (let say you have 5 cgroup sharing an rdma device)? What are
the consequences of under provisioning "device memory" to an
application? And if they are all just memory, can a sysadmin
provision more system memory in place of device memory (like, are they
interchangeable)? I guess I am confused because if device memory is
just memory (not rdma, nic specific) to sysadmins how would they know
to set the right amount?
Regards,
Kenny
> [1] ibv_alloc_dm()
> http://man7.org/linux/man-pages/man3/ibv_alloc_dm.3.html
> https://www.openfabrics.org/images/2018workshop/presentations/304_LLiss_OnDeviceMemory.pdf
> [2] https://github.com/linux-rdma/rdma-core/blob/master/libibverbs/
>
> >
> > I think we need to be careful about drawing the line between
> > duplication and over couplings between subsystems. I have other
> > thoughts and concerns and I will try to organize them into a response
> > in the next few days.
> >
> > Regards,
> > Kenny
> >
> >
> > > >
> > > > Is AMD interested in collaborating to help shape this framework?
> > > > It is intended to be device-neutral, so could be leveraged by various
> > > > types of devices.
> > > > If you have an alternative solution well underway, then maybe
> > > > we can work together to merge our efforts into one.
> > > > In the end, the DRM community is best served with common solution.
> > > >
> > > >
> > > > >
> > > > >>> and with future work, we could extend to:
> > > > >>> * track and control share of GPU time (reuse of cpu/cpuacct)
> > > > >>> * apply mask of allowed execution engines (reuse of cpusets)
> > > > >>>
> > > > >>> Instead of introducing a new cgroup subsystem for GPU devices, a new
> > > > >>> framework is proposed to allow devices to register with existing cgroup
> > > > >>> controllers, which creates per-device cgroup_subsys_state within the
> > > > >>> cgroup. This gives device drivers their own private cgroup controls
> > > > >>> (such as memory limits or other parameters) to be applied to device
> > > > >>> resources instead of host system resources.
> > > > >>> Device drivers (GPU or other) are then able to reuse the existing cgroup
> > > > >>> controls, instead of inventing similar ones.
> > > > >>>
> > > > >>> Per-device controls would be exposed in cgroup filesystem as:
> > > > >>> mount/<cgroup_name>/<subsys_name>.devices/<dev_name>/<subsys_files>
> > > > >>> such as (for example):
> > > > >>> mount/<cgroup_name>/memory.devices/<dev_name>/memory.max
> > > > >>> mount/<cgroup_name>/memory.devices/<dev_name>/memory.current
> > > > >>> mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.stat
> > > > >>> mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.weight
> > > > >>>
> > > > >>> The drm/i915 patch in this series is based on top of other RFC work [1]
> > > > >>> for i915 device memory support.
> > > > >>>
> > > > >>> AMD [2] and Intel [3] have proposed related work in this area within the
> > > > >>> last few years, listed below as reference. This new RFC reuses existing
> > > > >>> cgroup controllers and takes a different approach than prior work.
> > > > >>>
> > > > >>> Finally, some potential discussion points for this series:
> > > > >>> * merge proposed <subsys_name>.devices into a single devices directory?
> > > > >>> * allow devices to have multiple registrations for subsets of resources?
> > > > >>> * document a 'common charging policy' for device drivers to follow?
> > > > >>>
> > > > >>> [1] https://patchwork.freedesktop.org/series/56683/
> > > > >>> [2] https://lists.freedesktop.org/archives/dri-devel/2018-November/197106.html
> > > > >>> [3] https://lists.freedesktop.org/archives/intel-gfx/2018-January/153156.html
> > > > >>>
> > > > >>>
> > > > >>> Brian Welty (5):
> > > > >>> cgroup: Add cgroup_subsys per-device registration framework
> > > > >>> cgroup: Change kernfs_node for directories to store
> > > > >>> cgroup_subsys_state
> > > > >>> memcg: Add per-device support to memory cgroup subsystem
> > > > >>> drm: Add memory cgroup registration and DRIVER_CGROUPS feature bit
> > > > >>> drm/i915: Use memory cgroup for enforcing device memory limit
> > > > >>>
> > > > >>> drivers/gpu/drm/drm_drv.c | 12 +
> > > > >>> drivers/gpu/drm/drm_gem.c | 7 +
> > > > >>> drivers/gpu/drm/i915/i915_drv.c | 2 +-
> > > > >>> drivers/gpu/drm/i915/intel_memory_region.c | 24 +-
> > > > >>> include/drm/drm_device.h | 3 +
> > > > >>> include/drm/drm_drv.h | 8 +
> > > > >>> include/drm/drm_gem.h | 11 +
> > > > >>> include/linux/cgroup-defs.h | 28 ++
> > > > >>> include/linux/cgroup.h | 3 +
> > > > >>> include/linux/memcontrol.h | 10 +
> > > > >>> kernel/cgroup/cgroup-v1.c | 10 +-
> > > > >>> kernel/cgroup/cgroup.c | 310 ++++++++++++++++++---
> > > > >>> mm/memcontrol.c | 183 +++++++++++-
> > > > >>> 13 files changed, 552 insertions(+), 59 deletions(-)
> > > > >>>
> > > > >>> --
> > > > >>> 2.21.0
> > > > >>>
> > > > >> _______________________________________________
> > > > >> dri-devel mailing list
> > > > >> dri-devel@lists.freedesktop.org
> > > > >> https://lists.freedesktop.org/mailman/listinfo/dri-devel
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply
* Re: [RFC PATCH 0/5] cgroup support for GPU devices
From: Leon Romanovsky @ 2019-05-05 16:05 UTC (permalink / raw)
To: Kenny Ho
Cc: Parav Pandit, David Airlie, kenny.ho, intel-gfx, Welty, Brian,
Harish.Kasiviswanathan, cgroups, dri-devel, Michal Hocko,
Johannes Weiner, linux-mm, J??r??me Glisse, Li Zefan,
Vladimir Davydov, Rodrigo Vivi, Alex Deucher, Tejun Heo,
Christian K??nig, RDMA mailing list
In-Reply-To: <CAOWid-di8kcC2bYKq1KJo+rWfVjwQ13mcVRjaBjhFRzTO=c16Q@mail.gmail.com>
On Sun, May 05, 2019 at 10:21:30AM -0400, Kenny Ho wrote:
> On Sun, May 5, 2019 at 3:14 AM Leon Romanovsky <leon@kernel.org> wrote:
> > > > Doesn't RDMA already has a separate cgroup? Why not implement it there?
> > > >
> > >
> > > Hi Kenny, I can't answer for Leon, but I'm hopeful he agrees with rationale
> > > I gave in the cover letter. Namely, to implement in rdma controller, would
> > > mean duplicating existing memcg controls there.
> >
> > Exactly, I didn't feel comfortable to add notion of "device memory"
> > to RDMA cgroup and postponed that decision to later point of time.
> > RDMA operates with verbs objects and all our user space API is based around
> > that concept. At the end, system administrator will have hard time to
> > understand the differences between memcg and RDMA memory.
> Interesting. I actually don't understand this part (I worked in
> devops/sysadmin side of things but never with rdma.) Don't
> applications that use rdma require some awareness of rdma (I mean, you
> mentioned verbs and objects... or do they just use regular malloc for
> buffer allocation and then send it through some function?) As a user,
> I would have this question: why do I need to configure some part of
> rdma resources under rdma cgroup while other part of rdma resources in
> a different, seemingly unrelated cgroups.
We are talking about two different access patterns for this device
memory (DM). One is to use this device memory (DM) and second to configure/limit.
Usually those actions will be performed by different groups.
First group (programmers) is using special API [1] through libibverbs [2]
without any notion of cgroups or any limitations. Second group (sysadmins)
is less interested in application specifics and for them "device memory" means
"memory" and not "rdma, nic specific, internal memory".
[1] ibv_alloc_dm()
http://man7.org/linux/man-pages/man3/ibv_alloc_dm.3.html
https://www.openfabrics.org/images/2018workshop/presentations/304_LLiss_OnDeviceMemory.pdf
[2] https://github.com/linux-rdma/rdma-core/blob/master/libibverbs/
>
> I think we need to be careful about drawing the line between
> duplication and over couplings between subsystems. I have other
> thoughts and concerns and I will try to organize them into a response
> in the next few days.
>
> Regards,
> Kenny
>
>
> > >
> > > Is AMD interested in collaborating to help shape this framework?
> > > It is intended to be device-neutral, so could be leveraged by various
> > > types of devices.
> > > If you have an alternative solution well underway, then maybe
> > > we can work together to merge our efforts into one.
> > > In the end, the DRM community is best served with common solution.
> > >
> > >
> > > >
> > > >>> and with future work, we could extend to:
> > > >>> * track and control share of GPU time (reuse of cpu/cpuacct)
> > > >>> * apply mask of allowed execution engines (reuse of cpusets)
> > > >>>
> > > >>> Instead of introducing a new cgroup subsystem for GPU devices, a new
> > > >>> framework is proposed to allow devices to register with existing cgroup
> > > >>> controllers, which creates per-device cgroup_subsys_state within the
> > > >>> cgroup. This gives device drivers their own private cgroup controls
> > > >>> (such as memory limits or other parameters) to be applied to device
> > > >>> resources instead of host system resources.
> > > >>> Device drivers (GPU or other) are then able to reuse the existing cgroup
> > > >>> controls, instead of inventing similar ones.
> > > >>>
> > > >>> Per-device controls would be exposed in cgroup filesystem as:
> > > >>> mount/<cgroup_name>/<subsys_name>.devices/<dev_name>/<subsys_files>
> > > >>> such as (for example):
> > > >>> mount/<cgroup_name>/memory.devices/<dev_name>/memory.max
> > > >>> mount/<cgroup_name>/memory.devices/<dev_name>/memory.current
> > > >>> mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.stat
> > > >>> mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.weight
> > > >>>
> > > >>> The drm/i915 patch in this series is based on top of other RFC work [1]
> > > >>> for i915 device memory support.
> > > >>>
> > > >>> AMD [2] and Intel [3] have proposed related work in this area within the
> > > >>> last few years, listed below as reference. This new RFC reuses existing
> > > >>> cgroup controllers and takes a different approach than prior work.
> > > >>>
> > > >>> Finally, some potential discussion points for this series:
> > > >>> * merge proposed <subsys_name>.devices into a single devices directory?
> > > >>> * allow devices to have multiple registrations for subsets of resources?
> > > >>> * document a 'common charging policy' for device drivers to follow?
> > > >>>
> > > >>> [1] https://patchwork.freedesktop.org/series/56683/
> > > >>> [2] https://lists.freedesktop.org/archives/dri-devel/2018-November/197106.html
> > > >>> [3] https://lists.freedesktop.org/archives/intel-gfx/2018-January/153156.html
> > > >>>
> > > >>>
> > > >>> Brian Welty (5):
> > > >>> cgroup: Add cgroup_subsys per-device registration framework
> > > >>> cgroup: Change kernfs_node for directories to store
> > > >>> cgroup_subsys_state
> > > >>> memcg: Add per-device support to memory cgroup subsystem
> > > >>> drm: Add memory cgroup registration and DRIVER_CGROUPS feature bit
> > > >>> drm/i915: Use memory cgroup for enforcing device memory limit
> > > >>>
> > > >>> drivers/gpu/drm/drm_drv.c | 12 +
> > > >>> drivers/gpu/drm/drm_gem.c | 7 +
> > > >>> drivers/gpu/drm/i915/i915_drv.c | 2 +-
> > > >>> drivers/gpu/drm/i915/intel_memory_region.c | 24 +-
> > > >>> include/drm/drm_device.h | 3 +
> > > >>> include/drm/drm_drv.h | 8 +
> > > >>> include/drm/drm_gem.h | 11 +
> > > >>> include/linux/cgroup-defs.h | 28 ++
> > > >>> include/linux/cgroup.h | 3 +
> > > >>> include/linux/memcontrol.h | 10 +
> > > >>> kernel/cgroup/cgroup-v1.c | 10 +-
> > > >>> kernel/cgroup/cgroup.c | 310 ++++++++++++++++++---
> > > >>> mm/memcontrol.c | 183 +++++++++++-
> > > >>> 13 files changed, 552 insertions(+), 59 deletions(-)
> > > >>>
> > > >>> --
> > > >>> 2.21.0
> > > >>>
> > > >> _______________________________________________
> > > >> dri-devel mailing list
> > > >> dri-devel@lists.freedesktop.org
> > > >> https://lists.freedesktop.org/mailman/listinfo/dri-devel
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply
* Re: [RFC PATCH 0/5] cgroup support for GPU devices
From: Kenny Ho @ 2019-05-05 14:21 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Parav Pandit, David Airlie, kenny.ho, intel-gfx, Welty, Brian,
cgroups, dri-devel, Michal Hocko, Johannes Weiner, linux-mm,
J??r??me Glisse, Li Zefan, Vladimir Davydov, Alex Deucher,
Tejun Heo, Christian K??nig, RDMA mailing list
In-Reply-To: <20190505071436.GD6938@mtr-leonro.mtl.com>
On Sun, May 5, 2019 at 3:14 AM Leon Romanovsky <leon@kernel.org> wrote:
> > > Doesn't RDMA already has a separate cgroup? Why not implement it there?
> > >
> >
> > Hi Kenny, I can't answer for Leon, but I'm hopeful he agrees with rationale
> > I gave in the cover letter. Namely, to implement in rdma controller, would
> > mean duplicating existing memcg controls there.
>
> Exactly, I didn't feel comfortable to add notion of "device memory"
> to RDMA cgroup and postponed that decision to later point of time.
> RDMA operates with verbs objects and all our user space API is based around
> that concept. At the end, system administrator will have hard time to
> understand the differences between memcg and RDMA memory.
Interesting. I actually don't understand this part (I worked in
devops/sysadmin side of things but never with rdma.) Don't
applications that use rdma require some awareness of rdma (I mean, you
mentioned verbs and objects... or do they just use regular malloc for
buffer allocation and then send it through some function?) As a user,
I would have this question: why do I need to configure some part of
rdma resources under rdma cgroup while other part of rdma resources in
a different, seemingly unrelated cgroups.
I think we need to be careful about drawing the line between
duplication and over couplings between subsystems. I have other
thoughts and concerns and I will try to organize them into a response
in the next few days.
Regards,
Kenny
> >
> > Is AMD interested in collaborating to help shape this framework?
> > It is intended to be device-neutral, so could be leveraged by various
> > types of devices.
> > If you have an alternative solution well underway, then maybe
> > we can work together to merge our efforts into one.
> > In the end, the DRM community is best served with common solution.
> >
> >
> > >
> > >>> and with future work, we could extend to:
> > >>> * track and control share of GPU time (reuse of cpu/cpuacct)
> > >>> * apply mask of allowed execution engines (reuse of cpusets)
> > >>>
> > >>> Instead of introducing a new cgroup subsystem for GPU devices, a new
> > >>> framework is proposed to allow devices to register with existing cgroup
> > >>> controllers, which creates per-device cgroup_subsys_state within the
> > >>> cgroup. This gives device drivers their own private cgroup controls
> > >>> (such as memory limits or other parameters) to be applied to device
> > >>> resources instead of host system resources.
> > >>> Device drivers (GPU or other) are then able to reuse the existing cgroup
> > >>> controls, instead of inventing similar ones.
> > >>>
> > >>> Per-device controls would be exposed in cgroup filesystem as:
> > >>> mount/<cgroup_name>/<subsys_name>.devices/<dev_name>/<subsys_files>
> > >>> such as (for example):
> > >>> mount/<cgroup_name>/memory.devices/<dev_name>/memory.max
> > >>> mount/<cgroup_name>/memory.devices/<dev_name>/memory.current
> > >>> mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.stat
> > >>> mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.weight
> > >>>
> > >>> The drm/i915 patch in this series is based on top of other RFC work [1]
> > >>> for i915 device memory support.
> > >>>
> > >>> AMD [2] and Intel [3] have proposed related work in this area within the
> > >>> last few years, listed below as reference. This new RFC reuses existing
> > >>> cgroup controllers and takes a different approach than prior work.
> > >>>
> > >>> Finally, some potential discussion points for this series:
> > >>> * merge proposed <subsys_name>.devices into a single devices directory?
> > >>> * allow devices to have multiple registrations for subsets of resources?
> > >>> * document a 'common charging policy' for device drivers to follow?
> > >>>
> > >>> [1] https://patchwork.freedesktop.org/series/56683/
> > >>> [2] https://lists.freedesktop.org/archives/dri-devel/2018-November/197106.html
> > >>> [3] https://lists.freedesktop.org/archives/intel-gfx/2018-January/153156.html
> > >>>
> > >>>
> > >>> Brian Welty (5):
> > >>> cgroup: Add cgroup_subsys per-device registration framework
> > >>> cgroup: Change kernfs_node for directories to store
> > >>> cgroup_subsys_state
> > >>> memcg: Add per-device support to memory cgroup subsystem
> > >>> drm: Add memory cgroup registration and DRIVER_CGROUPS feature bit
> > >>> drm/i915: Use memory cgroup for enforcing device memory limit
> > >>>
> > >>> drivers/gpu/drm/drm_drv.c | 12 +
> > >>> drivers/gpu/drm/drm_gem.c | 7 +
> > >>> drivers/gpu/drm/i915/i915_drv.c | 2 +-
> > >>> drivers/gpu/drm/i915/intel_memory_region.c | 24 +-
> > >>> include/drm/drm_device.h | 3 +
> > >>> include/drm/drm_drv.h | 8 +
> > >>> include/drm/drm_gem.h | 11 +
> > >>> include/linux/cgroup-defs.h | 28 ++
> > >>> include/linux/cgroup.h | 3 +
> > >>> include/linux/memcontrol.h | 10 +
> > >>> kernel/cgroup/cgroup-v1.c | 10 +-
> > >>> kernel/cgroup/cgroup.c | 310 ++++++++++++++++++---
> > >>> mm/memcontrol.c | 183 +++++++++++-
> > >>> 13 files changed, 552 insertions(+), 59 deletions(-)
> > >>>
> > >>> --
> > >>> 2.21.0
> > >>>
> > >> _______________________________________________
> > >> dri-devel mailing list
> > >> dri-devel@lists.freedesktop.org
> > >> https://lists.freedesktop.org/mailman/listinfo/dri-devel
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply
* Re: [RFC PATCH 0/5] cgroup support for GPU devices
From: Leon Romanovsky @ 2019-05-05 7:14 UTC (permalink / raw)
To: Welty, Brian
Cc: Parav Pandit, David Airlie, kenny.ho, intel-gfx, Johannes Weiner,
cgroups, dri-devel, Michal Hocko, linux-mm, J??r??me Glisse,
Li Zefan, Kenny Ho, Vladimir Davydov, Alex Deucher, Tejun Heo,
Christian K??nig, RDMA mailing list
In-Reply-To: <bb001de0-e4e5-6b3f-7ced-9d0fb329635b@intel.com>
On Fri, May 03, 2019 at 02:14:33PM -0700, Welty, Brian wrote:
>
> On 5/2/2019 3:48 PM, Kenny Ho wrote:
> > On 5/2/2019 1:34 AM, Leon Romanovsky wrote:
> >> Count us (Mellanox) too, our RDMA devices are exposing special and
> >> limited in size device memory to the users and we would like to provide
> >> an option to use cgroup to control its exposure.
>
> Hi Leon, great to hear and happy to work with you and RDMA community
> to shape this framework for use by RDMA devices as well. The intent
> was to support more than GPU devices.
>
> Incidentally, I also wanted to ask about the rdma cgroup controller
> and if there is interest in updating the device registration implemented
> in that controller. It could use the cgroup_device_register() that is
> proposed here. But this is perhaps future work, so can discuss separately.
I'll try to take a look later this week.
>
>
> > Doesn't RDMA already has a separate cgroup? Why not implement it there?
> >
>
> Hi Kenny, I can't answer for Leon, but I'm hopeful he agrees with rationale
> I gave in the cover letter. Namely, to implement in rdma controller, would
> mean duplicating existing memcg controls there.
Exactly, I didn't feel comfortable to add notion of "device memory"
to RDMA cgroup and postponed that decision to later point of time.
RDMA operates with verbs objects and all our user space API is based around
that concept. At the end, system administrator will have hard time to
understand the differences between memcg and RDMA memory.
>
> Is AMD interested in collaborating to help shape this framework?
> It is intended to be device-neutral, so could be leveraged by various
> types of devices.
> If you have an alternative solution well underway, then maybe
> we can work together to merge our efforts into one.
> In the end, the DRM community is best served with common solution.
>
>
> >
> >>> and with future work, we could extend to:
> >>> * track and control share of GPU time (reuse of cpu/cpuacct)
> >>> * apply mask of allowed execution engines (reuse of cpusets)
> >>>
> >>> Instead of introducing a new cgroup subsystem for GPU devices, a new
> >>> framework is proposed to allow devices to register with existing cgroup
> >>> controllers, which creates per-device cgroup_subsys_state within the
> >>> cgroup. This gives device drivers their own private cgroup controls
> >>> (such as memory limits or other parameters) to be applied to device
> >>> resources instead of host system resources.
> >>> Device drivers (GPU or other) are then able to reuse the existing cgroup
> >>> controls, instead of inventing similar ones.
> >>>
> >>> Per-device controls would be exposed in cgroup filesystem as:
> >>> mount/<cgroup_name>/<subsys_name>.devices/<dev_name>/<subsys_files>
> >>> such as (for example):
> >>> mount/<cgroup_name>/memory.devices/<dev_name>/memory.max
> >>> mount/<cgroup_name>/memory.devices/<dev_name>/memory.current
> >>> mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.stat
> >>> mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.weight
> >>>
> >>> The drm/i915 patch in this series is based on top of other RFC work [1]
> >>> for i915 device memory support.
> >>>
> >>> AMD [2] and Intel [3] have proposed related work in this area within the
> >>> last few years, listed below as reference. This new RFC reuses existing
> >>> cgroup controllers and takes a different approach than prior work.
> >>>
> >>> Finally, some potential discussion points for this series:
> >>> * merge proposed <subsys_name>.devices into a single devices directory?
> >>> * allow devices to have multiple registrations for subsets of resources?
> >>> * document a 'common charging policy' for device drivers to follow?
> >>>
> >>> [1] https://patchwork.freedesktop.org/series/56683/
> >>> [2] https://lists.freedesktop.org/archives/dri-devel/2018-November/197106.html
> >>> [3] https://lists.freedesktop.org/archives/intel-gfx/2018-January/153156.html
> >>>
> >>>
> >>> Brian Welty (5):
> >>> cgroup: Add cgroup_subsys per-device registration framework
> >>> cgroup: Change kernfs_node for directories to store
> >>> cgroup_subsys_state
> >>> memcg: Add per-device support to memory cgroup subsystem
> >>> drm: Add memory cgroup registration and DRIVER_CGROUPS feature bit
> >>> drm/i915: Use memory cgroup for enforcing device memory limit
> >>>
> >>> drivers/gpu/drm/drm_drv.c | 12 +
> >>> drivers/gpu/drm/drm_gem.c | 7 +
> >>> drivers/gpu/drm/i915/i915_drv.c | 2 +-
> >>> drivers/gpu/drm/i915/intel_memory_region.c | 24 +-
> >>> include/drm/drm_device.h | 3 +
> >>> include/drm/drm_drv.h | 8 +
> >>> include/drm/drm_gem.h | 11 +
> >>> include/linux/cgroup-defs.h | 28 ++
> >>> include/linux/cgroup.h | 3 +
> >>> include/linux/memcontrol.h | 10 +
> >>> kernel/cgroup/cgroup-v1.c | 10 +-
> >>> kernel/cgroup/cgroup.c | 310 ++++++++++++++++++---
> >>> mm/memcontrol.c | 183 +++++++++++-
> >>> 13 files changed, 552 insertions(+), 59 deletions(-)
> >>>
> >>> --
> >>> 2.21.0
> >>>
> >> _______________________________________________
> >> dri-devel mailing list
> >> dri-devel@lists.freedesktop.org
> >> https://lists.freedesktop.org/mailman/listinfo/dri-devel
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply
* Re: [PATCH v13 16/20] IB/mlx4, arm64: untag user pointers in mlx4_get_umem_mr
From: Jason Gunthorpe @ 2019-05-03 23:52 UTC (permalink / raw)
To: Catalin Marinas
Cc: Leon Romanovsky, Andrey Konovalov, Will Deacon, Robin Murphy,
Kees Cook, Greg Kroah-Hartman, Andrew Morton, Vincenzo Frascino,
Eric Dumazet, David S. Miller, Yishai Hadas, linux-arm-kernel,
linux-mm, linux-arch, netdev, linux-rdma, linux-kernel,
Dmitry Vyukov, Kostya Serebryany, Evgeniy Stepanov,
Ramana Radhakrishnan
In-Reply-To: <20190503162846.GI55449@arrakis.emea.arm.com>
On Fri, May 03, 2019 at 05:28:46PM +0100, Catalin Marinas wrote:
> Thanks Jason and Leon for the information.
>
> On Thu, May 02, 2019 at 03:44:42PM -0300, Jason Gunthorpe wrote:
> > On Tue, Apr 30, 2019 at 12:16:25PM +0100, Catalin Marinas wrote:
> > > > Interesting, the followup question is why mlx4 is only one driver in IB which
> > > > needs such code in umem_mr. I'll take a look on it.
> > >
> > > I don't know. Just using the light heuristics of find_vma() shows some
> > > other places. For example, ib_umem_odp_get() gets the umem->address via
> > > ib_umem_start(). This was previously set in ib_umem_get() as called from
> > > mlx4_get_umem_mr(). Should the above patch have just untagged "start" on
> > > entry?
> >
> > I have a feeling that there needs to be something for this in the odp
> > code..
> >
> > Presumably mmu notifiers and what not also use untagged pointers? Most
> > likely then the umem should also be storing untagged pointers.
>
> Yes.
>
> > This probably becomes problematic because we do want the tag in cases
> > talking about the base VA of the MR..
>
> It depends on whether the tag is relevant to the kernel or not. The only
> useful case so far is for the kernel performing copy_form_user() etc.
> accesses so they'd get checked in the presence of hardware memory
> tagging (MTE; but it's not mandatory, a 0 tag would do as well).
>
> If we talk about a memory range where the content is relatively opaque
> (or irrelevant) to the kernel code, we don't really need the tag. I'm
> not familiar to RDMA but I presume it would be a device accessing such
> MR but not through the user VA directly.
RDMA exposes the user VA directly (the IOVA) as part of the wire
protocol, we must preserve the tag in these cases as that is what the
userspace is using for the pointer.
So the ODP stuff will definately need some adjusting when it interacts
with the mmu notifiers and get user pages.
Jason
^ permalink raw reply
* Re: [RFC PATCH 0/5] cgroup support for GPU devices
From: Welty, Brian @ 2019-05-03 21:14 UTC (permalink / raw)
To: Kenny Ho, Leon Romanovsky
Cc: Parav Pandit, David Airlie, kenny.ho, intel-gfx, Johannes Weiner,
Harish.Kasiviswanathan, cgroups, dri-devel, Michal Hocko,
linux-mm, Jérôme Glisse, Li Zefan, Vladimir Davydov,
Rodrigo Vivi, Alex Deucher, Tejun Heo, Christian König,
RDMA mailing list
In-Reply-To: <CAOWid-cYknxeTQvP9vQf3-i3Cpux+bs7uBs7_o-YMFjVCo19bg@mail.gmail.com>
On 5/2/2019 3:48 PM, Kenny Ho wrote:
> On 5/2/2019 1:34 AM, Leon Romanovsky wrote:
>> Count us (Mellanox) too, our RDMA devices are exposing special and
>> limited in size device memory to the users and we would like to provide
>> an option to use cgroup to control its exposure.
Hi Leon, great to hear and happy to work with you and RDMA community
to shape this framework for use by RDMA devices as well. The intent
was to support more than GPU devices.
Incidentally, I also wanted to ask about the rdma cgroup controller
and if there is interest in updating the device registration implemented
in that controller. It could use the cgroup_device_register() that is
proposed here. But this is perhaps future work, so can discuss separately.
> Doesn't RDMA already has a separate cgroup? Why not implement it there?
>
Hi Kenny, I can't answer for Leon, but I'm hopeful he agrees with rationale
I gave in the cover letter. Namely, to implement in rdma controller, would
mean duplicating existing memcg controls there.
Is AMD interested in collaborating to help shape this framework?
It is intended to be device-neutral, so could be leveraged by various
types of devices.
If you have an alternative solution well underway, then maybe
we can work together to merge our efforts into one.
In the end, the DRM community is best served with common solution.
>
>>> and with future work, we could extend to:
>>> * track and control share of GPU time (reuse of cpu/cpuacct)
>>> * apply mask of allowed execution engines (reuse of cpusets)
>>>
>>> Instead of introducing a new cgroup subsystem for GPU devices, a new
>>> framework is proposed to allow devices to register with existing cgroup
>>> controllers, which creates per-device cgroup_subsys_state within the
>>> cgroup. This gives device drivers their own private cgroup controls
>>> (such as memory limits or other parameters) to be applied to device
>>> resources instead of host system resources.
>>> Device drivers (GPU or other) are then able to reuse the existing cgroup
>>> controls, instead of inventing similar ones.
>>>
>>> Per-device controls would be exposed in cgroup filesystem as:
>>> mount/<cgroup_name>/<subsys_name>.devices/<dev_name>/<subsys_files>
>>> such as (for example):
>>> mount/<cgroup_name>/memory.devices/<dev_name>/memory.max
>>> mount/<cgroup_name>/memory.devices/<dev_name>/memory.current
>>> mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.stat
>>> mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.weight
>>>
>>> The drm/i915 patch in this series is based on top of other RFC work [1]
>>> for i915 device memory support.
>>>
>>> AMD [2] and Intel [3] have proposed related work in this area within the
>>> last few years, listed below as reference. This new RFC reuses existing
>>> cgroup controllers and takes a different approach than prior work.
>>>
>>> Finally, some potential discussion points for this series:
>>> * merge proposed <subsys_name>.devices into a single devices directory?
>>> * allow devices to have multiple registrations for subsets of resources?
>>> * document a 'common charging policy' for device drivers to follow?
>>>
>>> [1] https://patchwork.freedesktop.org/series/56683/
>>> [2] https://lists.freedesktop.org/archives/dri-devel/2018-November/197106.html
>>> [3] https://lists.freedesktop.org/archives/intel-gfx/2018-January/153156.html
>>>
>>>
>>> Brian Welty (5):
>>> cgroup: Add cgroup_subsys per-device registration framework
>>> cgroup: Change kernfs_node for directories to store
>>> cgroup_subsys_state
>>> memcg: Add per-device support to memory cgroup subsystem
>>> drm: Add memory cgroup registration and DRIVER_CGROUPS feature bit
>>> drm/i915: Use memory cgroup for enforcing device memory limit
>>>
>>> drivers/gpu/drm/drm_drv.c | 12 +
>>> drivers/gpu/drm/drm_gem.c | 7 +
>>> drivers/gpu/drm/i915/i915_drv.c | 2 +-
>>> drivers/gpu/drm/i915/intel_memory_region.c | 24 +-
>>> include/drm/drm_device.h | 3 +
>>> include/drm/drm_drv.h | 8 +
>>> include/drm/drm_gem.h | 11 +
>>> include/linux/cgroup-defs.h | 28 ++
>>> include/linux/cgroup.h | 3 +
>>> include/linux/memcontrol.h | 10 +
>>> kernel/cgroup/cgroup-v1.c | 10 +-
>>> kernel/cgroup/cgroup.c | 310 ++++++++++++++++++---
>>> mm/memcontrol.c | 183 +++++++++++-
>>> 13 files changed, 552 insertions(+), 59 deletions(-)
>>>
>>> --
>>> 2.21.0
>>>
>> _______________________________________________
>> dri-devel mailing list
>> dri-devel@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/dri-devel
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply
* Re: [PATCH] net: rds: fix spelling mistake "syctl" -> "sysctl"
From: Santosh Shilimkar @ 2019-05-03 17:05 UTC (permalink / raw)
To: Colin King, David S . Miller, netdev, linux-rdma, rds-devel
Cc: kernel-janitors, linux-kernel
In-Reply-To: <20190503121017.5227-1-colin.king@canonical.com>
On 5/3/2019 5:10 AM, Colin King wrote:
> From: Colin Ian King <colin.king@canonical.com>
>
> There is a spelling mistake in a pr_warn warning. Fix it.
>
> Signed-off-by: Colin Ian King <colin.king@canonical.com>
> ---
Acked-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>
^ permalink raw reply
* Re: [PATCH v14 13/17] IB/mlx4, arm64: untag user pointers in mlx4_get_umem_mr
From: Catalin Marinas @ 2019-05-03 17:03 UTC (permalink / raw)
To: Andrey Konovalov
Cc: Mark Rutland, Koenig-hpIqsD4AKlfQT0dZR+AlfA,
kvm-u79uwXL29TY76Z2rM5mHXA, Szabolcs Nagy, Will Deacon,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
linux-kselftest-u79uwXL29TY76Z2rM5mHXA, Chintan Pandya, Felix,
Vincenzo Frascino, Jacob Bramley, Leon Romanovsky,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, Dave Martin,
Evgeniy Stepanov, linux-media-u79uwXL29TY76Z2rM5mHXA,
Kevin Brodsky, Kees Cook, Ruben Ayrapetyan, Ramana Radhakrishnan,
Deucher-hpIqsD4AKlfQT0dZR+AlfA, Leon Romanovsky,
Alex Williamson <alex>
In-Reply-To: <05c0c078b8b5984af4cc3b105a58c711dcd83342.1556630205.git.andreyknvl-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
On Tue, Apr 30, 2019 at 03:25:09PM +0200, Andrey Konovalov wrote:
> This patch is a part of a series that extends arm64 kernel ABI to allow to
> pass tagged user pointers (with the top byte set to something else other
> than 0x00) as syscall arguments.
>
> mlx4_get_umem_mr() uses provided user pointers for vma lookups, which can
> only by done with untagged pointers.
>
> Untag user pointers in this function.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> Reviewed-by: Leon Romanovsky <leonro@mellanox.com>
> ---
> drivers/infiniband/hw/mlx4/mr.c | 7 ++++---
> 1 file changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/infiniband/hw/mlx4/mr.c b/drivers/infiniband/hw/mlx4/mr.c
> index 395379a480cb..9a35ed2c6a6f 100644
> --- a/drivers/infiniband/hw/mlx4/mr.c
> +++ b/drivers/infiniband/hw/mlx4/mr.c
> @@ -378,6 +378,7 @@ static struct ib_umem *mlx4_get_umem_mr(struct ib_udata *udata, u64 start,
> * again
> */
> if (!ib_access_writable(access_flags)) {
> + unsigned long untagged_start = untagged_addr(start);
> struct vm_area_struct *vma;
>
> down_read(¤t->mm->mmap_sem);
> @@ -386,9 +387,9 @@ static struct ib_umem *mlx4_get_umem_mr(struct ib_udata *udata, u64 start,
> * cover the memory, but for now it requires a single vma to
> * entirely cover the MR to support RO mappings.
> */
> - vma = find_vma(current->mm, start);
> - if (vma && vma->vm_end >= start + length &&
> - vma->vm_start <= start) {
> + vma = find_vma(current->mm, untagged_start);
> + if (vma && vma->vm_end >= untagged_start + length &&
> + vma->vm_start <= untagged_start) {
> if (vma->vm_flags & VM_WRITE)
> access_flags |= IB_ACCESS_LOCAL_WRITE;
> } else {
Discussion ongoing on the previous version of the patch but I'm more
inclined to do this in ib_uverbs_(re)reg_mr() on cmd.start.
--
Catalin
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply
* Re: [PATCH v14 10/17] fs, arm64: untag user pointers in fs/userfaultfd.c
From: Catalin Marinas @ 2019-05-03 16:56 UTC (permalink / raw)
To: Andrey Konovalov
Cc: linux-arm-kernel, linux-mm, linux-kernel, amd-gfx, dri-devel,
linux-rdma, linux-media, kvm, linux-kselftest, Vincenzo Frascino,
Will Deacon, Mark Rutland, Andrew Morton, Greg Kroah-Hartman,
Kees Cook, Yishai Hadas, Kuehling, Felix, Deucher, Alexander,
Koenig, Christian, Mauro Carvalho Chehab, Jens
In-Reply-To: <7d3b28689d47c0fa1b80628f248dbf78548da25f.1556630205.git.andreyknvl@google.com>
On Tue, Apr 30, 2019 at 03:25:06PM +0200, Andrey Konovalov wrote:
> This patch is a part of a series that extends arm64 kernel ABI to allow to
> pass tagged user pointers (with the top byte set to something else other
> than 0x00) as syscall arguments.
>
> userfaultfd_register() and userfaultfd_unregister() use provided user
> pointers for vma lookups, which can only by done with untagged pointers.
>
> Untag user pointers in these functions.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> ---
> fs/userfaultfd.c | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
> index f5de1e726356..fdee0db0e847 100644
> --- a/fs/userfaultfd.c
> +++ b/fs/userfaultfd.c
> @@ -1325,6 +1325,9 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx,
> goto out;
> }
>
> + uffdio_register.range.start =
> + untagged_addr(uffdio_register.range.start);
> +
> ret = validate_range(mm, uffdio_register.range.start,
> uffdio_register.range.len);
> if (ret)
> @@ -1514,6 +1517,8 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx,
> if (copy_from_user(&uffdio_unregister, buf, sizeof(uffdio_unregister)))
> goto out;
>
> + uffdio_unregister.start = untagged_addr(uffdio_unregister.start);
> +
> ret = validate_range(mm, uffdio_unregister.start,
> uffdio_unregister.len);
> if (ret)
Wouldn't it be easier to do this in validate_range()? There are a few
more calls in this file, though I didn't check whether a tagged address
would cause issues.
--
Catalin
^ permalink raw reply
* Re: [PATCH v14 08/17] mm, arm64: untag user pointers in get_vaddr_frames
From: Catalin Marinas @ 2019-05-03 16:51 UTC (permalink / raw)
To: Andrey Konovalov
Cc: Mark Rutland, Koenig-hpIqsD4AKlfQT0dZR+AlfA,
kvm-u79uwXL29TY76Z2rM5mHXA, Szabolcs Nagy, Will Deacon,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
linux-kselftest-u79uwXL29TY76Z2rM5mHXA, Chintan Pandya, Felix,
Vincenzo Frascino, Jacob Bramley, Leon Romanovsky,
linux-rdma-u79uwXL29TY76Z2rM5mHXA,
amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r, Dave Martin,
Evgeniy Stepanov, linux-media-u79uwXL29TY76Z2rM5mHXA,
Kevin Brodsky, Kees Cook, Ruben Ayrapetyan, Ramana Radhakrishnan,
Deucher-hpIqsD4AKlfQT0dZR+AlfA, Alex Williamson, Dmitry
In-Reply-To: <8e20df035de677029b3f970744ba2d35e2df1db3.1556630205.git.andreyknvl-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
On Tue, Apr 30, 2019 at 03:25:04PM +0200, Andrey Konovalov wrote:
> This patch is a part of a series that extends arm64 kernel ABI to allow to
> pass tagged user pointers (with the top byte set to something else other
> than 0x00) as syscall arguments.
>
> get_vaddr_frames uses provided user pointers for vma lookups, which can
> only by done with untagged pointers. Instead of locating and changing
> all callers of this function, perform untagging in it.
>
> Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
> ---
> mm/frame_vector.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/mm/frame_vector.c b/mm/frame_vector.c
> index c64dca6e27c2..c431ca81dad5 100644
> --- a/mm/frame_vector.c
> +++ b/mm/frame_vector.c
> @@ -46,6 +46,8 @@ int get_vaddr_frames(unsigned long start, unsigned int nr_frames,
> if (WARN_ON_ONCE(nr_frames > vec->nr_allocated))
> nr_frames = vec->nr_allocated;
>
> + start = untagged_addr(start);
> +
> down_read(&mm->mmap_sem);
> locked = 1;
> vma = find_vma_intersection(mm, start, start + 1);
Is this some buffer that the user may have malloc'ed? I got lost when
trying to track down the provenience of this buffer.
--
Catalin
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
^ permalink raw reply
* Re: [PATCH v13 16/20] IB/mlx4, arm64: untag user pointers in mlx4_get_umem_mr
From: Catalin Marinas @ 2019-05-03 16:28 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Leon Romanovsky, Andrey Konovalov, Will Deacon, Robin Murphy,
Kees Cook, Greg Kroah-Hartman, Andrew Morton, Vincenzo Frascino,
Eric Dumazet, David S. Miller, Yishai Hadas, linux-arm-kernel,
linux-mm, linux-arch, netdev, linux-rdma, linux-kernel,
Dmitry Vyukov, Kostya Serebryany, Evgeniy Stepanov,
Ramana Radhakrishnan
In-Reply-To: <20190502184442.GA31165@ziepe.ca>
Thanks Jason and Leon for the information.
On Thu, May 02, 2019 at 03:44:42PM -0300, Jason Gunthorpe wrote:
> On Tue, Apr 30, 2019 at 12:16:25PM +0100, Catalin Marinas wrote:
> > > Interesting, the followup question is why mlx4 is only one driver in IB which
> > > needs such code in umem_mr. I'll take a look on it.
> >
> > I don't know. Just using the light heuristics of find_vma() shows some
> > other places. For example, ib_umem_odp_get() gets the umem->address via
> > ib_umem_start(). This was previously set in ib_umem_get() as called from
> > mlx4_get_umem_mr(). Should the above patch have just untagged "start" on
> > entry?
>
> I have a feeling that there needs to be something for this in the odp
> code..
>
> Presumably mmu notifiers and what not also use untagged pointers? Most
> likely then the umem should also be storing untagged pointers.
Yes.
> This probably becomes problematic because we do want the tag in cases
> talking about the base VA of the MR..
It depends on whether the tag is relevant to the kernel or not. The only
useful case so far is for the kernel performing copy_form_user() etc.
accesses so they'd get checked in the presence of hardware memory
tagging (MTE; but it's not mandatory, a 0 tag would do as well).
If we talk about a memory range where the content is relatively opaque
(or irrelevant) to the kernel code, we don't really need the tag. I'm
not familiar to RDMA but I presume it would be a device accessing such
MR but not through the user VA directly. The tag is a property of the
buffer address/pointer when accessed by the CPU at that specific address
range. Any DMA or even kernel accessing it through the linear mapping
(get_user_pages()) would drop such tag.
--
Catalin
^ permalink raw reply
* [PATCH] net: rds: fix spelling mistake "syctl" -> "sysctl"
From: Colin King @ 2019-05-03 12:10 UTC (permalink / raw)
To: Santosh Shilimkar, David S . Miller, netdev, linux-rdma,
rds-devel
Cc: kernel-janitors, linux-kernel
From: Colin Ian King <colin.king@canonical.com>
There is a spelling mistake in a pr_warn warning. Fix it.
Signed-off-by: Colin Ian King <colin.king@canonical.com>
---
net/rds/tcp.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/net/rds/tcp.c b/net/rds/tcp.c
index faf726e00e27..66121bc6f34e 100644
--- a/net/rds/tcp.c
+++ b/net/rds/tcp.c
@@ -551,7 +551,7 @@ static __net_init int rds_tcp_init_net(struct net *net)
tbl = kmemdup(rds_tcp_sysctl_table,
sizeof(rds_tcp_sysctl_table), GFP_KERNEL);
if (!tbl) {
- pr_warn("could not set allocate syctl table\n");
+ pr_warn("could not set allocate sysctl table\n");
return -ENOMEM;
}
rtn->ctl_table = tbl;
--
2.20.1
^ permalink raw reply related
* Re: [RFC PATCH 0/5] cgroup support for GPU devices
From: Kenny Ho @ 2019-05-02 22:48 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Parav Pandit, David Airlie, intel-gfx, Brian Welty, cgroups,
dri-devel, Michal Hocko, Johannes Weiner, linux-mm,
Jérôme Glisse, Li Zefan, Vladimir Davydov, Alex Deucher,
Tejun Heo, Christian König, RDMA mailing list
In-Reply-To: <20190502083433.GP7676@mtr-leonro.mtl.com>
> Count us (Mellanox) too, our RDMA devices are exposing special and
> limited in size device memory to the users and we would like to provide
> an option to use cgroup to control its exposure.
Doesn't RDMA already has a separate cgroup? Why not implement it there?
> > and with future work, we could extend to:
> > * track and control share of GPU time (reuse of cpu/cpuacct)
> > * apply mask of allowed execution engines (reuse of cpusets)
> >
> > Instead of introducing a new cgroup subsystem for GPU devices, a new
> > framework is proposed to allow devices to register with existing cgroup
> > controllers, which creates per-device cgroup_subsys_state within the
> > cgroup. This gives device drivers their own private cgroup controls
> > (such as memory limits or other parameters) to be applied to device
> > resources instead of host system resources.
> > Device drivers (GPU or other) are then able to reuse the existing cgroup
> > controls, instead of inventing similar ones.
> >
> > Per-device controls would be exposed in cgroup filesystem as:
> > mount/<cgroup_name>/<subsys_name>.devices/<dev_name>/<subsys_files>
> > such as (for example):
> > mount/<cgroup_name>/memory.devices/<dev_name>/memory.max
> > mount/<cgroup_name>/memory.devices/<dev_name>/memory.current
> > mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.stat
> > mount/<cgroup_name>/cpu.devices/<dev_name>/cpu.weight
> >
> > The drm/i915 patch in this series is based on top of other RFC work [1]
> > for i915 device memory support.
> >
> > AMD [2] and Intel [3] have proposed related work in this area within the
> > last few years, listed below as reference. This new RFC reuses existing
> > cgroup controllers and takes a different approach than prior work.
> >
> > Finally, some potential discussion points for this series:
> > * merge proposed <subsys_name>.devices into a single devices directory?
> > * allow devices to have multiple registrations for subsets of resources?
> > * document a 'common charging policy' for device drivers to follow?
> >
> > [1] https://patchwork.freedesktop.org/series/56683/
> > [2] https://lists.freedesktop.org/archives/dri-devel/2018-November/197106.html
> > [3] https://lists.freedesktop.org/archives/intel-gfx/2018-January/153156.html
> >
> >
> > Brian Welty (5):
> > cgroup: Add cgroup_subsys per-device registration framework
> > cgroup: Change kernfs_node for directories to store
> > cgroup_subsys_state
> > memcg: Add per-device support to memory cgroup subsystem
> > drm: Add memory cgroup registration and DRIVER_CGROUPS feature bit
> > drm/i915: Use memory cgroup for enforcing device memory limit
> >
> > drivers/gpu/drm/drm_drv.c | 12 +
> > drivers/gpu/drm/drm_gem.c | 7 +
> > drivers/gpu/drm/i915/i915_drv.c | 2 +-
> > drivers/gpu/drm/i915/intel_memory_region.c | 24 +-
> > include/drm/drm_device.h | 3 +
> > include/drm/drm_drv.h | 8 +
> > include/drm/drm_gem.h | 11 +
> > include/linux/cgroup-defs.h | 28 ++
> > include/linux/cgroup.h | 3 +
> > include/linux/memcontrol.h | 10 +
> > kernel/cgroup/cgroup-v1.c | 10 +-
> > kernel/cgroup/cgroup.c | 310 ++++++++++++++++++---
> > mm/memcontrol.c | 183 +++++++++++-
> > 13 files changed, 552 insertions(+), 59 deletions(-)
> >
> > --
> > 2.21.0
> >
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox