* Re: [PATCH v10 0/11] seccomp: add thread sync ability
From: Kees Cook @ 2014-07-11 17:55 UTC (permalink / raw)
To: Oleg Nesterov
Cc: LKML, Andy Lutomirski, Alexei Starovoitov,
Michael Kerrisk (man-pages), Andrew Morton, Daniel Borkmann,
Will Drewry, Julien Tinnes, David Drysdale, Linux API,
x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips,
linux-arch, linux-security-module
In-Reply-To: <20140711164931.GA18473@redhat.com>
On Fri, Jul 11, 2014 at 9:49 AM, Oleg Nesterov <oleg@redhat.com> wrote:
> On 07/10, Kees Cook wrote:
>>
>> This adds the ability for threads to request seccomp filter
>> synchronization across their thread group (at filter attach time).
>> For example, for Chrome to make sure graphic driver threads are fully
>> confined after seccomp filters have been attached.
>>
>> To support this, locking on seccomp changes via thread-group-shared
>> sighand lock is introduced, along with refactoring of no_new_privs. Races
>> with thread creation are handled via delayed duplication of the seccomp
>> task struct field and cred_guard_mutex.
>>
>> This includes a new syscall (instead of adding a new prctl option),
>> as suggested by Andy Lutomirski and Michael Kerrisk.
>
> I do not not see any problems in this version,
Awesome! Thank you for all the reviews. :) If Andy and Michael are
happy with this too, I think this is in good shape. \o/
-Kees
>
> Reviewed-by: Oleg Nesterov <oleg@redhat.com>
>
--
Kees Cook
Chrome OS Security
^ permalink raw reply
* Re: [PATCH v3 1/3] mm: introduce fincore()
From: Naoya Horiguchi @ 2014-07-11 16:53 UTC (permalink / raw)
To: Dave Hansen
Cc: Andrew Morton, Konstantin Khlebnikov, Wu Fengguang,
Arnaldo Carvalho de Melo, Borislav Petkov, Kirill A. Shutemov,
Johannes Weiner, Rusty Russell, David Miller, Andres Freund,
linux-kernel, linux-mm, Christoph Hellwig, Dave Chinner,
Michael Kerrisk, Linux API, Naoya Horiguchi
In-Reply-To: <53BC717E.6020705@intel.com>
On Tue, Jul 08, 2014 at 03:32:30PM -0700, Dave Hansen wrote:
> On 07/08/2014 01:41 PM, Naoya Horiguchi wrote:
> >> > It would only set the first two bytes of a
> >> > 256k BMAP buffer since only two pages were encountered in the radix tree.
> > Hmm, this example shows me a problem, thanks.
> >
> > If the user knows the fd is for 1GB hugetlbfs file, it just prepares
> > the 2 bytes buffer, so no problem.
> > But if the user doesn't know whether the fd is from hugetlbfs file,
> > the user must prepare the large buffer, though only first few bytes
> > are used. And the more problematic is that the user could interpret
> > the data in buffer differently:
> > 1. only the first two 4kB-pages are loaded in the 2GB range,
> > 2. two 1GB-pages are loaded.
> > So for such callers, fincore() must notify the relevant page size
> > in some way on return.
> > Returning it via fincore_extra is my first thought but I'm not sure
> > if it's elegant enough.
>
> That does limit the interface to being used on a single page size per
> call, which doesn't sound too bad since we don't mix page sizes in a
> single file. But, you mentioned using this interface along with
> /proc/$pid/mem. How would this deal with a process which had two sizes
> of pages mapped?
Hmm, we should handle everything (including hugetlbfs) in 4kB page in
BMAP mode, because the position of the data in user buffer has the meaning.
And maybe it should be the case basically for in extensible modes, but
only in FINCORE_PGOFF mode (where no data of holes is passed to userspace,
and per-page entry contains offset information, so the in-buffer position
doesn't mean anything,) we can skip tail pages in the natural manner.
In this approach, fincore(FINCORE_PGOFF) returns not only pgoff, but also
page order (encoded in highest bits in pgoff field?). The callers must be
prepared to handle different page sizes.
So if users want to avoid lots of tail data for hugetlbfs pages,
using FINCORE_PGOFF mode is recommended for them.
That allows us to handle regular files, hugetlbfs files and /proc/$pid/mem
consistently.
> Another option would be to have userspace pass in its desired
> granularity. Such an interface could be used to find holes in a file
> fairly easily. But, introduces a whole new set of issues, like what
> BMAP means if only a part of the granule is in-core, and do you need a
> new option to differentiate BMAP_AND vs. BMAP_OR operations.
I don't see exactly what you mention here, but I agree that it makes
more complexity and might not be easy to keep code maintenability.
> I honestly think we need to take a step back and enumerate what you're
> trying to do here before going any further.
OK, I try it. (Please correct/add if you find something I should)
What: typical usecases is like below:
1. mincore()'s variant for page cache. Exporting residency information
(PageUpdate) is required.
2. Helping IO control from userspace. Exporting relevant page flags
(PageDirty and PageWriteback, or others if needed) and NUMA node
information is required.
3. Error page check. This is an essential part of "error reporting" patchset
I'm developing now, where I make error information sticky on page cache
tree to let userspace take proper actions (the final goal is to avoid
consuming corrupted data.) Exporting PageError is required.
4. (debugging) Page offset to PFN translation. This is for debugging,
so should be available only for privileged users.
How: attempt to do these better:
- extensiblility. As we experience now on mincore(), Page residency
information is not enough (for example to predict necessity of fsync.)
We want to avoid adding new syscalls per future requirements of new
information.
- efficiency. For handling large (sparse) files and/or hugepages, BMAP
type interface is not optimal, so "table" type interface is useful
to avoid meaningless data transfer.
Considering the objection for exporting bare page flags from Christoph,
I'm thinking that we had better export some "translated" page flags.
For example, we now only export residency info, so let's define it
as FINCORE_RESIDENCY bit, and let's define FINCORE_NEEDSYNC, which
makes kernel to export a bit of PageDirty && !PageWriteback.
Maybe we can combine them if we like, and if we do, the user buffer will
be filled with 2 bit entries per page on return.
This is extensible and the memory footprint is minimum.
Thanks,
Naoya Horiguchi
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v10 0/11] seccomp: add thread sync ability
From: Oleg Nesterov @ 2014-07-11 16:49 UTC (permalink / raw)
To: Kees Cook
Cc: linux-kernel, Andy Lutomirski, Alexei Starovoitov,
Michael Kerrisk (man-pages), Andrew Morton, Daniel Borkmann,
Will Drewry, Julien Tinnes, David Drysdale, linux-api, x86,
linux-arm-kernel, linux-mips, linux-arch, linux-security-module
In-Reply-To: <1405017631-27346-1-git-send-email-keescook@chromium.org>
On 07/10, Kees Cook wrote:
>
> This adds the ability for threads to request seccomp filter
> synchronization across their thread group (at filter attach time).
> For example, for Chrome to make sure graphic driver threads are fully
> confined after seccomp filters have been attached.
>
> To support this, locking on seccomp changes via thread-group-shared
> sighand lock is introduced, along with refactoring of no_new_privs. Races
> with thread creation are handled via delayed duplication of the seccomp
> task struct field and cred_guard_mutex.
>
> This includes a new syscall (instead of adding a new prctl option),
> as suggested by Andy Lutomirski and Michael Kerrisk.
I do not not see any problems in this version,
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
^ permalink raw reply
* [PATCH 60/83] hsa/radeon: Rearrange structures in kfd_ioctl.h
From: Oded Gabbay @ 2014-07-10 21:54 UTC (permalink / raw)
To: David Airlie, Alex Deucher, Jerome Glisse
Cc: linux-kernel, dri-devel, John Bridgman, Andrew Lewycky,
Joerg Roedel, Oded Gabbay, Ben Goz, Evgeny Pinchuk,
Alexey Skidanov, linux-api
In-Reply-To: <1405029279-6894-1-git-send-email-oded.gabbay@amd.com>
This patch rearranges the structures defined in kfd_ioctl.h so that
all the uint64_t variables are located at the start of each structure and
then all the uint32_t variables are located.
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
---
include/uapi/linux/kfd_ioctl.h | 51 ++++++++++++++++++++++--------------------
1 file changed, 27 insertions(+), 24 deletions(-)
diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h
index 509c4a0..3cedd1a 100644
--- a/include/uapi/linux/kfd_ioctl.h
+++ b/include/uapi/linux/kfd_ioctl.h
@@ -42,15 +42,15 @@ struct kfd_ioctl_get_version_args {
struct kfd_ioctl_create_queue_args {
uint64_t ring_base_address; /* to KFD */
+ uint64_t write_pointer_address; /* from KFD */
+ uint64_t read_pointer_address; /* from KFD */
+ uint64_t doorbell_address; /* from KFD */
+
uint32_t ring_size; /* to KFD */
uint32_t gpu_id; /* to KFD */
uint32_t queue_type; /* to KFD */
uint32_t queue_percentage; /* to KFD */
uint32_t queue_priority; /* to KFD */
-
- uint64_t write_pointer_address; /* from KFD */
- uint64_t read_pointer_address; /* from KFD */
- uint64_t doorbell_address; /* from KFD */
uint32_t queue_id; /* from KFD */
};
@@ -59,8 +59,9 @@ struct kfd_ioctl_destroy_queue_args {
};
struct kfd_ioctl_update_queue_args {
- uint32_t queue_id; /* to KFD */
uint64_t ring_base_address; /* to KFD */
+
+ uint32_t queue_id; /* to KFD */
uint32_t ring_size; /* to KFD */
uint32_t queue_percentage; /* to KFD */
uint32_t queue_priority; /* to KFD */
@@ -71,31 +72,33 @@ struct kfd_ioctl_update_queue_args {
#define KFD_IOC_CACHE_POLICY_NONCOHERENT 1
struct kfd_ioctl_set_memory_policy_args {
+ uint64_t alternate_aperture_base; /* to KFD */
+ uint64_t alternate_aperture_size; /* to KFD */
+
uint32_t gpu_id; /* to KFD */
uint32_t default_policy; /* to KFD */
uint32_t alternate_policy; /* to KFD */
- uint64_t alternate_aperture_base; /* to KFD */
- uint64_t alternate_aperture_size; /* to KFD */
};
struct kfd_ioctl_get_clock_counters_args {
- uint32_t gpu_id; /* to KFD */
uint64_t gpu_clock_counter; /* from KFD */
uint64_t cpu_clock_counter; /* from KFD */
uint64_t system_clock_counter; /* from KFD */
uint64_t system_clock_freq; /* from KFD */
+
+ uint32_t gpu_id; /* to KFD */
};
#define NUM_OF_SUPPORTED_GPUS 7
struct kfd_process_device_apertures {
- uint64_t lds_base;/* from KFD */
- uint64_t lds_limit;/* from KFD */
- uint64_t scratch_base;/* from KFD */
- uint64_t scratch_limit;/* from KFD */
- uint64_t gpuvm_base;/* from KFD */
- uint64_t gpuvm_limit;/* from KFD */
- uint32_t gpu_id;/* from KFD */
+ uint64_t lds_base; /* from KFD */
+ uint64_t lds_limit; /* from KFD */
+ uint64_t scratch_base; /* from KFD */
+ uint64_t scratch_limit; /* from KFD */
+ uint64_t gpuvm_base; /* from KFD */
+ uint64_t gpuvm_limit; /* from KFD */
+ uint32_t gpu_id; /* from KFD */
};
struct kfd_ioctl_get_process_apertures_args {
@@ -104,24 +107,24 @@ struct kfd_ioctl_get_process_apertures_args {
};
struct kfd_ioctl_pmc_acquire_access_args {
- uint32_t gpu_id; /* to KFD */
- uint64_t trace_id; /* to KFD */
+ uint64_t trace_id; /* to KFD */
+ uint32_t gpu_id; /* to KFD */
};
struct kfd_ioctl_pmc_release_access_args {
- uint32_t gpu_id; /* to KFD */
- uint64_t trace_id; /* to KFD */
+ uint64_t trace_id; /* to KFD */
+ uint32_t gpu_id; /* to KFD */
};
#define KFD_IOC_MAGIC 'K'
-#define KFD_IOC_GET_VERSION _IOR(KFD_IOC_MAGIC, 1, struct kfd_ioctl_get_version_args)
-#define KFD_IOC_CREATE_QUEUE _IOWR(KFD_IOC_MAGIC, 2, struct kfd_ioctl_create_queue_args)
-#define KFD_IOC_DESTROY_QUEUE _IOWR(KFD_IOC_MAGIC, 3, struct kfd_ioctl_destroy_queue_args)
+#define KFD_IOC_GET_VERSION _IOR(KFD_IOC_MAGIC, 1, struct kfd_ioctl_get_version_args)
+#define KFD_IOC_CREATE_QUEUE _IOWR(KFD_IOC_MAGIC, 2, struct kfd_ioctl_create_queue_args)
+#define KFD_IOC_DESTROY_QUEUE _IOWR(KFD_IOC_MAGIC, 3, struct kfd_ioctl_destroy_queue_args)
#define KFD_IOC_SET_MEMORY_POLICY _IOW(KFD_IOC_MAGIC, 4, struct kfd_ioctl_set_memory_policy_args)
#define KFD_IOC_GET_CLOCK_COUNTERS _IOWR(KFD_IOC_MAGIC, 5, struct kfd_ioctl_get_clock_counters_args)
-#define KFD_IOC_GET_PROCESS_APERTURES _IOR(KFD_IOC_MAGIC, 6, struct kfd_ioctl_get_process_apertures_args)
-#define KFD_IOC_UPDATE_QUEUE _IOW(KFD_IOC_MAGIC, 7, struct kfd_ioctl_update_queue_args)
+#define KFD_IOC_GET_PROCESS_APERTURES _IOR(KFD_IOC_MAGIC, 6, struct kfd_ioctl_get_process_apertures_args)
+#define KFD_IOC_UPDATE_QUEUE _IOW(KFD_IOC_MAGIC, 7, struct kfd_ioctl_update_queue_args)
#define KFD_IOC_PMC_ACQUIRE_ACCESS _IOW(KFD_IOC_MAGIC, 12, struct kfd_ioctl_pmc_acquire_access_args)
#define KFD_IOC_PMC_RELEASE_ACCESS _IOW(KFD_IOC_MAGIC, 13, struct kfd_ioctl_pmc_release_access_args)
--
1.9.1
^ permalink raw reply related
* [PATCH 59/83] hsa/radeon: Exclusive access for perf. counters
From: Oded Gabbay @ 2014-07-10 21:54 UTC (permalink / raw)
To: David Airlie, Alex Deucher, Jerome Glisse
Cc: linux-kernel, dri-devel, John Bridgman, Andrew Lewycky,
Joerg Roedel, Evgeny Pinchuk, Oded Gabbay, Ben Goz,
Alexey Skidanov, linux-api
In-Reply-To: <1405029279-6894-1-git-send-email-oded.gabbay@amd.com>
From: Evgeny Pinchuk <evgeny.pinchuk@amd.com>
Introducing IOCTL implementation for controlling exclusive access to performace counters.
The exclusive access is per GPU device.
Signed-off-by: Evgeny Pinchuk <evgeny.pinchuk@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
---
drivers/gpu/hsa/radeon/kfd_chardev.c | 61 ++++++++++++++++++++++++++++++++++++
drivers/gpu/hsa/radeon/kfd_device.c | 2 ++
drivers/gpu/hsa/radeon/kfd_priv.h | 5 +++
drivers/gpu/hsa/radeon/kfd_process.c | 8 +++--
include/uapi/linux/kfd_ioctl.h | 12 +++++++
5 files changed, 86 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/hsa/radeon/kfd_chardev.c b/drivers/gpu/hsa/radeon/kfd_chardev.c
index 80b702e..b39df68 100644
--- a/drivers/gpu/hsa/radeon/kfd_chardev.c
+++ b/drivers/gpu/hsa/radeon/kfd_chardev.c
@@ -387,6 +387,59 @@ static int kfd_ioctl_get_process_apertures(struct file *filp, struct kfd_process
return 0;
}
+static long
+kfd_ioctl_pmc_acquire_access(struct file *filp, struct kfd_process *p, void __user *arg)
+{
+ struct kfd_ioctl_pmc_acquire_access_args args;
+ struct kfd_dev *dev;
+ int err = -EBUSY;
+
+ if (copy_from_user(&args, arg, sizeof(args)))
+ return -EFAULT;
+
+ dev = radeon_kfd_device_by_id(args.gpu_id);
+ if (dev == NULL)
+ return -EINVAL;
+
+ spin_lock(&dev->pmc_access_lock);
+ if (dev->pmc_locking_process == NULL) {
+ dev->pmc_locking_process = p;
+ dev->pmc_locking_trace = args.trace_id;
+ err = 0;
+ } else if (dev->pmc_locking_process == p && dev->pmc_locking_trace == args.trace_id) {
+ /* Same trace already has an access. Returning success */
+ err = 0;
+ }
+
+ spin_unlock(&dev->pmc_access_lock);
+
+ return err;
+}
+
+static long
+kfd_ioctl_pmc_release_access(struct file *filp, struct kfd_process *p, void __user *arg)
+{
+ struct kfd_ioctl_pmc_release_access_args args;
+ struct kfd_dev *dev;
+ int err = -EINVAL;
+
+ if (copy_from_user(&args, arg, sizeof(args)))
+ return -EFAULT;
+
+ dev = radeon_kfd_device_by_id(args.gpu_id);
+ if (dev == NULL)
+ return -EINVAL;
+
+ spin_lock(&dev->pmc_access_lock);
+ if (dev->pmc_locking_process == p && dev->pmc_locking_trace == args.trace_id) {
+ dev->pmc_locking_process = NULL;
+ dev->pmc_locking_trace = 0;
+ err = 0;
+ }
+ spin_unlock(&dev->pmc_access_lock);
+
+ return err;
+}
static long
kfd_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)
@@ -427,6 +480,14 @@ kfd_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)
err = kfd_ioctl_update_queue(filep, process, (void __user *)arg);
break;
+ case KFD_IOC_PMC_ACQUIRE_ACCESS:
+ err = kfd_ioctl_pmc_acquire_access(filep, process, (void __user *) arg);
+ break;
+
+ case KFD_IOC_PMC_RELEASE_ACCESS:
+ err = kfd_ioctl_pmc_release_access(filep, process, (void __user *) arg);
+ break;
+
default:
dev_err(kfd_device,
"unknown ioctl cmd 0x%x, arg 0x%lx)\n",
diff --git a/drivers/gpu/hsa/radeon/kfd_device.c b/drivers/gpu/hsa/radeon/kfd_device.c
index c602e16..9af812b 100644
--- a/drivers/gpu/hsa/radeon/kfd_device.c
+++ b/drivers/gpu/hsa/radeon/kfd_device.c
@@ -185,6 +185,8 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
return false;
}
+ spin_lock_init(&kfd->pmc_access_lock);
+
kfd->init_complete = true;
dev_info(kfd_device, "added device (%x:%x)\n", kfd->pdev->vendor,
kfd->pdev->device);
diff --git a/drivers/gpu/hsa/radeon/kfd_priv.h b/drivers/gpu/hsa/radeon/kfd_priv.h
index 049671b..e6d4993 100644
--- a/drivers/gpu/hsa/radeon/kfd_priv.h
+++ b/drivers/gpu/hsa/radeon/kfd_priv.h
@@ -135,6 +135,11 @@ struct kfd_dev {
/* QCM Device instance */
struct device_queue_manager *dqm;
+
+ /* Performance counters exclusivity lock */
+ spinlock_t pmc_access_lock;
+ struct kfd_process *pmc_locking_process;
+ uint64_t pmc_locking_trace;
};
/* KGD2KFD callbacks */
diff --git a/drivers/gpu/hsa/radeon/kfd_process.c b/drivers/gpu/hsa/radeon/kfd_process.c
index f967c15..9bb5cab 100644
--- a/drivers/gpu/hsa/radeon/kfd_process.c
+++ b/drivers/gpu/hsa/radeon/kfd_process.c
@@ -96,9 +96,13 @@ static void free_process(struct kfd_process *p)
BUG_ON(p == NULL);
- /* doorbell mappings: automatic */
-
list_for_each_entry_safe(pdd, temp, &p->per_device_data, per_device_list) {
+ spin_lock(&pdd->dev->pmc_access_lock);
+ if (pdd->dev->pmc_locking_process == p) {
+ pdd->dev->pmc_locking_process = NULL;
+ pdd->dev->pmc_locking_trace = 0;
+ }
+ spin_unlock(&pdd->dev->pmc_access_lock);
amd_iommu_unbind_pasid(pdd->dev->pdev, p->pasid);
list_del(&pdd->per_device_list);
kfree(pdd);
diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h
index d58231d..509c4a0 100644
--- a/include/uapi/linux/kfd_ioctl.h
+++ b/include/uapi/linux/kfd_ioctl.h
@@ -103,6 +103,16 @@ struct kfd_ioctl_get_process_apertures_args {
uint8_t num_of_nodes; /* from KFD, should be in the range [1 - NUM_OF_SUPPORTED_GPUS]*/
};
+struct kfd_ioctl_pmc_acquire_access_args {
+ uint32_t gpu_id; /* to KFD */
+ uint64_t trace_id; /* to KFD */
+};
+
+struct kfd_ioctl_pmc_release_access_args {
+ uint32_t gpu_id; /* to KFD */
+ uint64_t trace_id; /* to KFD */
+};
+
#define KFD_IOC_MAGIC 'K'
#define KFD_IOC_GET_VERSION _IOR(KFD_IOC_MAGIC, 1, struct kfd_ioctl_get_version_args)
@@ -112,6 +122,8 @@ struct kfd_ioctl_get_process_apertures_args {
#define KFD_IOC_GET_CLOCK_COUNTERS _IOWR(KFD_IOC_MAGIC, 5, struct kfd_ioctl_get_clock_counters_args)
#define KFD_IOC_GET_PROCESS_APERTURES _IOR(KFD_IOC_MAGIC, 6, struct kfd_ioctl_get_process_apertures_args)
#define KFD_IOC_UPDATE_QUEUE _IOW(KFD_IOC_MAGIC, 7, struct kfd_ioctl_update_queue_args)
+#define KFD_IOC_PMC_ACQUIRE_ACCESS _IOW(KFD_IOC_MAGIC, 12, struct kfd_ioctl_pmc_acquire_access_args)
+#define KFD_IOC_PMC_RELEASE_ACCESS _IOW(KFD_IOC_MAGIC, 13, struct kfd_ioctl_pmc_release_access_args)
#pragma pack(pop)
--
1.9.1
^ permalink raw reply related
* [PATCH 55/83] hsa/radeon: Add IOCTL for update queue
From: Oded Gabbay @ 2014-07-10 21:54 UTC (permalink / raw)
To: David Airlie, Alex Deucher, Jerome Glisse
Cc: linux-kernel, dri-devel, John Bridgman, Andrew Lewycky,
Joerg Roedel, Ben Goz, Oded Gabbay, Evgeny Pinchuk,
Alexey Skidanov, linux-api
In-Reply-To: <1405029279-6894-1-git-send-email-oded.gabbay@amd.com>
From: Ben Goz <ben.goz@amd.com>
This patch adds a new IOCTL that enables the user to perform update to an HSA
queue.
Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
---
drivers/gpu/hsa/radeon/cik_mqds.h | 1 -
drivers/gpu/hsa/radeon/kfd_chardev.c | 29 ++++++++++++++++++++++
drivers/gpu/hsa/radeon/kfd_device_queue_manager.c | 1 -
drivers/gpu/hsa/radeon/kfd_device_queue_manager.h | 1 -
drivers/gpu/hsa/radeon/kfd_hw_pointer_store.c | 1 -
drivers/gpu/hsa/radeon/kfd_hw_pointer_store.h | 1 -
drivers/gpu/hsa/radeon/kfd_kernel_queue.c | 1 -
drivers/gpu/hsa/radeon/kfd_kernel_queue.h | 1 -
drivers/gpu/hsa/radeon/kfd_mqd_manager.c | 1 -
drivers/gpu/hsa/radeon/kfd_mqd_manager.h | 1 -
drivers/gpu/hsa/radeon/kfd_packet_manager.c | 23 ++++++++++++++---
drivers/gpu/hsa/radeon/kfd_process_queue_manager.c | 1 -
drivers/gpu/hsa/radeon/kfd_queue.c | 1 -
include/uapi/linux/kfd_ioctl.h | 9 +++++++
14 files changed, 58 insertions(+), 14 deletions(-)
diff --git a/drivers/gpu/hsa/radeon/cik_mqds.h b/drivers/gpu/hsa/radeon/cik_mqds.h
index 58945c8..35a35b4 100644
--- a/drivers/gpu/hsa/radeon/cik_mqds.h
+++ b/drivers/gpu/hsa/radeon/cik_mqds.h
@@ -19,7 +19,6 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
- * Author: Ben Goz
*/
#ifndef CIK_MQDS_H_
diff --git a/drivers/gpu/hsa/radeon/kfd_chardev.c b/drivers/gpu/hsa/radeon/kfd_chardev.c
index bb2ef02..9a77332 100644
--- a/drivers/gpu/hsa/radeon/kfd_chardev.c
+++ b/drivers/gpu/hsa/radeon/kfd_chardev.c
@@ -230,6 +230,31 @@ kfd_ioctl_destroy_queue(struct file *filp, struct kfd_process *p, void __user *a
return retval;
}
+static int
+kfd_ioctl_update_queue(struct file *filp, struct kfd_process *p, void __user *arg)
+{
+ int retval;
+ struct kfd_ioctl_update_queue_args args;
+ struct queue_properties properties;
+
+ if (copy_from_user(&args, arg, sizeof(args)))
+ return -EFAULT;
+
+ properties.queue_address = args.ring_base_address;
+ properties.queue_size = args.ring_size;
+ properties.queue_percent = args.queue_percentage;
+ properties.priority = args.queue_priority;
+
+ pr_debug("kfd: updating queue id %d for PASID %d\n", args.queue_id, p->pasid);
+
+ mutex_lock(&p->mutex);
+
+ retval = pqm_update_queue(&p->pqm, args.queue_id, &properties);
+
+ mutex_unlock(&p->mutex);
+
+ return retval;
+}
static long
kfd_ioctl_set_memory_policy(struct file *filep, struct kfd_process *p, void __user *arg)
@@ -398,6 +423,10 @@ kfd_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)
err = kfd_ioctl_get_process_apertures(filep, process, (void __user *)arg);
break;
+ case KFD_IOC_UPDATE_QUEUE:
+ err = kfd_ioctl_update_queue(filep, process, (void __user *)arg);
+ break;
+
default:
dev_err(kfd_device,
"unknown ioctl cmd 0x%x, arg 0x%lx)\n",
diff --git a/drivers/gpu/hsa/radeon/kfd_device_queue_manager.c b/drivers/gpu/hsa/radeon/kfd_device_queue_manager.c
index 9e21074..c2d91c9 100644
--- a/drivers/gpu/hsa/radeon/kfd_device_queue_manager.c
+++ b/drivers/gpu/hsa/radeon/kfd_device_queue_manager.c
@@ -19,7 +19,6 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
- * Author: Ben Goz
*/
#include <linux/slab.h>
diff --git a/drivers/gpu/hsa/radeon/kfd_device_queue_manager.h b/drivers/gpu/hsa/radeon/kfd_device_queue_manager.h
index 0529a96..fe9ef10 100644
--- a/drivers/gpu/hsa/radeon/kfd_device_queue_manager.h
+++ b/drivers/gpu/hsa/radeon/kfd_device_queue_manager.h
@@ -19,7 +19,6 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
- * Author: Ben Goz
*/
#ifndef DEVICE_QUEUE_MANAGER_H_
diff --git a/drivers/gpu/hsa/radeon/kfd_hw_pointer_store.c b/drivers/gpu/hsa/radeon/kfd_hw_pointer_store.c
index 1372fb2..4e71f7d 100644
--- a/drivers/gpu/hsa/radeon/kfd_hw_pointer_store.c
+++ b/drivers/gpu/hsa/radeon/kfd_hw_pointer_store.c
@@ -19,7 +19,6 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
- * Author: Ben Goz
*/
#include <linux/types.h>
diff --git a/drivers/gpu/hsa/radeon/kfd_hw_pointer_store.h b/drivers/gpu/hsa/radeon/kfd_hw_pointer_store.h
index be1d6cb..f384b7f 100644
--- a/drivers/gpu/hsa/radeon/kfd_hw_pointer_store.h
+++ b/drivers/gpu/hsa/radeon/kfd_hw_pointer_store.h
@@ -19,7 +19,6 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
- * Author: Ben Goz
*/
#ifndef HW_POINTER_STORE_H_
diff --git a/drivers/gpu/hsa/radeon/kfd_kernel_queue.c b/drivers/gpu/hsa/radeon/kfd_kernel_queue.c
index 61f420f..aa64693e 100644
--- a/drivers/gpu/hsa/radeon/kfd_kernel_queue.c
+++ b/drivers/gpu/hsa/radeon/kfd_kernel_queue.c
@@ -19,7 +19,6 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
- * Author: Ben Goz
*/
#include <linux/types.h>
diff --git a/drivers/gpu/hsa/radeon/kfd_kernel_queue.h b/drivers/gpu/hsa/radeon/kfd_kernel_queue.h
index 339376c..963e861 100644
--- a/drivers/gpu/hsa/radeon/kfd_kernel_queue.h
+++ b/drivers/gpu/hsa/radeon/kfd_kernel_queue.h
@@ -19,7 +19,6 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
- * Author: Ben Goz
*/
#ifndef KERNEL_QUEUE_H_
diff --git a/drivers/gpu/hsa/radeon/kfd_mqd_manager.c b/drivers/gpu/hsa/radeon/kfd_mqd_manager.c
index 14b248f..a3e9f7c 100644
--- a/drivers/gpu/hsa/radeon/kfd_mqd_manager.c
+++ b/drivers/gpu/hsa/radeon/kfd_mqd_manager.c
@@ -19,7 +19,6 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
- * Author: Ben Goz
*/
#include <linux/printk.h>
diff --git a/drivers/gpu/hsa/radeon/kfd_mqd_manager.h b/drivers/gpu/hsa/radeon/kfd_mqd_manager.h
index e7b39ee..8e7a5fd 100644
--- a/drivers/gpu/hsa/radeon/kfd_mqd_manager.h
+++ b/drivers/gpu/hsa/radeon/kfd_mqd_manager.h
@@ -19,7 +19,6 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
- * Author: Ben Goz
*/
#ifndef MQD_MANAGER_H_
diff --git a/drivers/gpu/hsa/radeon/kfd_packet_manager.c b/drivers/gpu/hsa/radeon/kfd_packet_manager.c
index 4967b7c..3fc8c34 100644
--- a/drivers/gpu/hsa/radeon/kfd_packet_manager.c
+++ b/drivers/gpu/hsa/radeon/kfd_packet_manager.c
@@ -1,9 +1,26 @@
/*
- * packet_manager.c
+ * Copyright 2014 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
*
- * Created on: Mar 16, 2014
- * Author: ben
*/
+
#include <linux/slab.h>
#include <linux/mutex.h>
#include "kfd_device_queue_manager.h"
diff --git a/drivers/gpu/hsa/radeon/kfd_process_queue_manager.c b/drivers/gpu/hsa/radeon/kfd_process_queue_manager.c
index 6e38ca4..fe74dd7 100644
--- a/drivers/gpu/hsa/radeon/kfd_process_queue_manager.c
+++ b/drivers/gpu/hsa/radeon/kfd_process_queue_manager.c
@@ -19,7 +19,6 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
- * Author: Ben Goz
*/
#include <linux/slab.h>
diff --git a/drivers/gpu/hsa/radeon/kfd_queue.c b/drivers/gpu/hsa/radeon/kfd_queue.c
index 78fe180..2d22cc1 100644
--- a/drivers/gpu/hsa/radeon/kfd_queue.c
+++ b/drivers/gpu/hsa/radeon/kfd_queue.c
@@ -19,7 +19,6 @@
* ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
- * Author: Ben Goz
*/
#include <linux/slab.h>
diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h
index 5134880..d58231d 100644
--- a/include/uapi/linux/kfd_ioctl.h
+++ b/include/uapi/linux/kfd_ioctl.h
@@ -58,6 +58,14 @@ struct kfd_ioctl_destroy_queue_args {
uint32_t queue_id; /* to KFD */
};
+struct kfd_ioctl_update_queue_args {
+ uint32_t queue_id; /* to KFD */
+ uint64_t ring_base_address; /* to KFD */
+ uint32_t ring_size; /* to KFD */
+ uint32_t queue_percentage; /* to KFD */
+ uint32_t queue_priority; /* to KFD */
+};
+
/* For kfd_ioctl_set_memory_policy_args.default_policy and alternate_policy */
#define KFD_IOC_CACHE_POLICY_COHERENT 0
#define KFD_IOC_CACHE_POLICY_NONCOHERENT 1
@@ -103,6 +111,7 @@ struct kfd_ioctl_get_process_apertures_args {
#define KFD_IOC_SET_MEMORY_POLICY _IOW(KFD_IOC_MAGIC, 4, struct kfd_ioctl_set_memory_policy_args)
#define KFD_IOC_GET_CLOCK_COUNTERS _IOWR(KFD_IOC_MAGIC, 5, struct kfd_ioctl_get_clock_counters_args)
#define KFD_IOC_GET_PROCESS_APERTURES _IOR(KFD_IOC_MAGIC, 6, struct kfd_ioctl_get_process_apertures_args)
+#define KFD_IOC_UPDATE_QUEUE _IOW(KFD_IOC_MAGIC, 7, struct kfd_ioctl_update_queue_args)
#pragma pack(pop)
--
1.9.1
^ permalink raw reply related
* [PATCH 54/83] hsa/radeon: Switch to new queue scheduler
From: Oded Gabbay @ 2014-07-10 21:54 UTC (permalink / raw)
To: David Airlie, Alex Deucher, Jerome Glisse
Cc: linux-kernel, dri-devel, John Bridgman, Andrew Lewycky,
Joerg Roedel, Ben Goz, Oded Gabbay, Evgeny Pinchuk,
Alexey Skidanov, linux-api
In-Reply-To: <1405029279-6894-1-git-send-email-oded.gabbay@amd.com>
From: Ben Goz <ben.goz@amd.com>
This patch makes the switch between the old KFD queue scheduler to the new KFD
queue scheduler. The new scheduler supports H/W CP scheduling, over-subscription
of queues and pre-emption of queues.
Signed-off-by: Ben Goz <ben.goz@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
---
drivers/gpu/hsa/radeon/kfd_aperture.c | 1 -
drivers/gpu/hsa/radeon/kfd_chardev.c | 107 +++++++++++++++------------------
drivers/gpu/hsa/radeon/kfd_device.c | 31 ++++++----
drivers/gpu/hsa/radeon/kfd_interrupt.c | 4 +-
drivers/gpu/hsa/radeon/kfd_priv.h | 2 +
drivers/gpu/hsa/radeon/kfd_process.c | 56 ++++-------------
include/uapi/linux/kfd_ioctl.h | 4 +-
7 files changed, 88 insertions(+), 117 deletions(-)
diff --git a/drivers/gpu/hsa/radeon/kfd_aperture.c b/drivers/gpu/hsa/radeon/kfd_aperture.c
index 9e2d6da..2c72b21 100644
--- a/drivers/gpu/hsa/radeon/kfd_aperture.c
+++ b/drivers/gpu/hsa/radeon/kfd_aperture.c
@@ -32,7 +32,6 @@
#include <uapi/linux/kfd_ioctl.h>
#include <linux/time.h>
#include "kfd_priv.h"
-#include "kfd_scheduler.h"
#include <linux/mm.h>
#include <uapi/asm-generic/mman-common.h>
#include <asm/processor.h>
diff --git a/drivers/gpu/hsa/radeon/kfd_chardev.c b/drivers/gpu/hsa/radeon/kfd_chardev.c
index 07cac88..bb2ef02 100644
--- a/drivers/gpu/hsa/radeon/kfd_chardev.c
+++ b/drivers/gpu/hsa/radeon/kfd_chardev.c
@@ -31,10 +31,11 @@
#include <uapi/linux/kfd_ioctl.h>
#include <linux/time.h>
#include "kfd_priv.h"
-#include "kfd_scheduler.h"
#include <linux/mm.h>
#include <uapi/asm-generic/mman-common.h>
#include <asm/processor.h>
+#include "kfd_hw_pointer_store.h"
+#include "kfd_device_queue_manager.h"
static long kfd_ioctl(struct file *, unsigned int, unsigned long);
static int kfd_open(struct inode *, struct file *);
@@ -128,24 +129,36 @@ kfd_ioctl_create_queue(struct file *filep, struct kfd_process *p, void __user *a
struct kfd_dev *dev;
int err = 0;
unsigned int queue_id;
- struct kfd_queue *queue;
struct kfd_process_device *pdd;
+ struct queue_properties q_properties;
+
+ memset(&q_properties, 0, sizeof(struct queue_properties));
if (copy_from_user(&args, arg, sizeof(args)))
return -EFAULT;
- dev = radeon_kfd_device_by_id(args.gpu_id);
- if (dev == NULL)
- return -EINVAL;
+ /* need to validate parameters */
+
+ q_properties.is_interop = false;
+ q_properties.queue_percent = args.queue_percentage;
+ q_properties.priority = args.queue_priority;
+ q_properties.queue_address = args.ring_base_address;
+ q_properties.queue_size = args.ring_size;
- queue = kzalloc(
- offsetof(struct kfd_queue, scheduler_queue) + dev->device_info->scheduler_class->queue_size,
- GFP_KERNEL);
- if (!queue)
- return -ENOMEM;
+ pr_debug("%s Arguments: Queue Percentage (%d, %d)\n"
+ "Queue Priority (%d, %d)\n"
+ "Queue Address (0x%llX, 0x%llX)\n"
+ "Queue Size (%u64, %ll)\n",
+ __func__,
+ q_properties.queue_percent, args.queue_percentage,
+ q_properties.priority, args.queue_priority,
+ q_properties.queue_address, args.ring_base_address,
+ q_properties.queue_size, args.ring_size);
- queue->dev = dev;
+ dev = radeon_kfd_device_by_id(args.gpu_id);
+ if (dev == NULL)
+ return -EINVAL;
mutex_lock(&p->mutex);
@@ -159,23 +172,14 @@ kfd_ioctl_create_queue(struct file *filep, struct kfd_process *p, void __user *a
p->pasid,
dev->id);
- if (!radeon_kfd_allocate_queue_id(p, &queue_id))
- goto err_allocate_queue_id;
-
- err = dev->device_info->scheduler_class->create_queue(dev->scheduler, pdd->scheduler_process,
- &queue->scheduler_queue,
- (void __user *)args.ring_base_address,
- args.ring_size,
- (void __user *)args.read_pointer_address,
- (void __user *)args.write_pointer_address,
- radeon_kfd_queue_id_to_doorbell(dev, p, queue_id));
- if (err)
+ err = pqm_create_queue(&p->pqm, dev, filep, &q_properties, 0, KFD_QUEUE_TYPE_COMPUTE, &queue_id);
+ if (err != 0)
goto err_create_queue;
- radeon_kfd_install_queue(p, queue_id, queue);
-
args.queue_id = queue_id;
- args.doorbell_address = (uint64_t)(uintptr_t)radeon_kfd_get_doorbell(filep, p, dev, queue_id);
+ args.read_pointer_address = (uint64_t)q_properties.read_ptr;
+ args.write_pointer_address = (uint64_t)q_properties.write_ptr;
+ args.doorbell_address = (uint64_t)q_properties.doorbell_ptr;
if (copy_to_user(arg, &args, sizeof(args))) {
err = -EFAULT;
@@ -198,12 +202,9 @@ kfd_ioctl_create_queue(struct file *filep, struct kfd_process *p, void __user *a
return 0;
err_copy_args_out:
- dev->device_info->scheduler_class->destroy_queue(dev->scheduler, &queue->scheduler_queue);
+ pqm_destroy_queue(&p->pqm, queue_id);
err_create_queue:
- radeon_kfd_remove_queue(p, queue_id);
-err_allocate_queue_id:
err_bind_process:
- kfree(queue);
mutex_unlock(&p->mutex);
return err;
}
@@ -211,36 +212,25 @@ err_bind_process:
static int
kfd_ioctl_destroy_queue(struct file *filp, struct kfd_process *p, void __user *arg)
{
+ int retval;
struct kfd_ioctl_destroy_queue_args args;
- struct kfd_queue *queue;
- struct kfd_dev *dev;
if (copy_from_user(&args, arg, sizeof(args)))
return -EFAULT;
- mutex_lock(&p->mutex);
-
- queue = radeon_kfd_get_queue(p, args.queue_id);
- if (!queue) {
- mutex_unlock(&p->mutex);
- return -EINVAL;
- }
-
- dev = queue->dev;
-
pr_debug("kfd: destroying queue id %d for PASID %d\n",
- args.queue_id,
- p->pasid);
+ args.queue_id,
+ p->pasid);
- radeon_kfd_remove_queue(p, args.queue_id);
- dev->device_info->scheduler_class->destroy_queue(dev->scheduler, &queue->scheduler_queue);
+ mutex_lock(&p->mutex);
- kfree(queue);
+ retval = pqm_destroy_queue(&p->pqm, args.queue_id);
mutex_unlock(&p->mutex);
- return 0;
+ return retval;
}
+
static long
kfd_ioctl_set_memory_policy(struct file *filep, struct kfd_process *p, void __user *arg)
{
@@ -281,12 +271,12 @@ kfd_ioctl_set_memory_policy(struct file *filep, struct kfd_process *p, void __us
alternate_policy = (args.alternate_policy == KFD_IOC_CACHE_POLICY_COHERENT)
? cache_policy_coherent : cache_policy_noncoherent;
- if (!dev->device_info->scheduler_class->set_cache_policy(dev->scheduler,
- pdd->scheduler_process,
- default_policy,
- alternate_policy,
- (void __user *)args.alternate_aperture_base,
- args.alternate_aperture_size))
+ if (!dev->dqm->set_cache_memory_policy(dev->dqm,
+ &pdd->qpd,
+ default_policy,
+ alternate_policy,
+ (void __user *)args.alternate_aperture_base,
+ args.alternate_aperture_size))
err = -EINVAL;
out:
@@ -432,11 +422,14 @@ kfd_mmap(struct file *filp, struct vm_area_struct *vma)
if (IS_ERR(process))
return PTR_ERR(process);
- if (pgoff < KFD_MMAP_DOORBELL_START)
- return -EINVAL;
-
- if (pgoff < KFD_MMAP_DOORBELL_END)
+ if (pgoff >= KFD_MMAP_DOORBELL_START && pgoff < KFD_MMAP_DOORBELL_END)
return radeon_kfd_doorbell_mmap(process, vma);
+ if (pgoff >= KFD_MMAP_RPTR_START && pgoff < KFD_MMAP_RPTR_END)
+ return radeon_kfd_hw_pointer_store_mmap(&process->read_ptr, vma);
+
+ if (pgoff >= KFD_MMAP_WPTR_START && pgoff < KFD_MMAP_WPTR_END)
+ return radeon_kfd_hw_pointer_store_mmap(&process->write_ptr, vma);
+
return -EINVAL;
}
diff --git a/drivers/gpu/hsa/radeon/kfd_device.c b/drivers/gpu/hsa/radeon/kfd_device.c
index 82febf4..c602e16 100644
--- a/drivers/gpu/hsa/radeon/kfd_device.c
+++ b/drivers/gpu/hsa/radeon/kfd_device.c
@@ -25,10 +25,9 @@
#include <linux/pci.h>
#include <linux/slab.h>
#include "kfd_priv.h"
-#include "kfd_scheduler.h"
+#include "kfd_device_queue_manager.h"
static const struct kfd_device_info kaveri_device_info = {
- .scheduler_class = &radeon_kfd_cik_static_scheduler_class,
.max_pasid_bits = 16,
.ih_ring_entry_size = 4 * sizeof(uint32_t)
};
@@ -121,7 +120,11 @@ device_iommu_pasid_init(struct kfd_dev *kfd)
}
pasid_limit = min_t(pasid_t, (pasid_t)1 << kfd->device_info->max_pasid_bits, iommu_info.max_pasids);
- pasid_limit = min_t(pasid_t, pasid_limit, kfd->doorbell_process_limit);
+ /*
+ * last pasid is used for kernel queues doorbells
+ * in the future the last pasid might be used for a kernel thread.
+ */
+ pasid_limit = min_t(pasid_t, pasid_limit, kfd->doorbell_process_limit - 1);
err = amd_iommu_init_device(kfd->pdev, pasid_limit);
if (err < 0) {
@@ -168,17 +171,26 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
amd_iommu_set_invalidate_ctx_cb(kfd->pdev, iommu_pasid_shutdown_callback);
- if (kfd->device_info->scheduler_class->create(kfd, &kfd->scheduler)) {
+ kfd->dqm = device_queue_manager_init(kfd);
+ if (!kfd->dqm) {
+ kfd_topology_remove_device(kfd);
amd_iommu_free_device(kfd->pdev);
return false;
}
- kfd->device_info->scheduler_class->start(kfd->scheduler);
+ if (kfd->dqm->start(kfd->dqm) != 0) {
+ device_queue_manager_uninit(kfd->dqm);
+ kfd_topology_remove_device(kfd);
+ amd_iommu_free_device(kfd->pdev);
+ return false;
+ }
kfd->init_complete = true;
dev_info(kfd_device, "added device (%x:%x)\n", kfd->pdev->vendor,
kfd->pdev->device);
+ pr_debug("kfd: Starting kfd with the following scheduling policy %d\n", sched_policy);
+
return true;
}
@@ -188,13 +200,10 @@ void kgd2kfd_device_exit(struct kfd_dev *kfd)
BUG_ON(err != 0);
- if (kfd->init_complete)
- kfd->device_info->scheduler_class->stop(kfd->scheduler);
-
radeon_kfd_interrupt_exit(kfd);
if (kfd->init_complete) {
- kfd->device_info->scheduler_class->destroy(kfd->scheduler);
+ device_queue_manager_uninit(kfd->dqm);
amd_iommu_free_device(kfd->pdev);
}
@@ -206,7 +215,7 @@ void kgd2kfd_suspend(struct kfd_dev *kfd)
BUG_ON(kfd == NULL);
if (kfd->init_complete) {
- kfd->device_info->scheduler_class->stop(kfd->scheduler);
+ kfd->dqm->stop(kfd->dqm);
amd_iommu_free_device(kfd->pdev);
}
}
@@ -225,7 +234,7 @@ int kgd2kfd_resume(struct kfd_dev *kfd)
if (err < 0)
return -ENXIO;
amd_iommu_set_invalidate_ctx_cb(kfd->pdev, iommu_pasid_shutdown_callback);
- kfd->device_info->scheduler_class->start(kfd->scheduler);
+ kfd->dqm->start(kfd->dqm);
}
return 0;
diff --git a/drivers/gpu/hsa/radeon/kfd_interrupt.c b/drivers/gpu/hsa/radeon/kfd_interrupt.c
index 2179780..1c9ad46 100644
--- a/drivers/gpu/hsa/radeon/kfd_interrupt.c
+++ b/drivers/gpu/hsa/radeon/kfd_interrupt.c
@@ -43,7 +43,6 @@
#include <linux/slab.h>
#include <linux/device.h>
#include "kfd_priv.h"
-#include "kfd_scheduler.h"
#define KFD_INTERRUPT_RING_SIZE 256
@@ -162,7 +161,7 @@ static void interrupt_wq(struct work_struct *work)
uint32_t ih_ring_entry[DIV_ROUND_UP(dev->device_info->ih_ring_entry_size, sizeof(uint32_t))];
while (dequeue_ih_ring_entry(dev, ih_ring_entry))
- dev->device_info->scheduler_class->interrupt_wq(dev->scheduler, ih_ring_entry);
+ ;
}
/* This is called directly from KGD at ISR. */
@@ -171,7 +170,6 @@ void kgd2kfd_interrupt(struct kfd_dev *kfd, const void *ih_ring_entry)
spin_lock(&kfd->interrupt_lock);
if (kfd->interrupts_active
- && kfd->device_info->scheduler_class->interrupt_isr(kfd->scheduler, ih_ring_entry)
&& enqueue_ih_ring_entry(kfd, ih_ring_entry))
schedule_work(&kfd->interrupt_work);
diff --git a/drivers/gpu/hsa/radeon/kfd_priv.h b/drivers/gpu/hsa/radeon/kfd_priv.h
index 0af4c71..049671b 100644
--- a/drivers/gpu/hsa/radeon/kfd_priv.h
+++ b/drivers/gpu/hsa/radeon/kfd_priv.h
@@ -441,6 +441,8 @@ void device_queue_manager_uninit(struct device_queue_manager *dqm);
struct kernel_queue *kernel_queue_init(struct kfd_dev *dev, enum kfd_queue_type type);
void kernel_queue_uninit(struct kernel_queue *kq);
+int get_vmid_from_pasid(struct kfd_dev *dev, pasid_t pasid , unsigned int *vmid);
+
/* Process Queue Manager */
struct process_queue_node {
struct queue *q;
diff --git a/drivers/gpu/hsa/radeon/kfd_process.c b/drivers/gpu/hsa/radeon/kfd_process.c
index 80136e6..f967c15 100644
--- a/drivers/gpu/hsa/radeon/kfd_process.c
+++ b/drivers/gpu/hsa/radeon/kfd_process.c
@@ -29,7 +29,6 @@
struct mm_struct;
#include "kfd_priv.h"
-#include "kfd_scheduler.h"
/* Initial size for the array of queues.
* The allocated size is doubled each time it is exceeded up to MAX_PROCESS_QUEUES. */
@@ -91,52 +90,15 @@ radeon_kfd_get_process(const struct task_struct *thread)
return process;
}
-/* Assumes that the kfd_process mutex is held.
- * (Or that it doesn't need to be held because the process is exiting.)
- *
- * dev_filter can be set to only destroy queues for one device.
- * Otherwise all queues for the process are destroyed.
- */
-static void
-destroy_queues(struct kfd_process *p, struct kfd_dev *dev_filter)
-{
- unsigned long queue_id;
-
- for_each_set_bit(queue_id, p->allocated_queue_bitmap, MAX_PROCESS_QUEUES) {
-
- struct kfd_queue *queue = radeon_kfd_get_queue(p, queue_id);
- struct kfd_dev *dev;
-
- BUG_ON(queue == NULL);
-
- dev = queue->dev;
-
- if (!dev_filter || dev == dev_filter) {
- struct kfd_process_device *pdd = radeon_kfd_get_process_device_data(dev, p);
-
- BUG_ON(pdd == NULL); /* A queue exists so pdd must. */
-
- radeon_kfd_remove_queue(p, queue_id);
- dev->device_info->scheduler_class->destroy_queue(dev->scheduler, &queue->scheduler_queue);
-
- kfree(queue);
- }
- }
-}
-
static void free_process(struct kfd_process *p)
{
struct kfd_process_device *pdd, *temp;
BUG_ON(p == NULL);
- destroy_queues(p, NULL);
-
/* doorbell mappings: automatic */
list_for_each_entry_safe(pdd, temp, &p->per_device_data, per_device_list) {
- pdd->dev->device_info->scheduler_class->deregister_process(pdd->dev->scheduler, pdd->scheduler_process);
- pdd->scheduler_process = NULL;
amd_iommu_unbind_pasid(pdd->dev->pdev, p->pasid);
list_del(&pdd->per_device_list);
kfree(pdd);
@@ -202,8 +164,17 @@ static struct kfd_process *create_process(const struct task_struct *thread)
INIT_LIST_HEAD(&process->per_device_data);
+ process->read_ptr.page_mapping = process->write_ptr.page_mapping = NULL;
+ err = pqm_init(&process->pqm, process);
+ if (err != 0)
+ goto err_process_pqm_init;
+
return process;
+err_process_pqm_init:
+ radeon_kfd_pasid_free(process->pasid);
+ list_del(&process->processes_list);
+ thread->mm->kfd_process = NULL;
err_alloc:
kfree(process->queues);
kfree(process);
@@ -222,6 +193,9 @@ radeon_kfd_get_process_device_data(struct kfd_dev *dev, struct kfd_process *p)
pdd = kzalloc(sizeof(*pdd), GFP_KERNEL);
if (pdd != NULL) {
pdd->dev = dev;
+ INIT_LIST_HEAD(&pdd->qpd.queues_list);
+ INIT_LIST_HEAD(&pdd->qpd.priv_queue_list);
+ pdd->qpd.dqm = dev->dqm;
list_add(&pdd->per_device_list, &p->per_device_data);
}
@@ -248,7 +222,6 @@ struct kfd_process_device *radeon_kfd_bind_process_to_device(struct kfd_dev *dev
if (err < 0)
return ERR_PTR(err);
- err = dev->device_info->scheduler_class->register_process(dev->scheduler, p, &pdd->scheduler_process);
if (err < 0) {
amd_iommu_unbind_pasid(dev->pdev, p->pasid);
return ERR_PTR(err);
@@ -282,10 +255,7 @@ void radeon_kfd_unbind_process_from_device(struct kfd_dev *dev, pasid_t pasid)
mutex_lock(&p->mutex);
- destroy_queues(p, dev);
-
- dev->device_info->scheduler_class->deregister_process(dev->scheduler, pdd->scheduler_process);
- pdd->scheduler_process = NULL;
+ pqm_uninit(&p->pqm);
/*
* Just mark pdd as unbound, because we still need it to call
diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h
index e5fcb8b..5134880 100644
--- a/include/uapi/linux/kfd_ioctl.h
+++ b/include/uapi/linux/kfd_ioctl.h
@@ -47,9 +47,9 @@ struct kfd_ioctl_create_queue_args {
uint32_t queue_type; /* to KFD */
uint32_t queue_percentage; /* to KFD */
uint32_t queue_priority; /* to KFD */
- uint64_t write_pointer_address; /* to KFD */
- uint64_t read_pointer_address; /* to KFD */
+ uint64_t write_pointer_address; /* from KFD */
+ uint64_t read_pointer_address; /* from KFD */
uint64_t doorbell_address; /* from KFD */
uint32_t queue_id; /* from KFD */
};
--
1.9.1
^ permalink raw reply related
* [PATCH 44/83] hsa/radeon: HSA64/HSA32 modes support
From: Oded Gabbay @ 2014-07-10 21:54 UTC (permalink / raw)
To: David Airlie, Alex Deucher, Jerome Glisse
Cc: linux-kernel, dri-devel, John Bridgman, Andrew Lewycky,
Joerg Roedel, Alexey Skidanov, Oded Gabbay, Evgeny Pinchuk,
Ben Goz, linux-api
In-Reply-To: <1405029279-6894-1-git-send-email-oded.gabbay@amd.com>
From: Alexey Skidanov <Alexey.Skidanov@amd.com>
Added apertures initialization and appropriate ioctl
Signed-off-by: Alexey Skidanov <Alexey.Skidanov@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
---
drivers/gpu/hsa/radeon/Makefile | 2 +-
drivers/gpu/hsa/radeon/kfd_aperture.c | 124 ++++++++++++++++++++++++++
drivers/gpu/hsa/radeon/kfd_chardev.c | 58 +++++++++++-
drivers/gpu/hsa/radeon/kfd_priv.h | 18 ++++
drivers/gpu/hsa/radeon/kfd_process.c | 17 ++++
drivers/gpu/hsa/radeon/kfd_sched_cik_static.c | 3 +-
drivers/gpu/hsa/radeon/kfd_topology.c | 27 ++++++
include/uapi/linux/kfd_ioctl.h | 18 ++++
8 files changed, 264 insertions(+), 3 deletions(-)
create mode 100644 drivers/gpu/hsa/radeon/kfd_aperture.c
diff --git a/drivers/gpu/hsa/radeon/Makefile b/drivers/gpu/hsa/radeon/Makefile
index 5422e6a..813b31f 100644
--- a/drivers/gpu/hsa/radeon/Makefile
+++ b/drivers/gpu/hsa/radeon/Makefile
@@ -5,6 +5,6 @@
radeon_kfd-y := kfd_module.o kfd_device.o kfd_chardev.o \
kfd_pasid.o kfd_topology.o kfd_process.o \
kfd_doorbell.o kfd_sched_cik_static.o kfd_registers.o \
- kfd_vidmem.o kfd_interrupt.o
+ kfd_vidmem.o kfd_interrupt.o kfd_aperture.o
obj-$(CONFIG_HSA_RADEON) += radeon_kfd.o
diff --git a/drivers/gpu/hsa/radeon/kfd_aperture.c b/drivers/gpu/hsa/radeon/kfd_aperture.c
new file mode 100644
index 0000000..9e2d6da
--- /dev/null
+++ b/drivers/gpu/hsa/radeon/kfd_aperture.c
@@ -0,0 +1,124 @@
+/*
+ * Copyright 2014 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include <linux/device.h>
+#include <linux/export.h>
+#include <linux/err.h>
+#include <linux/fs.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+#include <linux/compat.h>
+#include <uapi/linux/kfd_ioctl.h>
+#include <linux/time.h>
+#include "kfd_priv.h"
+#include "kfd_scheduler.h"
+#include <linux/mm.h>
+#include <uapi/asm-generic/mman-common.h>
+#include <asm/processor.h>
+
+
+#define MAKE_GPUVM_APP_BASE(gpu_num) (((uint64_t)(gpu_num) << 61) + 0x1000000000000)
+#define MAKE_GPUVM_APP_LIMIT(base) (((uint64_t)(base) & 0xFFFFFF0000000000) | 0xFFFFFFFFFF)
+#define MAKE_SCRATCH_APP_BASE(gpu_num) (((uint64_t)(gpu_num) << 61) + 0x100000000)
+#define MAKE_SCRATCH_APP_LIMIT(base) (((uint64_t)base & 0xFFFFFFFF00000000) | 0xFFFFFFFF)
+#define MAKE_LDS_APP_BASE(gpu_num) (((uint64_t)(gpu_num) << 61) + 0x0)
+#define MAKE_LDS_APP_LIMIT(base) (((uint64_t)(base) & 0xFFFFFFFF00000000) | 0xFFFFFFFF)
+
+#define HSA_32BIT_LDS_APP_SIZE 0x10000
+#define HSA_32BIT_LDS_APP_ALIGNMENT 0x10000
+
+static unsigned long kfd_reserve_aperture(struct kfd_process *process, unsigned long len, unsigned long alignment)
+{
+
+ unsigned long addr = 0;
+ unsigned long start_address;
+
+ /*
+ * Go bottom up and find the first available aligned address.
+ * We may narrow space to scan by getting mmap range limits.
+ */
+ for (start_address = alignment; start_address < (TASK_SIZE - alignment); start_address += alignment) {
+ addr = vm_mmap(NULL, start_address, len, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS, 0);
+ if (!IS_ERR_VALUE(addr)) {
+ if (addr == start_address)
+ return addr;
+ vm_munmap(addr, len);
+ }
+ }
+ return 0;
+
+}
+
+int kfd_init_apertures(struct kfd_process *process)
+{
+ uint8_t id = 0;
+ struct kfd_dev *dev;
+ struct kfd_process_device *pdd;
+
+ mutex_lock(&process->mutex);
+
+ /*Iterating over all devices*/
+ while ((dev = kfd_topology_enum_kfd_devices(id)) != NULL && id < NUM_OF_SUPPORTED_GPUS) {
+
+ pdd = radeon_kfd_get_process_device_data(dev, process);
+
+ /*for 64 bit process aperture will be statically reserved in the non canonical process address space
+ *for 32 bit process the aperture will be reserved in the process address space
+ */
+ if (process->is_32bit_user_mode) {
+ /*try to reserve aperture. continue on failure, just put the aperture size to be 0*/
+ pdd->lds_base = kfd_reserve_aperture(
+ process,
+ HSA_32BIT_LDS_APP_SIZE,
+ HSA_32BIT_LDS_APP_ALIGNMENT);
+
+ if (pdd->lds_base)
+ pdd->lds_limit = pdd->lds_base + HSA_32BIT_LDS_APP_SIZE - 1;
+ else
+ pdd->lds_limit = 0;
+
+ /*GPUVM and Scratch apertures are not supported*/
+ pdd->gpuvm_base = pdd->gpuvm_limit = pdd->scratch_base = pdd->scratch_limit = 0;
+ } else {
+ /*node id couldn't be 0 - the three MSB bits of aperture shoudn't be 0*/
+ pdd->lds_base = MAKE_LDS_APP_BASE(id + 1);
+ pdd->lds_limit = MAKE_LDS_APP_LIMIT(pdd->lds_base);
+ pdd->gpuvm_base = MAKE_GPUVM_APP_BASE(id + 1);
+ pdd->gpuvm_limit = MAKE_GPUVM_APP_LIMIT(pdd->gpuvm_base);
+ pdd->scratch_base = MAKE_SCRATCH_APP_BASE(id + 1);
+ pdd->scratch_limit = MAKE_SCRATCH_APP_LIMIT(pdd->scratch_base);
+ }
+
+ dev_dbg(kfd_device, "node id %u, gpu id %u, lds_base %llX lds_limit %llX gpuvm_base %llX gpuvm_limit %llX scratch_base %llX scratch_limit %llX",
+ id, pdd->dev->id, pdd->lds_base, pdd->lds_limit, pdd->gpuvm_base, pdd->gpuvm_limit, pdd->scratch_base, pdd->scratch_limit);
+
+ id++;
+ }
+
+ mutex_unlock(&process->mutex);
+
+ return 0;
+}
+
+
diff --git a/drivers/gpu/hsa/radeon/kfd_chardev.c b/drivers/gpu/hsa/radeon/kfd_chardev.c
index e95d597..07cac88 100644
--- a/drivers/gpu/hsa/radeon/kfd_chardev.c
+++ b/drivers/gpu/hsa/radeon/kfd_chardev.c
@@ -32,6 +32,9 @@
#include <linux/time.h>
#include "kfd_priv.h"
#include "kfd_scheduler.h"
+#include <linux/mm.h>
+#include <uapi/asm-generic/mman-common.h>
+#include <asm/processor.h>
static long kfd_ioctl(struct file *, unsigned int, unsigned long);
static int kfd_open(struct inode *, struct file *);
@@ -107,9 +110,13 @@ kfd_open(struct inode *inode, struct file *filep)
process = radeon_kfd_create_process(current);
if (IS_ERR(process))
return PTR_ERR(process);
+
process->is_32bit_user_mode = is_compat_task();
+
dev_info(kfd_device, "process %d opened, compat mode (32 bit) - %d\n",
- process->pasid, process->is_32bit_user_mode);
+ process->pasid, process->is_32bit_user_mode);
+
+ kfd_init_apertures(process);
return 0;
}
@@ -321,6 +328,51 @@ kfd_ioctl_get_clock_counters(struct file *filep, struct kfd_process *p, void __u
return 0;
}
+
+static int kfd_ioctl_get_process_apertures(struct file *filp, struct kfd_process *p, void __user *arg)
+{
+ struct kfd_ioctl_get_process_apertures_args args;
+ struct kfd_process_device *pdd;
+
+ dev_dbg(kfd_device, "get apertures for PASID %d", p->pasid);
+
+ if (copy_from_user(&args, arg, sizeof(args)))
+ return -EFAULT;
+
+ args.num_of_nodes = 0;
+
+ mutex_lock(&p->mutex);
+
+ /*if the process-device list isn't empty*/
+ if (kfd_has_process_device_data(p)) {
+ /* Run over all pdd of the process */
+ pdd = kfd_get_first_process_device_data(p);
+ do {
+
+ args.process_apertures[args.num_of_nodes].gpu_id = pdd->dev->id;
+ args.process_apertures[args.num_of_nodes].lds_base = pdd->lds_base;
+ args.process_apertures[args.num_of_nodes].lds_limit = pdd->lds_limit;
+ args.process_apertures[args.num_of_nodes].gpuvm_base = pdd->gpuvm_base;
+ args.process_apertures[args.num_of_nodes].gpuvm_limit = pdd->gpuvm_limit;
+ args.process_apertures[args.num_of_nodes].scratch_base = pdd->scratch_base;
+ args.process_apertures[args.num_of_nodes].scratch_limit = pdd->scratch_limit;
+
+ dev_dbg(kfd_device, "node id %u, gpu id %u, lds_base %llX lds_limit %llX gpuvm_base %llX gpuvm_limit %llX scratch_base %llX scratch_limit %llX",
+ args.num_of_nodes, pdd->dev->id, pdd->lds_base, pdd->lds_limit, pdd->gpuvm_base, pdd->gpuvm_limit, pdd->scratch_base, pdd->scratch_limit);
+ args.num_of_nodes++;
+ } while ((pdd = kfd_get_next_process_device_data(p, pdd)) != NULL &&
+ (args.num_of_nodes < NUM_OF_SUPPORTED_GPUS));
+ }
+
+ mutex_unlock(&p->mutex);
+
+ if (copy_to_user(arg, &args, sizeof(args)))
+ return -EFAULT;
+
+ return 0;
+}
+
+
static long
kfd_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)
{
@@ -352,6 +404,10 @@ kfd_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)
err = kfd_ioctl_get_clock_counters(filep, process, (void __user *)arg);
break;
+ case KFD_IOC_GET_PROCESS_APERTURES:
+ err = kfd_ioctl_get_process_apertures(filep, process, (void __user *)arg);
+ break;
+
default:
dev_err(kfd_device,
"unknown ioctl cmd 0x%x, arg 0x%lx)\n",
diff --git a/drivers/gpu/hsa/radeon/kfd_priv.h b/drivers/gpu/hsa/radeon/kfd_priv.h
index 9d3b1fc..28155bc 100644
--- a/drivers/gpu/hsa/radeon/kfd_priv.h
+++ b/drivers/gpu/hsa/radeon/kfd_priv.h
@@ -171,6 +171,16 @@ struct kfd_process_device {
/* Is this process/pasid bound to this device? (amd_iommu_bind_pasid) */
bool bound;
+
+ /*Apertures*/
+ uint64_t lds_base;
+ uint64_t lds_limit;
+ uint64_t gpuvm_base;
+ uint64_t gpuvm_limit;
+ uint64_t scratch_base;
+ uint64_t scratch_limit;
+
+
};
/* Process data */
@@ -212,6 +222,10 @@ void radeon_kfd_install_queue(struct kfd_process *p, unsigned int queue_id, stru
void radeon_kfd_remove_queue(struct kfd_process *p, unsigned int queue_id);
struct kfd_queue *radeon_kfd_get_queue(struct kfd_process *p, unsigned int queue_id);
+/* Process device data iterator */
+struct kfd_process_device *kfd_get_first_process_device_data(struct kfd_process *p);
+struct kfd_process_device *kfd_get_next_process_device_data(struct kfd_process *p, struct kfd_process_device *pdd);
+bool kfd_has_process_device_data(struct kfd_process *p);
/* PASIDs */
int radeon_kfd_pasid_init(void);
@@ -237,6 +251,7 @@ int kfd_topology_add_device(struct kfd_dev *gpu);
int kfd_topology_remove_device(struct kfd_dev *gpu);
struct kfd_dev *radeon_kfd_device_by_id(uint32_t gpu_id);
struct kfd_dev *radeon_kfd_device_by_pci_dev(const struct pci_dev *pdev);
+struct kfd_dev *kfd_topology_enum_kfd_devices(uint8_t idx);
/* MMIO registers */
#define WRITE_REG(dev, reg, value) radeon_kfd_write_reg((dev), (reg), (value))
@@ -253,4 +268,7 @@ void kgd2kfd_interrupt(struct kfd_dev *dev, const void *ih_ring_entry);
void kgd2kfd_suspend(struct kfd_dev *dev);
int kgd2kfd_resume(struct kfd_dev *dev);
+/*HSA apertures*/
+int kfd_init_apertures(struct kfd_process *process);
+
#endif
diff --git a/drivers/gpu/hsa/radeon/kfd_process.c b/drivers/gpu/hsa/radeon/kfd_process.c
index f89f855..80136e6 100644
--- a/drivers/gpu/hsa/radeon/kfd_process.c
+++ b/drivers/gpu/hsa/radeon/kfd_process.c
@@ -397,3 +397,20 @@ struct kfd_queue *radeon_kfd_get_queue(struct kfd_process *p, unsigned int queue
test_bit(queue_id, p->allocated_queue_bitmap)) ?
p->queues[queue_id] : NULL;
}
+
+struct kfd_process_device *kfd_get_first_process_device_data(struct kfd_process *p)
+{
+ return list_first_entry(&p->per_device_data, struct kfd_process_device, per_device_list);
+}
+
+struct kfd_process_device *kfd_get_next_process_device_data(struct kfd_process *p, struct kfd_process_device *pdd)
+{
+ if (list_is_last(&pdd->per_device_list, &p->per_device_data))
+ return NULL;
+ return list_next_entry(pdd, per_device_list);
+}
+
+bool kfd_has_process_device_data(struct kfd_process *p)
+{
+ return !(list_empty(&p->per_device_data));
+}
diff --git a/drivers/gpu/hsa/radeon/kfd_sched_cik_static.c b/drivers/gpu/hsa/radeon/kfd_sched_cik_static.c
index 7ee8125..30561a6 100644
--- a/drivers/gpu/hsa/radeon/kfd_sched_cik_static.c
+++ b/drivers/gpu/hsa/radeon/kfd_sched_cik_static.c
@@ -627,7 +627,8 @@ static void cik_static_deregister_process(struct kfd_scheduler *scheduler,
struct cik_static_private *priv = kfd_scheduler_to_private(scheduler);
struct cik_static_process *pp = kfd_process_to_private(scheduler_process);
- if (priv && pp) {
+
+ if (priv && pp) {
release_vmid(priv, pp->vmid);
kfree(pp);
}
diff --git a/drivers/gpu/hsa/radeon/kfd_topology.c b/drivers/gpu/hsa/radeon/kfd_topology.c
index 21bb66e..213ae7b 100644
--- a/drivers/gpu/hsa/radeon/kfd_topology.c
+++ b/drivers/gpu/hsa/radeon/kfd_topology.c
@@ -1201,3 +1201,30 @@ int kfd_topology_remove_device(struct kfd_dev *gpu)
return res;
}
+
+/*
+ * When idx is out of bounds, the function will return NULL
+ */
+struct kfd_dev *kfd_topology_enum_kfd_devices(uint8_t idx)
+{
+
+ struct kfd_topology_device *top_dev;
+ struct kfd_dev *device = NULL;
+ uint8_t device_idx = 0;
+
+ down_read(&topology_lock);
+
+ list_for_each_entry(top_dev, &topology_device_list, list) {
+ if (device_idx == idx) {
+ device = top_dev->gpu;
+ break;
+ }
+
+ device_idx++;
+ }
+
+ up_read(&topology_lock);
+
+ return device;
+
+}
diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h
index a7c3abd..e5fcb8b 100644
--- a/include/uapi/linux/kfd_ioctl.h
+++ b/include/uapi/linux/kfd_ioctl.h
@@ -78,6 +78,23 @@ struct kfd_ioctl_get_clock_counters_args {
uint64_t system_clock_freq; /* from KFD */
};
+#define NUM_OF_SUPPORTED_GPUS 7
+
+struct kfd_process_device_apertures {
+ uint64_t lds_base;/* from KFD */
+ uint64_t lds_limit;/* from KFD */
+ uint64_t scratch_base;/* from KFD */
+ uint64_t scratch_limit;/* from KFD */
+ uint64_t gpuvm_base;/* from KFD */
+ uint64_t gpuvm_limit;/* from KFD */
+ uint32_t gpu_id;/* from KFD */
+};
+
+struct kfd_ioctl_get_process_apertures_args {
+ struct kfd_process_device_apertures process_apertures[NUM_OF_SUPPORTED_GPUS];/* from KFD */
+ uint8_t num_of_nodes; /* from KFD, should be in the range [1 - NUM_OF_SUPPORTED_GPUS]*/
+};
+
#define KFD_IOC_MAGIC 'K'
#define KFD_IOC_GET_VERSION _IOR(KFD_IOC_MAGIC, 1, struct kfd_ioctl_get_version_args)
@@ -85,6 +102,7 @@ struct kfd_ioctl_get_clock_counters_args {
#define KFD_IOC_DESTROY_QUEUE _IOWR(KFD_IOC_MAGIC, 3, struct kfd_ioctl_destroy_queue_args)
#define KFD_IOC_SET_MEMORY_POLICY _IOW(KFD_IOC_MAGIC, 4, struct kfd_ioctl_set_memory_policy_args)
#define KFD_IOC_GET_CLOCK_COUNTERS _IOWR(KFD_IOC_MAGIC, 5, struct kfd_ioctl_get_clock_counters_args)
+#define KFD_IOC_GET_PROCESS_APERTURES _IOR(KFD_IOC_MAGIC, 6, struct kfd_ioctl_get_process_apertures_args)
#pragma pack(pop)
--
1.9.1
^ permalink raw reply related
* [PATCH 42/83] hsa/radeon: 32-bit processes support
From: Oded Gabbay @ 2014-07-10 21:53 UTC (permalink / raw)
To: David Airlie, Alex Deucher, Jerome Glisse
Cc: linux-kernel, dri-devel, John Bridgman, Andrew Lewycky,
Joerg Roedel, Alexey Skidanov, Oded Gabbay, Ben Goz,
Evgeny Pinchuk, linux-api
In-Reply-To: <1405029279-6894-1-git-send-email-oded.gabbay@amd.com>
From: Alexey Skidanov <Alexey.Skidanov@amd.com>
Initializing compat_ioctl properly. All ioctls args are packed.
Signed-off-by: Alexey Skidanov <Alexey.Skidanov@amd.com>
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
---
drivers/gpu/hsa/radeon/kfd_chardev.c | 7 +++++--
drivers/gpu/hsa/radeon/kfd_priv.h | 4 ++++
include/uapi/linux/kfd_ioctl.h | 2 +-
3 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/hsa/radeon/kfd_chardev.c b/drivers/gpu/hsa/radeon/kfd_chardev.c
index 75fe11f..e95d597 100644
--- a/drivers/gpu/hsa/radeon/kfd_chardev.c
+++ b/drivers/gpu/hsa/radeon/kfd_chardev.c
@@ -27,6 +27,7 @@
#include <linux/sched.h>
#include <linux/slab.h>
#include <linux/uaccess.h>
+#include <linux/compat.h>
#include <uapi/linux/kfd_ioctl.h>
#include <linux/time.h>
#include "kfd_priv.h"
@@ -41,6 +42,7 @@ static const char kfd_dev_name[] = "kfd";
static const struct file_operations kfd_fops = {
.owner = THIS_MODULE,
.unlocked_ioctl = kfd_ioctl,
+ .compat_ioctl = kfd_ioctl,
.open = kfd_open,
.mmap = kfd_mmap,
};
@@ -105,8 +107,9 @@ kfd_open(struct inode *inode, struct file *filep)
process = radeon_kfd_create_process(current);
if (IS_ERR(process))
return PTR_ERR(process);
-
- pr_debug("\nkfd: process %d opened dev/kfd", process->pasid);
+ process->is_32bit_user_mode = is_compat_task();
+ dev_info(kfd_device, "process %d opened, compat mode (32 bit) - %d\n",
+ process->pasid, process->is_32bit_user_mode);
return 0;
}
diff --git a/drivers/gpu/hsa/radeon/kfd_priv.h b/drivers/gpu/hsa/radeon/kfd_priv.h
index 8b877ca..9d3b1fc 100644
--- a/drivers/gpu/hsa/radeon/kfd_priv.h
+++ b/drivers/gpu/hsa/radeon/kfd_priv.h
@@ -194,6 +194,10 @@ struct kfd_process {
size_t queue_array_size;
struct kfd_queue **queues; /* Size is queue_array_size, up to MAX_PROCESS_QUEUES. */
unsigned long allocated_queue_bitmap[DIV_ROUND_UP(MAX_PROCESS_QUEUES, BITS_PER_LONG)];
+
+ /*Is the user space process 32 bit?*/
+ bool is_32bit_user_mode;
+
};
struct kfd_process *radeon_kfd_create_process(const struct task_struct *);
diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h
index 5b9517e..a7c3abd 100644
--- a/include/uapi/linux/kfd_ioctl.h
+++ b/include/uapi/linux/kfd_ioctl.h
@@ -29,7 +29,7 @@
#define KFD_IOCTL_CURRENT_VERSION 1
/* The 64-bit ABI is the authoritative version. */
-#pragma pack(push, 8)
+#pragma pack(push, 1)
struct kfd_ioctl_get_version_args {
uint32_t min_supported_version; /* from KFD */
--
1.9.1
^ permalink raw reply related
* [PATCH 32/83] hsa/radeon: implementing IOCTL for clock counters
From: Oded Gabbay @ 2014-07-10 21:53 UTC (permalink / raw)
To: David Airlie, Alex Deucher, Jerome Glisse
Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, John Bridgman,
Andrew Lewycky, Joerg Roedel, Evgeny Pinchuk, Oded Gabbay,
Ben Goz, Alexey Skidanov, linux-api-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <1405029279-6894-1-git-send-email-oded.gabbay-5C7GfCeVMHo@public.gmane.org>
From: Evgeny Pinchuk <evgeny.pinchuk-5C7GfCeVMHo@public.gmane.org>
Implemented new IOCTL to query the CPU and GPU clock counters.
Signed-off-by: Evgeny Pinchuk <evgeny.pinchuk-5C7GfCeVMHo@public.gmane.org>
Signed-off-by: Oded Gabbay <oded.gabbay-5C7GfCeVMHo@public.gmane.org>
---
drivers/gpu/hsa/radeon/kfd_chardev.c | 37 ++++++++++++++++++++++++++++++++++++
include/uapi/linux/kfd_ioctl.h | 9 +++++++++
2 files changed, 46 insertions(+)
diff --git a/drivers/gpu/hsa/radeon/kfd_chardev.c b/drivers/gpu/hsa/radeon/kfd_chardev.c
index ddaf357..d6fa980 100644
--- a/drivers/gpu/hsa/radeon/kfd_chardev.c
+++ b/drivers/gpu/hsa/radeon/kfd_chardev.c
@@ -28,6 +28,7 @@
#include <linux/slab.h>
#include <linux/uaccess.h>
#include <uapi/linux/kfd_ioctl.h>
+#include <linux/time.h>
#include "kfd_priv.h"
#include "kfd_scheduler.h"
@@ -284,6 +285,38 @@ out:
return err;
}
+static long
+kfd_ioctl_get_clock_counters(struct file *filep, struct kfd_process *p, void __user *arg)
+{
+ struct kfd_ioctl_get_clock_counters_args args;
+ struct kfd_dev *dev;
+ struct timespec time;
+
+ if (copy_from_user(&args, arg, sizeof(args)))
+ return -EFAULT;
+
+ dev = radeon_kfd_device_by_id(args.gpu_id);
+ if (dev == NULL)
+ return -EINVAL;
+
+ /* Reading GPU clock counter from KGD */
+ args.gpu_clock_counter = kfd2kgd->get_gpu_clock_counter(dev->kgd);
+
+ /* No access to rdtsc. Using raw monotonic time */
+ getrawmonotonic(&time);
+ args.cpu_clock_counter = time.tv_nsec;
+
+ get_monotonic_boottime(&time);
+ args.system_clock_counter = time.tv_nsec;
+
+ /* Since the counter is in nano-seconds we use 1GHz frequency */
+ args.system_clock_freq = 1000000000;
+
+ if (copy_to_user(arg, &args, sizeof(args)))
+ return -EFAULT;
+
+ return 0;
+}
static long
kfd_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)
@@ -312,6 +345,10 @@ kfd_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)
err = kfd_ioctl_set_memory_policy(filep, process, (void __user *)arg);
break;
+ case KFD_IOC_GET_CLOCK_COUNTERS:
+ err = kfd_ioctl_get_clock_counters(filep, process, (void __user *)arg);
+ break;
+
default:
dev_err(kfd_device,
"unknown ioctl cmd 0x%x, arg 0x%lx)\n",
diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h
index 928e628..5b9517e 100644
--- a/include/uapi/linux/kfd_ioctl.h
+++ b/include/uapi/linux/kfd_ioctl.h
@@ -70,12 +70,21 @@ struct kfd_ioctl_set_memory_policy_args {
uint64_t alternate_aperture_size; /* to KFD */
};
+struct kfd_ioctl_get_clock_counters_args {
+ uint32_t gpu_id; /* to KFD */
+ uint64_t gpu_clock_counter; /* from KFD */
+ uint64_t cpu_clock_counter; /* from KFD */
+ uint64_t system_clock_counter; /* from KFD */
+ uint64_t system_clock_freq; /* from KFD */
+};
+
#define KFD_IOC_MAGIC 'K'
#define KFD_IOC_GET_VERSION _IOR(KFD_IOC_MAGIC, 1, struct kfd_ioctl_get_version_args)
#define KFD_IOC_CREATE_QUEUE _IOWR(KFD_IOC_MAGIC, 2, struct kfd_ioctl_create_queue_args)
#define KFD_IOC_DESTROY_QUEUE _IOWR(KFD_IOC_MAGIC, 3, struct kfd_ioctl_destroy_queue_args)
#define KFD_IOC_SET_MEMORY_POLICY _IOW(KFD_IOC_MAGIC, 4, struct kfd_ioctl_set_memory_policy_args)
+#define KFD_IOC_GET_CLOCK_COUNTERS _IOWR(KFD_IOC_MAGIC, 5, struct kfd_ioctl_get_clock_counters_args)
#pragma pack(pop)
--
1.9.1
^ permalink raw reply related
* [PATCH 27/83] hsa/radeon: Implement hsaKmtSetMemoryPolicy
From: Oded Gabbay @ 2014-07-10 21:50 UTC (permalink / raw)
To: David Airlie, Alex Deucher, Jerome Glisse
Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, John Bridgman,
Andrew Lewycky, Joerg Roedel, Oded Gabbay, Ben Goz,
Evgeny Pinchuk, Alexey Skidanov, linux-api-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <1405029027-6085-1-git-send-email-oded.gabbay-5C7GfCeVMHo@public.gmane.org>
From: Andrew Lewycky <Andrew.Lewycky-5C7GfCeVMHo@public.gmane.org>
This patch adds support in KFD for the hsaKmtSetMemoryPolicy
HSA thunk API call
Signed-off-by: Andrew Lewycky <Andrew.Lewycky-5C7GfCeVMHo@public.gmane.org>
Signed-off-by: Oded Gabbay <oded.gabbay-5C7GfCeVMHo@public.gmane.org>
---
drivers/gpu/hsa/radeon/cik_regs.h | 1 +
drivers/gpu/hsa/radeon/kfd_chardev.c | 59 +++++++++++++++++
drivers/gpu/hsa/radeon/kfd_sched_cik_static.c | 91 +++++++++++++++++++++++++--
drivers/gpu/hsa/radeon/kfd_scheduler.h | 12 ++++
include/uapi/linux/kfd_ioctl.h | 13 ++++
5 files changed, 172 insertions(+), 4 deletions(-)
diff --git a/drivers/gpu/hsa/radeon/cik_regs.h b/drivers/gpu/hsa/radeon/cik_regs.h
index 813cdc4..93f7b34 100644
--- a/drivers/gpu/hsa/radeon/cik_regs.h
+++ b/drivers/gpu/hsa/radeon/cik_regs.h
@@ -54,6 +54,7 @@
#define APE1_MTYPE(x) ((x) << 7)
/* valid for both DEFAULT_MTYPE and APE1_MTYPE */
+#define MTYPE_CACHED 0
#define MTYPE_NONCACHED 3
diff --git a/drivers/gpu/hsa/radeon/kfd_chardev.c b/drivers/gpu/hsa/radeon/kfd_chardev.c
index e0b276d..ddaf357 100644
--- a/drivers/gpu/hsa/radeon/kfd_chardev.c
+++ b/drivers/gpu/hsa/radeon/kfd_chardev.c
@@ -231,6 +231,61 @@ kfd_ioctl_destroy_queue(struct file *filp, struct kfd_process *p, void __user *a
}
static long
+kfd_ioctl_set_memory_policy(struct file *filep, struct kfd_process *p, void __user *arg)
+{
+ struct kfd_ioctl_set_memory_policy_args args;
+ struct kfd_dev *dev;
+ int err = 0;
+ struct kfd_process_device *pdd;
+ enum cache_policy default_policy, alternate_policy;
+
+ if (copy_from_user(&args, arg, sizeof(args)))
+ return -EFAULT;
+
+ if (args.default_policy != KFD_IOC_CACHE_POLICY_COHERENT
+ && args.default_policy != KFD_IOC_CACHE_POLICY_NONCOHERENT) {
+ return -EINVAL;
+ }
+
+ if (args.alternate_policy != KFD_IOC_CACHE_POLICY_COHERENT
+ && args.alternate_policy != KFD_IOC_CACHE_POLICY_NONCOHERENT) {
+ return -EINVAL;
+ }
+
+ dev = radeon_kfd_device_by_id(args.gpu_id);
+ if (dev == NULL)
+ return -EINVAL;
+
+ mutex_lock(&p->mutex);
+
+ pdd = radeon_kfd_bind_process_to_device(dev, p);
+ if (IS_ERR(pdd) < 0) {
+ err = PTR_ERR(pdd);
+ goto out;
+ }
+
+ default_policy = (args.default_policy == KFD_IOC_CACHE_POLICY_COHERENT)
+ ? cache_policy_coherent : cache_policy_noncoherent;
+
+ alternate_policy = (args.alternate_policy == KFD_IOC_CACHE_POLICY_COHERENT)
+ ? cache_policy_coherent : cache_policy_noncoherent;
+
+ if (!dev->device_info->scheduler_class->set_cache_policy(dev->scheduler,
+ pdd->scheduler_process,
+ default_policy,
+ alternate_policy,
+ (void __user *)args.alternate_aperture_base,
+ args.alternate_aperture_size))
+ err = -EINVAL;
+
+out:
+ mutex_unlock(&p->mutex);
+
+ return err;
+}
+
+
+static long
kfd_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)
{
struct kfd_process *process;
@@ -253,6 +308,10 @@ kfd_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)
err = kfd_ioctl_destroy_queue(filep, process, (void __user *)arg);
break;
+ case KFD_IOC_SET_MEMORY_POLICY:
+ err = kfd_ioctl_set_memory_policy(filep, process, (void __user *)arg);
+ break;
+
default:
dev_err(kfd_device,
"unknown ioctl cmd 0x%x, arg 0x%lx)\n",
diff --git a/drivers/gpu/hsa/radeon/kfd_sched_cik_static.c b/drivers/gpu/hsa/radeon/kfd_sched_cik_static.c
index 9add5e5..3c3e7d6 100644
--- a/drivers/gpu/hsa/radeon/kfd_sched_cik_static.c
+++ b/drivers/gpu/hsa/radeon/kfd_sched_cik_static.c
@@ -162,6 +162,10 @@ struct cik_static_private {
struct cik_static_process {
unsigned int vmid;
pasid_t pasid;
+
+ uint32_t sh_mem_config;
+ uint32_t ape1_base;
+ uint32_t ape1_limit;
};
struct cik_static_queue {
@@ -346,6 +350,7 @@ static void init_ats(struct cik_static_private *priv)
sh_mem_config = ALIGNMENT_MODE(SH_MEM_ALIGNMENT_MODE_UNALIGNED);
sh_mem_config |= DEFAULT_MTYPE(MTYPE_NONCACHED);
+ sh_mem_config |= APE1_MTYPE(MTYPE_NONCACHED);
WRITE_REG(priv->dev, SH_MEM_CONFIG, sh_mem_config);
@@ -562,14 +567,26 @@ static void release_vmid(struct cik_static_private *priv, unsigned int vmid)
set_bit(vmid, &priv->free_vmid_mask);
}
+static void program_sh_mem_settings(struct cik_static_private *sched,
+ struct cik_static_process *proc)
+{
+ lock_srbm_index(sched);
+
+ vmid_select(sched, proc->vmid);
+
+ WRITE_REG(sched->dev, SH_MEM_CONFIG, proc->sh_mem_config);
+
+ WRITE_REG(sched->dev, SH_MEM_APE1_BASE, proc->ape1_base);
+ WRITE_REG(sched->dev, SH_MEM_APE1_LIMIT, proc->ape1_limit);
+
+ unlock_srbm_index(sched);
+}
+
static void setup_vmid_for_process(struct cik_static_private *priv, struct cik_static_process *p)
{
set_vmid_pasid_mapping(priv, p->vmid, p->pasid);
- /*
- * SH_MEM_CONFIG and others need to be programmed differently
- * for 32/64-bit processes. And maybe other reasons.
- */
+ program_sh_mem_settings(priv, p);
}
static int
@@ -591,6 +608,12 @@ cik_static_register_process(struct kfd_scheduler *scheduler, struct kfd_process
hwp->pasid = process->pasid;
+ hwp->sh_mem_config = ALIGNMENT_MODE(SH_MEM_ALIGNMENT_MODE_UNALIGNED)
+ | DEFAULT_MTYPE(MTYPE_NONCACHED)
+ | APE1_MTYPE(MTYPE_NONCACHED);
+ hwp->ape1_base = 1;
+ hwp->ape1_limit = 0;
+
setup_vmid_for_process(priv, hwp);
*scheduler_process = (struct kfd_scheduler_process *)hwp;
@@ -894,6 +917,64 @@ cik_static_interrupt_wq(struct kfd_scheduler *scheduler, const void *ih_ring_ent
{
}
+/* Low bits must be 0000/FFFF as required by HW, high bits must be 0 to stay in user mode. */
+#define APE1_FIXED_BITS_MASK 0xFFFF80000000FFFFULL
+#define APE1_LIMIT_ALIGNMENT 0xFFFF /* APE1 limit is inclusive and 64K aligned. */
+
+static bool cik_static_set_cache_policy(struct kfd_scheduler *scheduler,
+ struct kfd_scheduler_process *process,
+ enum cache_policy default_policy,
+ enum cache_policy alternate_policy,
+ void __user *alternate_aperture_base,
+ uint64_t alternate_aperture_size)
+{
+ struct cik_static_private *sched = kfd_scheduler_to_private(scheduler);
+ struct cik_static_process *proc = kfd_process_to_private(process);
+
+ uint32_t default_mtype;
+ uint32_t ape1_mtype;
+
+ if (alternate_aperture_size == 0) {
+ /* base > limit disables APE1 */
+ proc->ape1_base = 1;
+ proc->ape1_limit = 0;
+ } else {
+ /*
+ * In FSA64, APE1_Base[63:0] = { 16{SH_MEM_APE1_BASE[31]}, SH_MEM_APE1_BASE[31:0], 0x0000 }
+ * APE1_Limit[63:0] = { 16{SH_MEM_APE1_LIMIT[31]}, SH_MEM_APE1_LIMIT[31:0], 0xFFFF }
+ * Verify that the base and size parameters can be represented in this format
+ * and convert them. Additionally restrict APE1 to user-mode addresses.
+ */
+
+ uint64_t base = (uintptr_t)alternate_aperture_base;
+ uint64_t limit = base + alternate_aperture_size - 1;
+
+ if (limit <= base)
+ return false;
+
+ if ((base & APE1_FIXED_BITS_MASK) != 0)
+ return false;
+
+ if ((limit & APE1_FIXED_BITS_MASK) != APE1_LIMIT_ALIGNMENT)
+ return false;
+
+ proc->ape1_base = base >> 16;
+ proc->ape1_limit = limit >> 16;
+ }
+
+ default_mtype = (default_policy == cache_policy_coherent) ? MTYPE_NONCACHED : MTYPE_CACHED;
+ ape1_mtype = (alternate_policy == cache_policy_coherent) ? MTYPE_NONCACHED : MTYPE_CACHED;
+
+ proc->sh_mem_config = ALIGNMENT_MODE(SH_MEM_ALIGNMENT_MODE_UNALIGNED)
+ | DEFAULT_MTYPE(default_mtype)
+ | APE1_MTYPE(ape1_mtype);
+
+ program_sh_mem_settings(sched, proc);
+
+ return true;
+}
+
+
const struct kfd_scheduler_class radeon_kfd_cik_static_scheduler_class = {
.name = "CIK static scheduler",
.create = cik_static_create,
@@ -908,4 +989,6 @@ const struct kfd_scheduler_class radeon_kfd_cik_static_scheduler_class = {
.interrupt_isr = cik_static_interrupt_isr,
.interrupt_wq = cik_static_interrupt_wq,
+
+ .set_cache_policy = cik_static_set_cache_policy,
};
diff --git a/drivers/gpu/hsa/radeon/kfd_scheduler.h b/drivers/gpu/hsa/radeon/kfd_scheduler.h
index e5a93c4..9dc2994 100644
--- a/drivers/gpu/hsa/radeon/kfd_scheduler.h
+++ b/drivers/gpu/hsa/radeon/kfd_scheduler.h
@@ -31,6 +31,11 @@ struct kfd_scheduler;
struct kfd_scheduler_process;
struct kfd_scheduler_queue;
+enum cache_policy {
+ cache_policy_coherent,
+ cache_policy_noncoherent
+};
+
struct kfd_scheduler_class {
const char *name;
@@ -58,6 +63,13 @@ struct kfd_scheduler_class {
bool (*interrupt_isr)(struct kfd_scheduler *, const void *ih_ring_entry);
void (*interrupt_wq)(struct kfd_scheduler *, const void *ih_ring_entry);
+
+ bool (*set_cache_policy)(struct kfd_scheduler *scheduler,
+ struct kfd_scheduler_process *process,
+ enum cache_policy default_policy,
+ enum cache_policy alternate_policy,
+ void __user *alternate_aperture_base,
+ uint64_t alternate_aperture_size);
};
extern const struct kfd_scheduler_class radeon_kfd_cik_static_scheduler_class;
diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h
index dcc5fe0..928e628 100644
--- a/include/uapi/linux/kfd_ioctl.h
+++ b/include/uapi/linux/kfd_ioctl.h
@@ -58,11 +58,24 @@ struct kfd_ioctl_destroy_queue_args {
uint32_t queue_id; /* to KFD */
};
+/* For kfd_ioctl_set_memory_policy_args.default_policy and alternate_policy */
+#define KFD_IOC_CACHE_POLICY_COHERENT 0
+#define KFD_IOC_CACHE_POLICY_NONCOHERENT 1
+
+struct kfd_ioctl_set_memory_policy_args {
+ uint32_t gpu_id; /* to KFD */
+ uint32_t default_policy; /* to KFD */
+ uint32_t alternate_policy; /* to KFD */
+ uint64_t alternate_aperture_base; /* to KFD */
+ uint64_t alternate_aperture_size; /* to KFD */
+};
+
#define KFD_IOC_MAGIC 'K'
#define KFD_IOC_GET_VERSION _IOR(KFD_IOC_MAGIC, 1, struct kfd_ioctl_get_version_args)
#define KFD_IOC_CREATE_QUEUE _IOWR(KFD_IOC_MAGIC, 2, struct kfd_ioctl_create_queue_args)
#define KFD_IOC_DESTROY_QUEUE _IOWR(KFD_IOC_MAGIC, 3, struct kfd_ioctl_destroy_queue_args)
+#define KFD_IOC_SET_MEMORY_POLICY _IOW(KFD_IOC_MAGIC, 4, struct kfd_ioctl_set_memory_policy_args)
#pragma pack(pop)
--
1.9.1
^ permalink raw reply related
* [PATCH 13/83] hsa/radeon: Add 2 new IOCTL to kfd, CREATE_QUEUE and DESTROY_QUEUE
From: Oded Gabbay @ 2014-07-10 21:50 UTC (permalink / raw)
To: David Airlie, Alex Deucher, Jerome Glisse
Cc: linux-kernel, dri-devel, John Bridgman, Andrew Lewycky,
Joerg Roedel, Oded Gabbay, Alexey Skidanov, Ben Goz,
Evgeny Pinchuk, linux-api
In-Reply-To: <1405029027-6085-1-git-send-email-oded.gabbay@amd.com>
This patch adds 2 new IOCTL to kfd driver.
The first IOCTL is KFD_IOC_CREATE_QUEUE that is used by the user-mode
application to create a compute queue on the GPU.
The second IOCTL is KFD_IOC_DESTROY_QUEUE that is used by the
user-mode application to destroy an existing compute queue on the GPU.
Signed-off-by: Oded Gabbay <oded.gabbay@amd.com>
---
drivers/gpu/hsa/radeon/kfd_chardev.c | 155 ++++++++++++++++++++++++++++++++++
drivers/gpu/hsa/radeon/kfd_doorbell.c | 11 +++
include/uapi/linux/kfd_ioctl.h | 69 +++++++++++++++
3 files changed, 235 insertions(+)
create mode 100644 include/uapi/linux/kfd_ioctl.h
diff --git a/drivers/gpu/hsa/radeon/kfd_chardev.c b/drivers/gpu/hsa/radeon/kfd_chardev.c
index 0b5bc74..4e7d5d0 100644
--- a/drivers/gpu/hsa/radeon/kfd_chardev.c
+++ b/drivers/gpu/hsa/radeon/kfd_chardev.c
@@ -27,11 +27,13 @@
#include <linux/sched.h>
#include <linux/slab.h>
#include <linux/uaccess.h>
+#include <uapi/linux/kfd_ioctl.h>
#include "kfd_priv.h"
#include "kfd_scheduler.h"
static long kfd_ioctl(struct file *, unsigned int, unsigned long);
static int kfd_open(struct inode *, struct file *);
+static int kfd_mmap(struct file *, struct vm_area_struct *);
static const char kfd_dev_name[] = "kfd";
@@ -108,17 +110,170 @@ kfd_open(struct inode *inode, struct file *filep)
return 0;
}
+static long
+kfd_ioctl_create_queue(struct file *filep, struct kfd_process *p, void __user *arg)
+{
+ struct kfd_ioctl_create_queue_args args;
+ struct kfd_dev *dev;
+ int err = 0;
+ unsigned int queue_id;
+ struct kfd_queue *queue;
+ struct kfd_process_device *pdd;
+
+ if (copy_from_user(&args, arg, sizeof(args)))
+ return -EFAULT;
+
+ dev = radeon_kfd_device_by_id(args.gpu_id);
+ if (dev == NULL)
+ return -EINVAL;
+
+ queue = kzalloc(
+ offsetof(struct kfd_queue, scheduler_queue) + dev->device_info->scheduler_class->queue_size,
+ GFP_KERNEL);
+
+ if (!queue)
+ return -ENOMEM;
+
+ queue->dev = dev;
+
+ mutex_lock(&p->mutex);
+
+ pdd = radeon_kfd_bind_process_to_device(dev, p);
+ if (IS_ERR(pdd) < 0) {
+ err = PTR_ERR(pdd);
+ goto err_bind_pasid;
+ }
+
+ pr_debug("kfd: creating queue number %d for PASID %d on GPU 0x%x\n",
+ pdd->queue_count,
+ p->pasid,
+ dev->id);
+
+ if (pdd->queue_count++ == 0) {
+ err = dev->device_info->scheduler_class->register_process(dev->scheduler, p, &pdd->scheduler_process);
+ if (err < 0)
+ goto err_register_process;
+ }
+
+ if (!radeon_kfd_allocate_queue_id(p, &queue_id))
+ goto err_allocate_queue_id;
+
+ err = dev->device_info->scheduler_class->create_queue(dev->scheduler, pdd->scheduler_process,
+ &queue->scheduler_queue,
+ (void __user *)args.ring_base_address,
+ args.ring_size,
+ (void __user *)args.read_pointer_address,
+ (void __user *)args.write_pointer_address,
+ radeon_kfd_queue_id_to_doorbell(dev, p, queue_id));
+ if (err)
+ goto err_create_queue;
+
+ radeon_kfd_install_queue(p, queue_id, queue);
+
+ args.queue_id = queue_id;
+ args.doorbell_address = (uint64_t)(uintptr_t)radeon_kfd_get_doorbell(filep, p, dev, queue_id);
+
+ if (copy_to_user(arg, &args, sizeof(args))) {
+ err = -EFAULT;
+ goto err_copy_args_out;
+ }
+
+ mutex_unlock(&p->mutex);
+
+ pr_debug("kfd: queue id %d was created successfully.\n"
+ " ring buffer address == 0x%016llX\n"
+ " read ptr address == 0x%016llX\n"
+ " write ptr address == 0x%016llX\n"
+ " doorbell address == 0x%016llX\n",
+ args.queue_id,
+ args.ring_base_address,
+ args.read_pointer_address,
+ args.write_pointer_address,
+ args.doorbell_address);
+
+ return 0;
+
+err_copy_args_out:
+ dev->device_info->scheduler_class->destroy_queue(dev->scheduler, &queue->scheduler_queue);
+err_create_queue:
+ radeon_kfd_remove_queue(p, queue_id);
+err_allocate_queue_id:
+ if (--pdd->queue_count == 0) {
+ dev->device_info->scheduler_class->deregister_process(dev->scheduler, pdd->scheduler_process);
+ pdd->scheduler_process = NULL;
+ }
+err_register_process:
+err_bind_pasid:
+ kfree(queue);
+ mutex_unlock(&p->mutex);
+ return err;
+}
+
+static int
+kfd_ioctl_destroy_queue(struct file *filp, struct kfd_process *p, void __user *arg)
+{
+ struct kfd_ioctl_destroy_queue_args args;
+ struct kfd_queue *queue;
+ struct kfd_dev *dev;
+ struct kfd_process_device *pdd;
+
+ if (copy_from_user(&args, arg, sizeof(args)))
+ return -EFAULT;
+
+ mutex_lock(&p->mutex);
+
+ queue = radeon_kfd_get_queue(p, args.queue_id);
+ if (!queue) {
+ mutex_unlock(&p->mutex);
+ return -EINVAL;
+ }
+
+ dev = queue->dev;
+
+ pr_debug("kfd: destroying queue id %d for PASID %d\n",
+ args.queue_id,
+ p->pasid);
+
+ radeon_kfd_remove_queue(p, args.queue_id);
+ dev->device_info->scheduler_class->destroy_queue(dev->scheduler, &queue->scheduler_queue);
+
+ kfree(queue);
+
+ pdd = radeon_kfd_get_process_device_data(dev, p);
+ BUG_ON(pdd == NULL); /* Because a queue exists. */
+
+ if (--pdd->queue_count == 0) {
+ dev->device_info->scheduler_class->deregister_process(dev->scheduler, pdd->scheduler_process);
+ pdd->scheduler_process = NULL;
+ }
+
+ mutex_unlock(&p->mutex);
+ return 0;
+}
static long
kfd_ioctl(struct file *filep, unsigned int cmd, unsigned long arg)
{
+ struct kfd_process *process;
long err = -EINVAL;
dev_info(kfd_device,
"ioctl cmd 0x%x (#%d), arg 0x%lx\n",
cmd, _IOC_NR(cmd), arg);
+ process = radeon_kfd_get_process(current);
+ if (IS_ERR(process))
+ return PTR_ERR(process);
+
switch (cmd) {
+ case KFD_IOC_CREATE_QUEUE:
+ err = kfd_ioctl_create_queue(filep, process, (void __user *)arg);
+ break;
+
+ case KFD_IOC_DESTROY_QUEUE:
+ err = kfd_ioctl_destroy_queue(filep, process, (void __user *)arg);
+ break;
+
default:
dev_err(kfd_device,
"unknown ioctl cmd 0x%x, arg 0x%lx)\n",
diff --git a/drivers/gpu/hsa/radeon/kfd_doorbell.c b/drivers/gpu/hsa/radeon/kfd_doorbell.c
index e1d8506..3de8a02 100644
--- a/drivers/gpu/hsa/radeon/kfd_doorbell.c
+++ b/drivers/gpu/hsa/radeon/kfd_doorbell.c
@@ -155,3 +155,14 @@ doorbell_t __user *radeon_kfd_get_doorbell(struct file *devkfd, struct kfd_proce
return &pdd->doorbell_mapping[doorbell_index];
}
+/*
+ * queue_ids are in the range [0,MAX_PROCESS_QUEUES) and are mapped 1:1
+ * to doorbells with the process's doorbell page
+ */
+unsigned int radeon_kfd_queue_id_to_doorbell(struct kfd_dev *kfd, struct kfd_process *process, unsigned int queue_id)
+{
+ /* doorbell_id_offset accounts for doorbells taken by KGD.
+ * pasid * doorbell_process_allocation/sizeof(doorbell_t) adjusts to the process's doorbells */
+ return kfd->doorbell_id_offset + process->pasid * (doorbell_process_allocation()/sizeof(doorbell_t)) + queue_id;
+}
+
diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h
new file mode 100644
index 0000000..dcc5fe0
--- /dev/null
+++ b/include/uapi/linux/kfd_ioctl.h
@@ -0,0 +1,69 @@
+/*
+ * Copyright 2014 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef KFD_IOCTL_H_INCLUDED
+#define KFD_IOCTL_H_INCLUDED
+
+#include <linux/types.h>
+#include <linux/ioctl.h>
+
+#define KFD_IOCTL_CURRENT_VERSION 1
+
+/* The 64-bit ABI is the authoritative version. */
+#pragma pack(push, 8)
+
+struct kfd_ioctl_get_version_args {
+ uint32_t min_supported_version; /* from KFD */
+ uint32_t max_supported_version; /* from KFD */
+};
+
+/* For kfd_ioctl_create_queue_args.queue_type. */
+#define KFD_IOC_QUEUE_TYPE_COMPUTE 0
+#define KFD_IOC_QUEUE_TYPE_SDMA 1
+
+struct kfd_ioctl_create_queue_args {
+ uint64_t ring_base_address; /* to KFD */
+ uint32_t ring_size; /* to KFD */
+ uint32_t gpu_id; /* to KFD */
+ uint32_t queue_type; /* to KFD */
+ uint32_t queue_percentage; /* to KFD */
+ uint32_t queue_priority; /* to KFD */
+ uint64_t write_pointer_address; /* to KFD */
+ uint64_t read_pointer_address; /* to KFD */
+
+ uint64_t doorbell_address; /* from KFD */
+ uint32_t queue_id; /* from KFD */
+};
+
+struct kfd_ioctl_destroy_queue_args {
+ uint32_t queue_id; /* to KFD */
+};
+
+#define KFD_IOC_MAGIC 'K'
+
+#define KFD_IOC_GET_VERSION _IOR(KFD_IOC_MAGIC, 1, struct kfd_ioctl_get_version_args)
+#define KFD_IOC_CREATE_QUEUE _IOWR(KFD_IOC_MAGIC, 2, struct kfd_ioctl_create_queue_args)
+#define KFD_IOC_DESTROY_QUEUE _IOWR(KFD_IOC_MAGIC, 3, struct kfd_ioctl_destroy_queue_args)
+
+#pragma pack(pop)
+
+#endif
--
1.9.1
^ permalink raw reply related
* [PATCH v10 11/11] seccomp: implement SECCOMP_FILTER_FLAG_TSYNC
From: Kees Cook @ 2014-07-10 18:40 UTC (permalink / raw)
To: linux-kernel
Cc: Kees Cook, Oleg Nesterov, Andy Lutomirski, Alexei Starovoitov,
Michael Kerrisk (man-pages), Andrew Morton, Daniel Borkmann,
Will Drewry, Julien Tinnes, David Drysdale, linux-api, x86,
linux-arm-kernel, linux-mips, linux-arch, linux-security-module
In-Reply-To: <1405017631-27346-1-git-send-email-keescook@chromium.org>
Applying restrictive seccomp filter programs to large or diverse
codebases often requires handling threads which may be started early in
the process lifetime (e.g., by code that is linked in). While it is
possible to apply permissive programs prior to process start up, it is
difficult to further restrict the kernel ABI to those threads after that
point.
This change adds a new seccomp syscall flag to SECCOMP_SET_MODE_FILTER for
synchronizing thread group seccomp filters at filter installation time.
When calling seccomp(SECCOMP_SET_MODE_FILTER, SECCOMP_FILTER_FLAG_TSYNC,
filter) an attempt will be made to synchronize all threads in current's
threadgroup to its new seccomp filter program. This is possible iff all
threads are using a filter that is an ancestor to the filter current is
attempting to synchronize to. NULL filters (where the task is running as
SECCOMP_MODE_NONE) are also treated as ancestors allowing threads to be
transitioned into SECCOMP_MODE_FILTER. If prctrl(PR_SET_NO_NEW_PRIVS,
...) has been set on the calling thread, no_new_privs will be set for
all synchronized threads too. On success, 0 is returned. On failure,
the pid of one of the failing threads will be returned and no filters
will have been applied.
The race conditions against another thread are:
- requesting TSYNC (already handled by sighand lock)
- performing a clone (already handled by sighand lock)
- changing its filter (already handled by sighand lock)
- calling exec (handled by cred_guard_mutex)
The clone case is assisted by the fact that new threads will have their
seccomp state duplicated from their parent before appearing on the tasklist.
Holding cred_guard_mutex means that seccomp filters cannot be assigned
while in the middle of another thread's exec (potentially bypassing
no_new_privs or similar). The call to de_thread() may kill threads waiting
for the mutex.
Changes across threads to the filter pointer includes a barrier.
Based on patches by Will Drewry.
Suggested-by: Julien Tinnes <jln@chromium.org>
Signed-off-by: Kees Cook <keescook@chromium.org>
---
fs/exec.c | 2 +-
include/linux/seccomp.h | 2 +
include/uapi/linux/seccomp.h | 3 +
kernel/seccomp.c | 130 +++++++++++++++++++++++++++++++++++++++++-
4 files changed, 135 insertions(+), 2 deletions(-)
diff --git a/fs/exec.c b/fs/exec.c
index 0f5c272410f6..ab1f1200ce5d 100644
--- a/fs/exec.c
+++ b/fs/exec.c
@@ -1216,7 +1216,7 @@ EXPORT_SYMBOL(install_exec_creds);
/*
* determine how safe it is to execute the proposed program
* - the caller must hold ->cred_guard_mutex to protect against
- * PTRACE_ATTACH
+ * PTRACE_ATTACH or seccomp thread-sync
*/
static void check_unsafe_exec(struct linux_binprm *bprm)
{
diff --git a/include/linux/seccomp.h b/include/linux/seccomp.h
index 9ff98b4bfe2e..15de2a711518 100644
--- a/include/linux/seccomp.h
+++ b/include/linux/seccomp.h
@@ -3,6 +3,8 @@
#include <uapi/linux/seccomp.h>
+#define SECCOMP_FILTER_FLAG_MASK ~(SECCOMP_FILTER_FLAG_TSYNC)
+
#ifdef CONFIG_SECCOMP
#include <linux/thread_info.h>
diff --git a/include/uapi/linux/seccomp.h b/include/uapi/linux/seccomp.h
index b258878ba754..0f238a43ff1e 100644
--- a/include/uapi/linux/seccomp.h
+++ b/include/uapi/linux/seccomp.h
@@ -14,6 +14,9 @@
#define SECCOMP_SET_MODE_STRICT 0
#define SECCOMP_SET_MODE_FILTER 1
+/* Valid flags for SECCOMP_SET_MODE_FILTER */
+#define SECCOMP_FILTER_FLAG_TSYNC 1
+
/*
* All BPF programs must return a 32-bit value.
* The bottom 16-bits are for optional return data.
diff --git a/kernel/seccomp.c b/kernel/seccomp.c
index c6bd5f196d9f..33dbb3907c8a 100644
--- a/kernel/seccomp.c
+++ b/kernel/seccomp.c
@@ -26,6 +26,7 @@
#ifdef CONFIG_SECCOMP_FILTER
#include <asm/syscall.h>
#include <linux/filter.h>
+#include <linux/pid.h>
#include <linux/ptrace.h>
#include <linux/security.h>
#include <linux/tracehook.h>
@@ -225,6 +226,109 @@ static inline void seccomp_assign_mode(struct task_struct *task,
}
#ifdef CONFIG_SECCOMP_FILTER
+/* Returns 1 if the candidate is an ancestor. */
+static int is_ancestor(struct seccomp_filter *candidate,
+ struct seccomp_filter *child)
+{
+ /* NULL is the root ancestor. */
+ if (candidate == NULL)
+ return 1;
+ for (; child; child = child->prev)
+ if (child == candidate)
+ return 1;
+ return 0;
+}
+
+/**
+ * seccomp_can_sync_threads: checks if all threads can be synchronized
+ *
+ * Expects sighand and cred_guard_mutex locks to be held.
+ *
+ * Returns 0 on success, -ve on error, or the pid of a thread which was
+ * either not in the correct seccomp mode or it did not have an ancestral
+ * seccomp filter.
+ */
+static inline pid_t seccomp_can_sync_threads(void)
+{
+ struct task_struct *thread, *caller;
+
+ BUG_ON(!mutex_is_locked(¤t->signal->cred_guard_mutex));
+ BUG_ON(!spin_is_locked(¤t->sighand->siglock));
+
+ if (current->seccomp.mode != SECCOMP_MODE_FILTER)
+ return -EACCES;
+
+ /* Validate all threads being eligible for synchronization. */
+ caller = current;
+ for_each_thread(caller, thread) {
+ pid_t failed;
+
+ if (thread->seccomp.mode == SECCOMP_MODE_DISABLED ||
+ (thread->seccomp.mode == SECCOMP_MODE_FILTER &&
+ is_ancestor(thread->seccomp.filter,
+ caller->seccomp.filter)))
+ continue;
+
+ /* Return the first thread that cannot be synchronized. */
+ failed = task_pid_vnr(thread);
+ /* If the pid cannot be resolved, then return -ESRCH */
+ if (unlikely(WARN_ON(failed == 0)))
+ failed = -ESRCH;
+ return failed;
+ }
+
+ return 0;
+}
+
+/**
+ * seccomp_sync_threads: sets all threads to use current's filter
+ *
+ * Expects sighand and cred_guard_mutex locks to be held, and for
+ * seccomp_can_sync_threads() to have returned success already
+ * without dropping the locks.
+ *
+ */
+static inline void seccomp_sync_threads(void)
+{
+ struct task_struct *thread, *caller;
+
+ BUG_ON(!mutex_is_locked(¤t->signal->cred_guard_mutex));
+ BUG_ON(!spin_is_locked(¤t->sighand->siglock));
+
+ /* Synchronize all threads. */
+ caller = current;
+ for_each_thread(caller, thread) {
+ /* Get a task reference for the new leaf node. */
+ get_seccomp_filter(caller);
+ /*
+ * Drop the task reference to the shared ancestor since
+ * current's path will hold a reference. (This also
+ * allows a put before the assignment.)
+ */
+ put_seccomp_filter(thread);
+ smp_store_release(&thread->seccomp.filter,
+ caller->seccomp.filter);
+ /*
+ * Opt the other thread into seccomp if needed.
+ * As threads are considered to be trust-realm
+ * equivalent (see ptrace_may_access), it is safe to
+ * allow one thread to transition the other.
+ */
+ if (thread->seccomp.mode == SECCOMP_MODE_DISABLED) {
+ /*
+ * Don't let an unprivileged task work around
+ * the no_new_privs restriction by creating
+ * a thread that sets it up, enters seccomp,
+ * then dies.
+ */
+ if (task_no_new_privs(caller))
+ task_set_no_new_privs(thread);
+
+ seccomp_assign_mode(thread, SECCOMP_MODE_FILTER);
+ }
+ }
+}
+
/**
* seccomp_prepare_filter: Prepares a seccomp filter for use.
* @fprog: BPF program to install
@@ -362,6 +466,15 @@ static long seccomp_attach_filter(unsigned int flags,
if (total_insns > MAX_INSNS_PER_PATH)
return -ENOMEM;
+ /* If thread sync has been requested, check that it is possible. */
+ if (flags & SECCOMP_FILTER_FLAG_TSYNC) {
+ int ret;
+
+ ret = seccomp_can_sync_threads();
+ if (ret)
+ return ret;
+ }
+
/*
* If there is an existing filter, make it the prev and don't drop its
* task reference.
@@ -369,6 +482,10 @@ static long seccomp_attach_filter(unsigned int flags,
filter->prev = current->seccomp.filter;
current->seccomp.filter = filter;
+ /* Now that the new filter is in place, synchronize to all threads. */
+ if (flags & SECCOMP_FILTER_FLAG_TSYNC)
+ seccomp_sync_threads();
+
return 0;
}
@@ -588,7 +705,7 @@ static long seccomp_set_mode_filter(unsigned int flags,
long ret = -EINVAL;
/* Validate flags. */
- if (flags != 0)
+ if (flags & SECCOMP_FILTER_FLAG_MASK)
return -EINVAL;
/* Prepare the new filter before holding any locks. */
@@ -596,6 +713,14 @@ static long seccomp_set_mode_filter(unsigned int flags,
if (IS_ERR(prepared))
return PTR_ERR(prepared);
+ /*
+ * Make sure we cannot change seccomp or nnp state via TSYNC
+ * while another thread is in the middle of calling exec.
+ */
+ if (flags & SECCOMP_FILTER_FLAG_TSYNC &&
+ mutex_lock_killable(¤t->signal->cred_guard_mutex))
+ goto out_free;
+
spin_lock_irq(¤t->sighand->siglock);
if (!seccomp_check_mode(seccomp_mode))
@@ -610,6 +735,9 @@ static long seccomp_set_mode_filter(unsigned int flags,
seccomp_assign_mode(current, seccomp_mode);
out:
spin_unlock_irq(¤t->sighand->siglock);
+ if (flags & SECCOMP_FILTER_FLAG_TSYNC)
+ mutex_unlock(¤t->signal->cred_guard_mutex);
+out_free:
seccomp_filter_free(prepared);
return ret;
}
--
1.7.9.5
^ permalink raw reply related
* [PATCH v10 10/11] seccomp: allow mode setting across threads
From: Kees Cook @ 2014-07-10 18:40 UTC (permalink / raw)
To: linux-kernel
Cc: linux-arch, linux-mips, Will Drewry, Kees Cook,
linux-security-module, linux-api, x86, Oleg Nesterov,
Andy Lutomirski, Daniel Borkmann, Julien Tinnes,
Michael Kerrisk (man-pages), Andrew Morton, David Drysdale,
linux-arm-kernel, Alexei Starovoitov
In-Reply-To: <1405017631-27346-1-git-send-email-keescook@chromium.org>
This changes the mode setting helper to allow threads to change the
seccomp mode from another thread. We must maintain barriers to keep
TIF_SECCOMP synchronized with the rest of the seccomp state.
Signed-off-by: Kees Cook <keescook@chromium.org>
---
kernel/seccomp.c | 36 +++++++++++++++++++++++++-----------
1 file changed, 25 insertions(+), 11 deletions(-)
diff --git a/kernel/seccomp.c b/kernel/seccomp.c
index 6087e018155c..c6bd5f196d9f 100644
--- a/kernel/seccomp.c
+++ b/kernel/seccomp.c
@@ -173,21 +173,24 @@ static int seccomp_check_filter(struct sock_filter *filter, unsigned int flen)
*/
static u32 seccomp_run_filters(int syscall)
{
- struct seccomp_filter *f;
+ struct seccomp_filter *f = ACCESS_ONCE(current->seccomp.filter);
struct seccomp_data sd;
u32 ret = SECCOMP_RET_ALLOW;
/* Ensure unexpected behavior doesn't result in failing open. */
- if (WARN_ON(current->seccomp.filter == NULL))
+ if (unlikely(WARN_ON(f == NULL)))
return SECCOMP_RET_KILL;
+ /* Make sure cross-thread synced filter points somewhere sane. */
+ smp_read_barrier_depends();
+
populate_seccomp_data(&sd);
/*
* All filters in the list are evaluated and the lowest BPF return
* value always takes priority (ignoring the DATA).
*/
- for (f = current->seccomp.filter; f; f = f->prev) {
+ for (; f; f = f->prev) {
u32 cur_ret = SK_RUN_FILTER(f->prog, (void *)&sd);
if ((cur_ret & SECCOMP_RET_ACTION) < (ret & SECCOMP_RET_ACTION))
@@ -207,12 +210,18 @@ static inline bool seccomp_check_mode(unsigned long seccomp_mode)
return true;
}
-static inline void seccomp_assign_mode(unsigned long seccomp_mode)
+static inline void seccomp_assign_mode(struct task_struct *task,
+ unsigned long seccomp_mode)
{
- BUG_ON(!spin_is_locked(¤t->sighand->siglock));
+ BUG_ON(!spin_is_locked(&task->sighand->siglock));
- current->seccomp.mode = seccomp_mode;
- set_tsk_thread_flag(current, TIF_SECCOMP);
+ task->seccomp.mode = seccomp_mode;
+ /*
+ * Make sure TIF_SECCOMP cannot be set before the mode (and
+ * filter) is set.
+ */
+ smp_mb__before_atomic();
+ set_tsk_thread_flag(task, TIF_SECCOMP);
}
#ifdef CONFIG_SECCOMP_FILTER
@@ -433,12 +442,17 @@ static int mode1_syscalls_32[] = {
int __secure_computing(int this_syscall)
{
- int mode = current->seccomp.mode;
int exit_sig = 0;
int *syscall;
u32 ret;
- switch (mode) {
+ /*
+ * Make sure that any changes to mode from another thread have
+ * been seen after TIF_SECCOMP was seen.
+ */
+ rmb();
+
+ switch (current->seccomp.mode) {
case SECCOMP_MODE_STRICT:
syscall = mode1_syscalls;
#ifdef CONFIG_COMPAT
@@ -543,7 +557,7 @@ static long seccomp_set_mode_strict(void)
#ifdef TIF_NOTSC
disable_TSC();
#endif
- seccomp_assign_mode(seccomp_mode);
+ seccomp_assign_mode(current, seccomp_mode);
ret = 0;
out:
@@ -593,7 +607,7 @@ static long seccomp_set_mode_filter(unsigned int flags,
/* Do not free the successfully attached filter. */
prepared = NULL;
- seccomp_assign_mode(seccomp_mode);
+ seccomp_assign_mode(current, seccomp_mode);
out:
spin_unlock_irq(¤t->sighand->siglock);
seccomp_filter_free(prepared);
--
1.7.9.5
^ permalink raw reply related
* [PATCH v10 09/11] seccomp: introduce writer locking
From: Kees Cook @ 2014-07-10 18:40 UTC (permalink / raw)
To: linux-kernel
Cc: Kees Cook, Oleg Nesterov, Andy Lutomirski, Alexei Starovoitov,
Michael Kerrisk (man-pages), Andrew Morton, Daniel Borkmann,
Will Drewry, Julien Tinnes, David Drysdale, linux-api, x86,
linux-arm-kernel, linux-mips, linux-arch, linux-security-module
In-Reply-To: <1405017631-27346-1-git-send-email-keescook@chromium.org>
Normally, task_struct.seccomp.filter is only ever read or modified by
the task that owns it (current). This property aids in fast access
during system call filtering as read access is lockless.
Updating the pointer from another task, however, opens up race
conditions. To allow cross-thread filter pointer updates, writes to
the seccomp fields are now protected by the sighand spinlock (which
is unique to the thread group). Read access remains lockless because
pointer updates themselves are atomic. However, writes (or cloning)
often entail additional checking (like maximum instruction counts)
which require locking to perform safely.
In the case of cloning threads, the child is invisible to the system
until it enters the task list. To make sure a child can't be cloned from
a thread and left in a prior state, seccomp duplication is additionally
moved under the sighand lock. Then parent and child are certain have
the same seccomp state when they exit the lock.
Based on patches by Will Drewry and David Drysdale.
Signed-off-by: Kees Cook <keescook@chromium.org>
---
include/linux/seccomp.h | 6 +++---
kernel/fork.c | 45 ++++++++++++++++++++++++++++++++++++++++++++-
kernel/seccomp.c | 16 +++++++++++++++-
3 files changed, 62 insertions(+), 5 deletions(-)
diff --git a/include/linux/seccomp.h b/include/linux/seccomp.h
index 4054b0994071..9ff98b4bfe2e 100644
--- a/include/linux/seccomp.h
+++ b/include/linux/seccomp.h
@@ -14,11 +14,11 @@ struct seccomp_filter;
*
* @mode: indicates one of the valid values above for controlled
* system calls available to a process.
- * @filter: The metadata and ruleset for determining what system calls
- * are allowed for a task.
+ * @filter: must always point to a valid seccomp-filter or NULL as it is
+ * accessed without locking during system call entry.
*
* @filter must only be accessed from the context of current as there
- * is no locking.
+ * is no read locking.
*/
struct seccomp {
int mode;
diff --git a/kernel/fork.c b/kernel/fork.c
index 6a13c46cd87d..ffc1b43e351f 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -315,6 +315,15 @@ static struct task_struct *dup_task_struct(struct task_struct *orig)
goto free_ti;
tsk->stack = ti;
+#ifdef CONFIG_SECCOMP
+ /*
+ * We must handle setting up seccomp filters once we're under
+ * the sighand lock in case orig has changed between now and
+ * then. Until then, filter must be NULL to avoid messing up
+ * the usage counts on the error path calling free_task.
+ */
+ tsk->seccomp.filter = NULL;
+#endif
setup_thread_stack(tsk, orig);
clear_user_return_notifier(tsk);
@@ -1081,6 +1090,35 @@ static int copy_signal(unsigned long clone_flags, struct task_struct *tsk)
return 0;
}
+static void copy_seccomp(struct task_struct *p)
+{
+#ifdef CONFIG_SECCOMP
+ /*
+ * Must be called with sighand->lock held, which is common to
+ * all threads in the group. Holding cred_guard_mutex is not
+ * needed because this new task is not yet running and cannot
+ * be racing exec.
+ */
+ BUG_ON(!spin_is_locked(¤t->sighand->siglock));
+
+ /* Ref-count the new filter user, and assign it. */
+ get_seccomp_filter(current);
+ p->seccomp = current->seccomp;
+
+ /*
+ * Explicitly enable no_new_privs here in case it got set
+ * between the task_struct being duplicated and holding the
+ * sighand lock. The seccomp state and nnp must be in sync.
+ */
+ if (task_no_new_privs(current))
+ task_set_no_new_privs(p);
+
+ /* If we have a seccomp mode, enable the thread flag. */
+ if (p->seccomp.mode != SECCOMP_MODE_DISABLED)
+ set_tsk_thread_flag(p, TIF_SECCOMP);
+#endif
+}
+
SYSCALL_DEFINE1(set_tid_address, int __user *, tidptr)
{
current->clear_child_tid = tidptr;
@@ -1196,7 +1234,6 @@ static struct task_struct *copy_process(unsigned long clone_flags,
goto fork_out;
ftrace_graph_init_task(p);
- get_seccomp_filter(p);
rt_mutex_init_task(p);
@@ -1437,6 +1474,12 @@ static struct task_struct *copy_process(unsigned long clone_flags,
spin_lock(¤t->sighand->siglock);
/*
+ * Copy seccomp details explicitly here, in case they were changed
+ * before holding sighand lock.
+ */
+ copy_seccomp(p);
+
+ /*
* Process group and session signals need to be delivered to just the
* parent before the fork or both the parent and the child after the
* fork. Restart if a signal comes in before we add the new process to
diff --git a/kernel/seccomp.c b/kernel/seccomp.c
index 502e54d7f86d..6087e018155c 100644
--- a/kernel/seccomp.c
+++ b/kernel/seccomp.c
@@ -199,6 +199,8 @@ static u32 seccomp_run_filters(int syscall)
static inline bool seccomp_check_mode(unsigned long seccomp_mode)
{
+ BUG_ON(!spin_is_locked(¤t->sighand->siglock));
+
if (current->seccomp.mode && current->seccomp.mode != seccomp_mode)
return false;
@@ -207,6 +209,8 @@ static inline bool seccomp_check_mode(unsigned long seccomp_mode)
static inline void seccomp_assign_mode(unsigned long seccomp_mode)
{
+ BUG_ON(!spin_is_locked(¤t->sighand->siglock));
+
current->seccomp.mode = seccomp_mode;
set_tsk_thread_flag(current, TIF_SECCOMP);
}
@@ -330,6 +334,8 @@ out:
* @flags: flags to change filter behavior
* @filter: seccomp filter to add to the current process
*
+ * Caller must be holding current->sighand->siglock lock.
+ *
* Returns 0 on success, -ve on error.
*/
static long seccomp_attach_filter(unsigned int flags,
@@ -338,6 +344,8 @@ static long seccomp_attach_filter(unsigned int flags,
unsigned long total_insns;
struct seccomp_filter *walker;
+ BUG_ON(!spin_is_locked(¤t->sighand->siglock));
+
/* Validate resulting filter length. */
total_insns = filter->prog->len;
for (walker = current->seccomp.filter; walker; walker = walker->prev)
@@ -527,6 +535,8 @@ static long seccomp_set_mode_strict(void)
const unsigned long seccomp_mode = SECCOMP_MODE_STRICT;
long ret = -EINVAL;
+ spin_lock_irq(¤t->sighand->siglock);
+
if (!seccomp_check_mode(seccomp_mode))
goto out;
@@ -537,6 +547,7 @@ static long seccomp_set_mode_strict(void)
ret = 0;
out:
+ spin_unlock_irq(¤t->sighand->siglock);
return ret;
}
@@ -564,13 +575,15 @@ static long seccomp_set_mode_filter(unsigned int flags,
/* Validate flags. */
if (flags != 0)
- goto out;
+ return -EINVAL;
/* Prepare the new filter before holding any locks. */
prepared = seccomp_prepare_user_filter(filter);
if (IS_ERR(prepared))
return PTR_ERR(prepared);
+ spin_lock_irq(¤t->sighand->siglock);
+
if (!seccomp_check_mode(seccomp_mode))
goto out;
@@ -582,6 +595,7 @@ static long seccomp_set_mode_filter(unsigned int flags,
seccomp_assign_mode(seccomp_mode);
out:
+ spin_unlock_irq(¤t->sighand->siglock);
seccomp_filter_free(prepared);
return ret;
}
--
1.7.9.5
^ permalink raw reply related
* [PATCH v10 08/11] seccomp: split filter prep from check and apply
From: Kees Cook @ 2014-07-10 18:40 UTC (permalink / raw)
To: linux-kernel
Cc: linux-arch, linux-mips, Will Drewry, Kees Cook,
linux-security-module, linux-api, x86, Oleg Nesterov,
Andy Lutomirski, Daniel Borkmann, Julien Tinnes,
Michael Kerrisk (man-pages), Andrew Morton, David Drysdale,
linux-arm-kernel, Alexei Starovoitov
In-Reply-To: <1405017631-27346-1-git-send-email-keescook@chromium.org>
In preparation for adding seccomp locking, move filter creation away
from where it is checked and applied. This will allow for locking where
no memory allocation is happening. The validation, filter attachment,
and seccomp mode setting can all happen under the future locks.
Signed-off-by: Kees Cook <keescook@chromium.org>
---
kernel/seccomp.c | 93 +++++++++++++++++++++++++++++++++++++-----------------
1 file changed, 64 insertions(+), 29 deletions(-)
diff --git a/kernel/seccomp.c b/kernel/seccomp.c
index 137e40c7ae3b..502e54d7f86d 100644
--- a/kernel/seccomp.c
+++ b/kernel/seccomp.c
@@ -18,6 +18,7 @@
#include <linux/compat.h>
#include <linux/sched.h>
#include <linux/seccomp.h>
+#include <linux/slab.h>
#include <linux/syscalls.h>
/* #define SECCOMP_DEBUG 1 */
@@ -27,7 +28,6 @@
#include <linux/filter.h>
#include <linux/ptrace.h>
#include <linux/security.h>
-#include <linux/slab.h>
#include <linux/tracehook.h>
#include <linux/uaccess.h>
@@ -213,27 +213,21 @@ static inline void seccomp_assign_mode(unsigned long seccomp_mode)
#ifdef CONFIG_SECCOMP_FILTER
/**
- * seccomp_attach_filter: Attaches a seccomp filter to current.
+ * seccomp_prepare_filter: Prepares a seccomp filter for use.
* @fprog: BPF program to install
*
- * Returns 0 on success or an errno on failure.
+ * Returns filter on success or an ERR_PTR on failure.
*/
-static long seccomp_attach_filter(struct sock_fprog *fprog)
+static struct seccomp_filter *seccomp_prepare_filter(struct sock_fprog *fprog)
{
struct seccomp_filter *filter;
unsigned long fp_size = fprog->len * sizeof(struct sock_filter);
- unsigned long total_insns = fprog->len;
struct sock_filter *fp;
int new_len;
long ret;
if (fprog->len == 0 || fprog->len > BPF_MAXINSNS)
- return -EINVAL;
-
- for (filter = current->seccomp.filter; filter; filter = filter->prev)
- total_insns += filter->prog->len + 4; /* include a 4 instr penalty */
- if (total_insns > MAX_INSNS_PER_PATH)
- return -ENOMEM;
+ return ERR_PTR(-EINVAL);
/*
* Installing a seccomp filter requires that the task has
@@ -244,11 +238,11 @@ static long seccomp_attach_filter(struct sock_fprog *fprog)
if (!task_no_new_privs(current) &&
security_capable_noaudit(current_cred(), current_user_ns(),
CAP_SYS_ADMIN) != 0)
- return -EACCES;
+ return ERR_PTR(-EACCES);
fp = kzalloc(fp_size, GFP_KERNEL|__GFP_NOWARN);
if (!fp)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
/* Copy the instructions from fprog. */
ret = -EFAULT;
@@ -292,13 +286,7 @@ static long seccomp_attach_filter(struct sock_fprog *fprog)
sk_filter_select_runtime(filter->prog);
- /*
- * If there is an existing filter, make it the prev and don't drop its
- * task reference.
- */
- filter->prev = current->seccomp.filter;
- current->seccomp.filter = filter;
- return 0;
+ return filter;
free_filter_prog:
kfree(filter->prog);
@@ -306,19 +294,20 @@ free_filter:
kfree(filter);
free_prog:
kfree(fp);
- return ret;
+ return ERR_PTR(ret);
}
/**
- * seccomp_attach_user_filter - attaches a user-supplied sock_fprog
+ * seccomp_prepare_user_filter - prepares a user-supplied sock_fprog
* @user_filter: pointer to the user data containing a sock_fprog.
*
* Returns 0 on success and non-zero otherwise.
*/
-static long seccomp_attach_user_filter(const char __user *user_filter)
+static struct seccomp_filter *
+seccomp_prepare_user_filter(const char __user *user_filter)
{
struct sock_fprog fprog;
- long ret = -EFAULT;
+ struct seccomp_filter *filter = ERR_PTR(-EFAULT);
#ifdef CONFIG_COMPAT
if (is_compat_task()) {
@@ -331,9 +320,39 @@ static long seccomp_attach_user_filter(const char __user *user_filter)
#endif
if (copy_from_user(&fprog, user_filter, sizeof(fprog)))
goto out;
- ret = seccomp_attach_filter(&fprog);
+ filter = seccomp_prepare_filter(&fprog);
out:
- return ret;
+ return filter;
+}
+
+/**
+ * seccomp_attach_filter: validate and attach filter
+ * @flags: flags to change filter behavior
+ * @filter: seccomp filter to add to the current process
+ *
+ * Returns 0 on success, -ve on error.
+ */
+static long seccomp_attach_filter(unsigned int flags,
+ struct seccomp_filter *filter)
+{
+ unsigned long total_insns;
+ struct seccomp_filter *walker;
+
+ /* Validate resulting filter length. */
+ total_insns = filter->prog->len;
+ for (walker = current->seccomp.filter; walker; walker = walker->prev)
+ total_insns += walker->prog->len + 4; /* 4 instr penalty */
+ if (total_insns > MAX_INSNS_PER_PATH)
+ return -ENOMEM;
+
+ /*
+ * If there is an existing filter, make it the prev and don't drop its
+ * task reference.
+ */
+ filter->prev = current->seccomp.filter;
+ current->seccomp.filter = filter;
+
+ return 0;
}
/* get_seccomp_filter - increments the reference count of the filter on @tsk */
@@ -346,6 +365,14 @@ void get_seccomp_filter(struct task_struct *tsk)
atomic_inc(&orig->usage);
}
+static inline void seccomp_filter_free(struct seccomp_filter *filter)
+{
+ if (filter) {
+ sk_filter_free(filter->prog);
+ kfree(filter);
+ }
+}
+
/* put_seccomp_filter - decrements the ref count of tsk->seccomp.filter */
void put_seccomp_filter(struct task_struct *tsk)
{
@@ -354,8 +381,7 @@ void put_seccomp_filter(struct task_struct *tsk)
while (orig && atomic_dec_and_test(&orig->usage)) {
struct seccomp_filter *freeme = orig;
orig = orig->prev;
- sk_filter_free(freeme->prog);
- kfree(freeme);
+ seccomp_filter_free(freeme);
}
}
@@ -533,21 +559,30 @@ static long seccomp_set_mode_filter(unsigned int flags,
const char __user *filter)
{
const unsigned long seccomp_mode = SECCOMP_MODE_FILTER;
+ struct seccomp_filter *prepared = NULL;
long ret = -EINVAL;
/* Validate flags. */
if (flags != 0)
goto out;
+ /* Prepare the new filter before holding any locks. */
+ prepared = seccomp_prepare_user_filter(filter);
+ if (IS_ERR(prepared))
+ return PTR_ERR(prepared);
+
if (!seccomp_check_mode(seccomp_mode))
goto out;
- ret = seccomp_attach_user_filter(filter);
+ ret = seccomp_attach_filter(flags, prepared);
if (ret)
goto out;
+ /* Do not free the successfully attached filter. */
+ prepared = NULL;
seccomp_assign_mode(seccomp_mode);
out:
+ seccomp_filter_free(prepared);
return ret;
}
#else
--
1.7.9.5
^ permalink raw reply related
* [PATCH v10 07/11] sched: move no_new_privs into new atomic flags
From: Kees Cook @ 2014-07-10 18:40 UTC (permalink / raw)
To: linux-kernel
Cc: linux-arch, linux-mips, Will Drewry, Kees Cook,
linux-security-module, linux-api, x86, Oleg Nesterov,
Andy Lutomirski, Daniel Borkmann, Julien Tinnes,
Michael Kerrisk (man-pages), Andrew Morton, David Drysdale,
linux-arm-kernel, Alexei Starovoitov
In-Reply-To: <1405017631-27346-1-git-send-email-keescook@chromium.org>
Since seccomp transitions between threads requires updates to the
no_new_privs flag to be atomic, the flag must be part of an atomic flag
set. This moves the nnp flag into a separate task field, and introduces
accessors.
Signed-off-by: Kees Cook <keescook@chromium.org>
---
fs/exec.c | 4 ++--
include/linux/sched.h | 18 +++++++++++++++---
kernel/seccomp.c | 2 +-
kernel/sys.c | 4 ++--
security/apparmor/domain.c | 4 ++--
5 files changed, 22 insertions(+), 10 deletions(-)
diff --git a/fs/exec.c b/fs/exec.c
index a3d33fe592d6..0f5c272410f6 100644
--- a/fs/exec.c
+++ b/fs/exec.c
@@ -1234,7 +1234,7 @@ static void check_unsafe_exec(struct linux_binprm *bprm)
* This isn't strictly necessary, but it makes it harder for LSMs to
* mess up.
*/
- if (current->no_new_privs)
+ if (task_no_new_privs(current))
bprm->unsafe |= LSM_UNSAFE_NO_NEW_PRIVS;
t = p;
@@ -1272,7 +1272,7 @@ int prepare_binprm(struct linux_binprm *bprm)
bprm->cred->egid = current_egid();
if (!(bprm->file->f_path.mnt->mnt_flags & MNT_NOSUID) &&
- !current->no_new_privs &&
+ !task_no_new_privs(current) &&
kuid_has_mapping(bprm->cred->user_ns, inode->i_uid) &&
kgid_has_mapping(bprm->cred->user_ns, inode->i_gid)) {
/* Set-uid? */
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 306f4f0c987a..0fd19055bb64 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1307,13 +1307,12 @@ struct task_struct {
* execve */
unsigned in_iowait:1;
- /* task may not gain privileges */
- unsigned no_new_privs:1;
-
/* Revert to default priority/policy when forking */
unsigned sched_reset_on_fork:1;
unsigned sched_contributes_to_load:1;
+ unsigned long atomic_flags; /* Flags needing atomic access. */
+
pid_t pid;
pid_t tgid;
@@ -1967,6 +1966,19 @@ static inline void memalloc_noio_restore(unsigned int flags)
current->flags = (current->flags & ~PF_MEMALLOC_NOIO) | flags;
}
+/* Per-process atomic flags. */
+#define PFA_NO_NEW_PRIVS 0x00000001 /* May not gain new privileges. */
+
+static inline bool task_no_new_privs(struct task_struct *p)
+{
+ return test_bit(PFA_NO_NEW_PRIVS, &p->atomic_flags);
+}
+
+static inline void task_set_no_new_privs(struct task_struct *p)
+{
+ set_bit(PFA_NO_NEW_PRIVS, &p->atomic_flags);
+}
+
/*
* task->jobctl flags
*/
diff --git a/kernel/seccomp.c b/kernel/seccomp.c
index 2f83496d6016..137e40c7ae3b 100644
--- a/kernel/seccomp.c
+++ b/kernel/seccomp.c
@@ -241,7 +241,7 @@ static long seccomp_attach_filter(struct sock_fprog *fprog)
* This avoids scenarios where unprivileged tasks can affect the
* behavior of privileged children.
*/
- if (!current->no_new_privs &&
+ if (!task_no_new_privs(current) &&
security_capable_noaudit(current_cred(), current_user_ns(),
CAP_SYS_ADMIN) != 0)
return -EACCES;
diff --git a/kernel/sys.c b/kernel/sys.c
index 66a751ebf9d9..ce8129192a26 100644
--- a/kernel/sys.c
+++ b/kernel/sys.c
@@ -1990,12 +1990,12 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3,
if (arg2 != 1 || arg3 || arg4 || arg5)
return -EINVAL;
- current->no_new_privs = 1;
+ task_set_no_new_privs(current);
break;
case PR_GET_NO_NEW_PRIVS:
if (arg2 || arg3 || arg4 || arg5)
return -EINVAL;
- return current->no_new_privs ? 1 : 0;
+ return task_no_new_privs(current) ? 1 : 0;
case PR_GET_THP_DISABLE:
if (arg2 || arg3 || arg4 || arg5)
return -EINVAL;
diff --git a/security/apparmor/domain.c b/security/apparmor/domain.c
index 452567d3a08e..d97cba3e3849 100644
--- a/security/apparmor/domain.c
+++ b/security/apparmor/domain.c
@@ -621,7 +621,7 @@ int aa_change_hat(const char *hats[], int count, u64 token, bool permtest)
* There is no exception for unconfined as change_hat is not
* available.
*/
- if (current->no_new_privs)
+ if (task_no_new_privs(current))
return -EPERM;
/* released below */
@@ -776,7 +776,7 @@ int aa_change_profile(const char *ns_name, const char *hname, bool onexec,
* no_new_privs is set because this aways results in a reduction
* of permissions.
*/
- if (current->no_new_privs && !unconfined(profile)) {
+ if (task_no_new_privs(current) && !unconfined(profile)) {
put_cred(cred);
return -EPERM;
}
--
1.7.9.5
^ permalink raw reply related
* [PATCH v10 06/11] MIPS: add seccomp syscall
From: Kees Cook @ 2014-07-10 18:40 UTC (permalink / raw)
To: linux-kernel
Cc: Kees Cook, Oleg Nesterov, Andy Lutomirski, Alexei Starovoitov,
Michael Kerrisk (man-pages), Andrew Morton, Daniel Borkmann,
Will Drewry, Julien Tinnes, David Drysdale, linux-api, x86,
linux-arm-kernel, linux-mips, linux-arch, linux-security-module
In-Reply-To: <1405017631-27346-1-git-send-email-keescook@chromium.org>
Wires up the new seccomp syscall.
Signed-off-by: Kees Cook <keescook@chromium.org>
---
arch/mips/include/uapi/asm/unistd.h | 15 +++++++++------
arch/mips/kernel/scall32-o32.S | 1 +
arch/mips/kernel/scall64-64.S | 1 +
arch/mips/kernel/scall64-n32.S | 1 +
arch/mips/kernel/scall64-o32.S | 1 +
5 files changed, 13 insertions(+), 6 deletions(-)
diff --git a/arch/mips/include/uapi/asm/unistd.h b/arch/mips/include/uapi/asm/unistd.h
index 5805414777e0..9bc13eaf9d67 100644
--- a/arch/mips/include/uapi/asm/unistd.h
+++ b/arch/mips/include/uapi/asm/unistd.h
@@ -372,16 +372,17 @@
#define __NR_sched_setattr (__NR_Linux + 349)
#define __NR_sched_getattr (__NR_Linux + 350)
#define __NR_renameat2 (__NR_Linux + 351)
+#define __NR_seccomp (__NR_Linux + 352)
/*
* Offset of the last Linux o32 flavoured syscall
*/
-#define __NR_Linux_syscalls 351
+#define __NR_Linux_syscalls 352
#endif /* _MIPS_SIM == _MIPS_SIM_ABI32 */
#define __NR_O32_Linux 4000
-#define __NR_O32_Linux_syscalls 351
+#define __NR_O32_Linux_syscalls 352
#if _MIPS_SIM == _MIPS_SIM_ABI64
@@ -701,16 +702,17 @@
#define __NR_sched_setattr (__NR_Linux + 309)
#define __NR_sched_getattr (__NR_Linux + 310)
#define __NR_renameat2 (__NR_Linux + 311)
+#define __NR_seccomp (__NR_Linux + 312)
/*
* Offset of the last Linux 64-bit flavoured syscall
*/
-#define __NR_Linux_syscalls 311
+#define __NR_Linux_syscalls 312
#endif /* _MIPS_SIM == _MIPS_SIM_ABI64 */
#define __NR_64_Linux 5000
-#define __NR_64_Linux_syscalls 311
+#define __NR_64_Linux_syscalls 312
#if _MIPS_SIM == _MIPS_SIM_NABI32
@@ -1034,15 +1036,16 @@
#define __NR_sched_setattr (__NR_Linux + 313)
#define __NR_sched_getattr (__NR_Linux + 314)
#define __NR_renameat2 (__NR_Linux + 315)
+#define __NR_seccomp (__NR_Linux + 316)
/*
* Offset of the last N32 flavoured syscall
*/
-#define __NR_Linux_syscalls 315
+#define __NR_Linux_syscalls 316
#endif /* _MIPS_SIM == _MIPS_SIM_NABI32 */
#define __NR_N32_Linux 6000
-#define __NR_N32_Linux_syscalls 315
+#define __NR_N32_Linux_syscalls 316
#endif /* _UAPI_ASM_UNISTD_H */
diff --git a/arch/mips/kernel/scall32-o32.S b/arch/mips/kernel/scall32-o32.S
index 3245474f19d5..ab02d14f1b5c 100644
--- a/arch/mips/kernel/scall32-o32.S
+++ b/arch/mips/kernel/scall32-o32.S
@@ -578,3 +578,4 @@ EXPORT(sys_call_table)
PTR sys_sched_setattr
PTR sys_sched_getattr /* 4350 */
PTR sys_renameat2
+ PTR sys_seccomp
diff --git a/arch/mips/kernel/scall64-64.S b/arch/mips/kernel/scall64-64.S
index be2fedd4ae33..010dccf128ec 100644
--- a/arch/mips/kernel/scall64-64.S
+++ b/arch/mips/kernel/scall64-64.S
@@ -431,4 +431,5 @@ EXPORT(sys_call_table)
PTR sys_sched_setattr
PTR sys_sched_getattr /* 5310 */
PTR sys_renameat2
+ PTR sys_seccomp
.size sys_call_table,.-sys_call_table
diff --git a/arch/mips/kernel/scall64-n32.S b/arch/mips/kernel/scall64-n32.S
index c1dbcda4b816..c3b3b6525df5 100644
--- a/arch/mips/kernel/scall64-n32.S
+++ b/arch/mips/kernel/scall64-n32.S
@@ -424,4 +424,5 @@ EXPORT(sysn32_call_table)
PTR sys_sched_setattr
PTR sys_sched_getattr
PTR sys_renameat2 /* 6315 */
+ PTR sys_seccomp
.size sysn32_call_table,.-sysn32_call_table
diff --git a/arch/mips/kernel/scall64-o32.S b/arch/mips/kernel/scall64-o32.S
index f1343ccd7ed7..bb1550b1f501 100644
--- a/arch/mips/kernel/scall64-o32.S
+++ b/arch/mips/kernel/scall64-o32.S
@@ -557,4 +557,5 @@ EXPORT(sys32_call_table)
PTR sys_sched_setattr
PTR sys_sched_getattr /* 4350 */
PTR sys_renameat2
+ PTR sys_seccomp
.size sys32_call_table,.-sys32_call_table
--
1.7.9.5
^ permalink raw reply related
* [PATCH v10 05/11] ARM: add seccomp syscall
From: Kees Cook @ 2014-07-10 18:40 UTC (permalink / raw)
To: linux-kernel
Cc: Kees Cook, Oleg Nesterov, Andy Lutomirski, Alexei Starovoitov,
Michael Kerrisk (man-pages), Andrew Morton, Daniel Borkmann,
Will Drewry, Julien Tinnes, David Drysdale, linux-api, x86,
linux-arm-kernel, linux-mips, linux-arch, linux-security-module
In-Reply-To: <1405017631-27346-1-git-send-email-keescook@chromium.org>
Wires up the new seccomp syscall.
Signed-off-by: Kees Cook <keescook@chromium.org>
---
arch/arm/include/uapi/asm/unistd.h | 1 +
arch/arm/kernel/calls.S | 1 +
2 files changed, 2 insertions(+)
diff --git a/arch/arm/include/uapi/asm/unistd.h b/arch/arm/include/uapi/asm/unistd.h
index ba94446c72d9..e21b4a069701 100644
--- a/arch/arm/include/uapi/asm/unistd.h
+++ b/arch/arm/include/uapi/asm/unistd.h
@@ -409,6 +409,7 @@
#define __NR_sched_setattr (__NR_SYSCALL_BASE+380)
#define __NR_sched_getattr (__NR_SYSCALL_BASE+381)
#define __NR_renameat2 (__NR_SYSCALL_BASE+382)
+#define __NR_seccomp (__NR_SYSCALL_BASE+383)
/*
* This may need to be greater than __NR_last_syscall+1 in order to
diff --git a/arch/arm/kernel/calls.S b/arch/arm/kernel/calls.S
index 8f51bdcdacbb..bea85f97f363 100644
--- a/arch/arm/kernel/calls.S
+++ b/arch/arm/kernel/calls.S
@@ -392,6 +392,7 @@
/* 380 */ CALL(sys_sched_setattr)
CALL(sys_sched_getattr)
CALL(sys_renameat2)
+ CALL(sys_seccomp)
#ifndef syscalls_counted
.equ syscalls_padding, ((NR_syscalls + 3) & ~3) - NR_syscalls
#define syscalls_counted
--
1.7.9.5
^ permalink raw reply related
* [PATCH v10 04/11] seccomp: add "seccomp" syscall
From: Kees Cook @ 2014-07-10 18:40 UTC (permalink / raw)
To: linux-kernel
Cc: Kees Cook, Oleg Nesterov, Andy Lutomirski, Alexei Starovoitov,
Michael Kerrisk (man-pages), Andrew Morton, Daniel Borkmann,
Will Drewry, Julien Tinnes, David Drysdale, linux-api, x86,
linux-arm-kernel, linux-mips, linux-arch, linux-security-module
In-Reply-To: <1405017631-27346-1-git-send-email-keescook@chromium.org>
This adds the new "seccomp" syscall with both an "operation" and "flags"
parameter for future expansion. The third argument is a pointer value,
used with the SECCOMP_SET_MODE_FILTER operation. Currently, flags must
be 0. This is functionally equivalent to prctl(PR_SET_SECCOMP, ...).
In addition to the TSYNC flag later in this patch series, there is a
non-zero chance that this syscall could be used for configuring a fixed
argument area for seccomp-tracer-aware processes to pass syscall arguments
in the future. Hence, the use of "seccomp" not simply "seccomp_add_filter"
for this syscall. Additionally, this syscall uses operation, flags,
and user pointer for arguments because strictly passing arguments via
a user pointer would mean seccomp itself would be unable to trivially
filter the seccomp syscall itself.
Signed-off-by: Kees Cook <keescook@chromium.org>
---
arch/Kconfig | 1 +
arch/x86/syscalls/syscall_32.tbl | 1 +
arch/x86/syscalls/syscall_64.tbl | 1 +
include/linux/syscalls.h | 2 ++
include/uapi/asm-generic/unistd.h | 4 ++-
include/uapi/linux/seccomp.h | 4 +++
kernel/seccomp.c | 55 +++++++++++++++++++++++++++++++++----
kernel/sys_ni.c | 3 ++
8 files changed, 65 insertions(+), 6 deletions(-)
diff --git a/arch/Kconfig b/arch/Kconfig
index 97ff872c7acc..0eae9df35b88 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -321,6 +321,7 @@ config HAVE_ARCH_SECCOMP_FILTER
- secure_computing is called from a ptrace_event()-safe context
- secure_computing return value is checked and a return value of -1
results in the system call being skipped immediately.
+ - seccomp syscall wired up
config SECCOMP_FILTER
def_bool y
diff --git a/arch/x86/syscalls/syscall_32.tbl b/arch/x86/syscalls/syscall_32.tbl
index d6b867921612..7527eac24122 100644
--- a/arch/x86/syscalls/syscall_32.tbl
+++ b/arch/x86/syscalls/syscall_32.tbl
@@ -360,3 +360,4 @@
351 i386 sched_setattr sys_sched_setattr
352 i386 sched_getattr sys_sched_getattr
353 i386 renameat2 sys_renameat2
+354 i386 seccomp sys_seccomp
diff --git a/arch/x86/syscalls/syscall_64.tbl b/arch/x86/syscalls/syscall_64.tbl
index ec255a1646d2..16272a6c12b7 100644
--- a/arch/x86/syscalls/syscall_64.tbl
+++ b/arch/x86/syscalls/syscall_64.tbl
@@ -323,6 +323,7 @@
314 common sched_setattr sys_sched_setattr
315 common sched_getattr sys_sched_getattr
316 common renameat2 sys_renameat2
+317 common seccomp sys_seccomp
#
# x32-specific system call numbers start at 512 to avoid cache impact
diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
index b0881a0ed322..1713977ee26f 100644
--- a/include/linux/syscalls.h
+++ b/include/linux/syscalls.h
@@ -866,4 +866,6 @@ asmlinkage long sys_process_vm_writev(pid_t pid,
asmlinkage long sys_kcmp(pid_t pid1, pid_t pid2, int type,
unsigned long idx1, unsigned long idx2);
asmlinkage long sys_finit_module(int fd, const char __user *uargs, int flags);
+asmlinkage long sys_seccomp(unsigned int op, unsigned int flags,
+ const char __user *uargs);
#endif
diff --git a/include/uapi/asm-generic/unistd.h b/include/uapi/asm-generic/unistd.h
index 333640608087..65acbf0e2867 100644
--- a/include/uapi/asm-generic/unistd.h
+++ b/include/uapi/asm-generic/unistd.h
@@ -699,9 +699,11 @@ __SYSCALL(__NR_sched_setattr, sys_sched_setattr)
__SYSCALL(__NR_sched_getattr, sys_sched_getattr)
#define __NR_renameat2 276
__SYSCALL(__NR_renameat2, sys_renameat2)
+#define __NR_seccomp 277
+__SYSCALL(__NR_seccomp, sys_seccomp)
#undef __NR_syscalls
-#define __NR_syscalls 277
+#define __NR_syscalls 278
/*
* All syscalls below here should go away really,
diff --git a/include/uapi/linux/seccomp.h b/include/uapi/linux/seccomp.h
index ac2dc9f72973..b258878ba754 100644
--- a/include/uapi/linux/seccomp.h
+++ b/include/uapi/linux/seccomp.h
@@ -10,6 +10,10 @@
#define SECCOMP_MODE_STRICT 1 /* uses hard-coded filter. */
#define SECCOMP_MODE_FILTER 2 /* uses user-supplied filter. */
+/* Valid operations for seccomp syscall. */
+#define SECCOMP_SET_MODE_STRICT 0
+#define SECCOMP_SET_MODE_FILTER 1
+
/*
* All BPF programs must return a 32-bit value.
* The bottom 16-bits are for optional return data.
diff --git a/kernel/seccomp.c b/kernel/seccomp.c
index 812cea2e7ffb..2f83496d6016 100644
--- a/kernel/seccomp.c
+++ b/kernel/seccomp.c
@@ -18,6 +18,7 @@
#include <linux/compat.h>
#include <linux/sched.h>
#include <linux/seccomp.h>
+#include <linux/syscalls.h>
/* #define SECCOMP_DEBUG 1 */
@@ -314,7 +315,7 @@ free_prog:
*
* Returns 0 on success and non-zero otherwise.
*/
-static long seccomp_attach_user_filter(char __user *user_filter)
+static long seccomp_attach_user_filter(const char __user *user_filter)
{
struct sock_fprog fprog;
long ret = -EFAULT;
@@ -517,6 +518,7 @@ out:
#ifdef CONFIG_SECCOMP_FILTER
/**
* seccomp_set_mode_filter: internal function for setting seccomp filter
+ * @flags: flags to change filter behavior
* @filter: struct sock_fprog containing filter
*
* This function may be called repeatedly to install additional filters.
@@ -527,11 +529,16 @@ out:
*
* Returns 0 on success or -EINVAL on failure.
*/
-static long seccomp_set_mode_filter(char __user *filter)
+static long seccomp_set_mode_filter(unsigned int flags,
+ const char __user *filter)
{
const unsigned long seccomp_mode = SECCOMP_MODE_FILTER;
long ret = -EINVAL;
+ /* Validate flags. */
+ if (flags != 0)
+ goto out;
+
if (!seccomp_check_mode(seccomp_mode))
goto out;
@@ -544,12 +551,35 @@ out:
return ret;
}
#else
-static inline long seccomp_set_mode_filter(char __user *filter)
+static inline long seccomp_set_mode_filter(unsigned int flags,
+ const char __user *filter)
{
return -EINVAL;
}
#endif
+/* Common entry point for both prctl and syscall. */
+static long do_seccomp(unsigned int op, unsigned int flags,
+ const char __user *uargs)
+{
+ switch (op) {
+ case SECCOMP_SET_MODE_STRICT:
+ if (flags != 0 || uargs != NULL)
+ return -EINVAL;
+ return seccomp_set_mode_strict();
+ case SECCOMP_SET_MODE_FILTER:
+ return seccomp_set_mode_filter(flags, uargs);
+ default:
+ return -EINVAL;
+ }
+}
+
+SYSCALL_DEFINE3(seccomp, unsigned int, op, unsigned int, flags,
+ const char __user *, uargs)
+{
+ return do_seccomp(op, flags, uargs);
+}
+
/**
* prctl_set_seccomp: configures current->seccomp.mode
* @seccomp_mode: requested mode to use
@@ -559,12 +589,27 @@ static inline long seccomp_set_mode_filter(char __user *filter)
*/
long prctl_set_seccomp(unsigned long seccomp_mode, char __user *filter)
{
+ unsigned int op;
+ char __user *uargs;
+
switch (seccomp_mode) {
case SECCOMP_MODE_STRICT:
- return seccomp_set_mode_strict();
+ op = SECCOMP_SET_MODE_STRICT;
+ /*
+ * Setting strict mode through prctl always ignored filter,
+ * so make sure it is always NULL here to pass the internal
+ * check in do_seccomp().
+ */
+ uargs = NULL;
+ break;
case SECCOMP_MODE_FILTER:
- return seccomp_set_mode_filter(filter);
+ op = SECCOMP_SET_MODE_FILTER;
+ uargs = filter;
+ break;
default:
return -EINVAL;
}
+
+ /* prctl interface doesn't have flags, so they are always zero. */
+ return do_seccomp(op, 0, uargs);
}
diff --git a/kernel/sys_ni.c b/kernel/sys_ni.c
index 36441b51b5df..2904a2105914 100644
--- a/kernel/sys_ni.c
+++ b/kernel/sys_ni.c
@@ -213,3 +213,6 @@ cond_syscall(compat_sys_open_by_handle_at);
/* compare kernel pointers */
cond_syscall(sys_kcmp);
+
+/* operate on Secure Computing state */
+cond_syscall(sys_seccomp);
--
1.7.9.5
^ permalink raw reply related
* [PATCH v10 03/11] seccomp: split mode setting routines
From: Kees Cook @ 2014-07-10 18:40 UTC (permalink / raw)
To: linux-kernel
Cc: linux-arch, linux-mips, Will Drewry, Kees Cook,
linux-security-module, linux-api, x86, Oleg Nesterov,
Andy Lutomirski, Daniel Borkmann, Julien Tinnes,
Michael Kerrisk (man-pages), Andrew Morton, David Drysdale,
linux-arm-kernel, Alexei Starovoitov
In-Reply-To: <1405017631-27346-1-git-send-email-keescook@chromium.org>
Separates the two mode setting paths to make things more readable with
fewer #ifdefs within function bodies.
Signed-off-by: Kees Cook <keescook@chromium.org>
---
kernel/seccomp.c | 71 ++++++++++++++++++++++++++++++++++++------------------
1 file changed, 48 insertions(+), 23 deletions(-)
diff --git a/kernel/seccomp.c b/kernel/seccomp.c
index 03a5959b7930..812cea2e7ffb 100644
--- a/kernel/seccomp.c
+++ b/kernel/seccomp.c
@@ -489,48 +489,66 @@ long prctl_get_seccomp(void)
}
/**
- * seccomp_set_mode: internal function for setting seccomp mode
- * @seccomp_mode: requested mode to use
- * @filter: optional struct sock_fprog for use with SECCOMP_MODE_FILTER
- *
- * This function may be called repeatedly with a @seccomp_mode of
- * SECCOMP_MODE_FILTER to install additional filters. Every filter
- * successfully installed will be evaluated (in reverse order) for each system
- * call the task makes.
+ * seccomp_set_mode_strict: internal function for setting strict seccomp
*
* Once current->seccomp.mode is non-zero, it may not be changed.
*
* Returns 0 on success or -EINVAL on failure.
*/
-static long seccomp_set_mode(unsigned long seccomp_mode, char __user *filter)
+static long seccomp_set_mode_strict(void)
{
+ const unsigned long seccomp_mode = SECCOMP_MODE_STRICT;
long ret = -EINVAL;
if (!seccomp_check_mode(seccomp_mode))
goto out;
- switch (seccomp_mode) {
- case SECCOMP_MODE_STRICT:
- ret = 0;
#ifdef TIF_NOTSC
- disable_TSC();
+ disable_TSC();
#endif
- break;
+ seccomp_assign_mode(seccomp_mode);
+ ret = 0;
+
+out:
+
+ return ret;
+}
+
#ifdef CONFIG_SECCOMP_FILTER
- case SECCOMP_MODE_FILTER:
- ret = seccomp_attach_user_filter(filter);
- if (ret)
- goto out;
- break;
-#endif
- default:
+/**
+ * seccomp_set_mode_filter: internal function for setting seccomp filter
+ * @filter: struct sock_fprog containing filter
+ *
+ * This function may be called repeatedly to install additional filters.
+ * Every filter successfully installed will be evaluated (in reverse order)
+ * for each system call the task makes.
+ *
+ * Once current->seccomp.mode is non-zero, it may not be changed.
+ *
+ * Returns 0 on success or -EINVAL on failure.
+ */
+static long seccomp_set_mode_filter(char __user *filter)
+{
+ const unsigned long seccomp_mode = SECCOMP_MODE_FILTER;
+ long ret = -EINVAL;
+
+ if (!seccomp_check_mode(seccomp_mode))
+ goto out;
+
+ ret = seccomp_attach_user_filter(filter);
+ if (ret)
goto out;
- }
seccomp_assign_mode(seccomp_mode);
out:
return ret;
}
+#else
+static inline long seccomp_set_mode_filter(char __user *filter)
+{
+ return -EINVAL;
+}
+#endif
/**
* prctl_set_seccomp: configures current->seccomp.mode
@@ -541,5 +559,12 @@ out:
*/
long prctl_set_seccomp(unsigned long seccomp_mode, char __user *filter)
{
- return seccomp_set_mode(seccomp_mode, filter);
+ switch (seccomp_mode) {
+ case SECCOMP_MODE_STRICT:
+ return seccomp_set_mode_strict();
+ case SECCOMP_MODE_FILTER:
+ return seccomp_set_mode_filter(filter);
+ default:
+ return -EINVAL;
+ }
}
--
1.7.9.5
^ permalink raw reply related
* [PATCH v10 02/11] seccomp: extract check/assign mode helpers
From: Kees Cook @ 2014-07-10 18:40 UTC (permalink / raw)
To: linux-kernel
Cc: Kees Cook, Oleg Nesterov, Andy Lutomirski, Alexei Starovoitov,
Michael Kerrisk (man-pages), Andrew Morton, Daniel Borkmann,
Will Drewry, Julien Tinnes, David Drysdale, linux-api, x86,
linux-arm-kernel, linux-mips, linux-arch, linux-security-module
In-Reply-To: <1405017631-27346-1-git-send-email-keescook@chromium.org>
To support splitting mode 1 from mode 2, extract the mode checking and
assignment logic into common functions.
Signed-off-by: Kees Cook <keescook@chromium.org>
---
kernel/seccomp.c | 22 ++++++++++++++++++----
1 file changed, 18 insertions(+), 4 deletions(-)
diff --git a/kernel/seccomp.c b/kernel/seccomp.c
index afb916c7e890..03a5959b7930 100644
--- a/kernel/seccomp.c
+++ b/kernel/seccomp.c
@@ -194,7 +194,23 @@ static u32 seccomp_run_filters(int syscall)
}
return ret;
}
+#endif /* CONFIG_SECCOMP_FILTER */
+static inline bool seccomp_check_mode(unsigned long seccomp_mode)
+{
+ if (current->seccomp.mode && current->seccomp.mode != seccomp_mode)
+ return false;
+
+ return true;
+}
+
+static inline void seccomp_assign_mode(unsigned long seccomp_mode)
+{
+ current->seccomp.mode = seccomp_mode;
+ set_tsk_thread_flag(current, TIF_SECCOMP);
+}
+
+#ifdef CONFIG_SECCOMP_FILTER
/**
* seccomp_attach_filter: Attaches a seccomp filter to current.
* @fprog: BPF program to install
@@ -490,8 +506,7 @@ static long seccomp_set_mode(unsigned long seccomp_mode, char __user *filter)
{
long ret = -EINVAL;
- if (current->seccomp.mode &&
- current->seccomp.mode != seccomp_mode)
+ if (!seccomp_check_mode(seccomp_mode))
goto out;
switch (seccomp_mode) {
@@ -512,8 +527,7 @@ static long seccomp_set_mode(unsigned long seccomp_mode, char __user *filter)
goto out;
}
- current->seccomp.mode = seccomp_mode;
- set_thread_flag(TIF_SECCOMP);
+ seccomp_assign_mode(seccomp_mode);
out:
return ret;
}
--
1.7.9.5
^ permalink raw reply related
* [PATCH v10 01/11] seccomp: create internal mode-setting function
From: Kees Cook @ 2014-07-10 18:40 UTC (permalink / raw)
To: linux-kernel
Cc: Kees Cook, Oleg Nesterov, Andy Lutomirski, Alexei Starovoitov,
Michael Kerrisk (man-pages), Andrew Morton, Daniel Borkmann,
Will Drewry, Julien Tinnes, David Drysdale, linux-api, x86,
linux-arm-kernel, linux-mips, linux-arch, linux-security-module
In-Reply-To: <1405017631-27346-1-git-send-email-keescook@chromium.org>
In preparation for having other callers of the seccomp mode setting
logic, split the prctl entry point away from the core logic that performs
seccomp mode setting.
Signed-off-by: Kees Cook <keescook@chromium.org>
---
kernel/seccomp.c | 16 ++++++++++++++--
1 file changed, 14 insertions(+), 2 deletions(-)
diff --git a/kernel/seccomp.c b/kernel/seccomp.c
index 301bbc24739c..afb916c7e890 100644
--- a/kernel/seccomp.c
+++ b/kernel/seccomp.c
@@ -473,7 +473,7 @@ long prctl_get_seccomp(void)
}
/**
- * prctl_set_seccomp: configures current->seccomp.mode
+ * seccomp_set_mode: internal function for setting seccomp mode
* @seccomp_mode: requested mode to use
* @filter: optional struct sock_fprog for use with SECCOMP_MODE_FILTER
*
@@ -486,7 +486,7 @@ long prctl_get_seccomp(void)
*
* Returns 0 on success or -EINVAL on failure.
*/
-long prctl_set_seccomp(unsigned long seccomp_mode, char __user *filter)
+static long seccomp_set_mode(unsigned long seccomp_mode, char __user *filter)
{
long ret = -EINVAL;
@@ -517,3 +517,15 @@ long prctl_set_seccomp(unsigned long seccomp_mode, char __user *filter)
out:
return ret;
}
+
+/**
+ * prctl_set_seccomp: configures current->seccomp.mode
+ * @seccomp_mode: requested mode to use
+ * @filter: optional struct sock_fprog for use with SECCOMP_MODE_FILTER
+ *
+ * Returns 0 on success or -EINVAL on failure.
+ */
+long prctl_set_seccomp(unsigned long seccomp_mode, char __user *filter)
+{
+ return seccomp_set_mode(seccomp_mode, filter);
+}
--
1.7.9.5
^ permalink raw reply related
* [PATCH v10 0/11] seccomp: add thread sync ability
From: Kees Cook @ 2014-07-10 18:40 UTC (permalink / raw)
To: linux-kernel
Cc: Kees Cook, Oleg Nesterov, Andy Lutomirski, Alexei Starovoitov,
Michael Kerrisk (man-pages), Andrew Morton, Daniel Borkmann,
Will Drewry, Julien Tinnes, David Drysdale, linux-api, x86,
linux-arm-kernel, linux-mips, linux-arch, linux-security-module
This adds the ability for threads to request seccomp filter
synchronization across their thread group (at filter attach time).
For example, for Chrome to make sure graphic driver threads are fully
confined after seccomp filters have been attached.
To support this, locking on seccomp changes via thread-group-shared
sighand lock is introduced, along with refactoring of no_new_privs. Races
with thread creation are handled via delayed duplication of the seccomp
task struct field and cred_guard_mutex.
This includes a new syscall (instead of adding a new prctl option),
as suggested by Andy Lutomirski and Michael Kerrisk.
Thanks!
-Kees
v10:
- dropped pending-kill checks (oleg)
- tweaked memory barriers (oleg)
v9:
- rearranged/split patches to make things more reviewable
- added use of cred_guard_mutex to solve exec race (oleg, luto)
- added barriers for TIF_SECCOMP vs seccomp.mode race (oleg, luto)
- fixed missed copying of nnp state after v8 refactor (oleg)
v8:
- drop use of tasklist_lock, appears redundant against sighand (oleg)
- reduced use of smp_load_acquire to logical minimum (oleg)
- change nnp to a task struct held atomic flags field (oleg, luto)
- drop needless irqflags changes in fork.c for holding sighand lock (oleg)
- cleaned up use of thread for-each loop (oleg)
- rearranged patch order to keep syscall changes adjacent
- added example code to manpage (mtk)
v7:
- rebase on Linus's tree (merged with network bpf changes)
- wrote manpage text documenting API (follows this series)
v6:
- switch from seccomp-specific lock to thread-group lock to gain atomicity
- implement seccomp syscall across all architectures with seccomp filter
- clean up sparse warnings around locking
v5:
- move includes around (drysdale)
- drop set_nnp return value (luto)
- use smp_load_acquire/store_release (luto)
- merge nnp changes to seccomp always, fewer ifdef (luto)
v4:
- cleaned up locking further, as noticed by David Drysdale
v3:
- added SECCOMP_EXT_ACT_FILTER for new filter install options
v2:
- reworked to avoid clone races
^ permalink raw reply
* Re: [PATCH v9 09/11] seccomp: introduce writer locking
From: Oleg Nesterov @ 2014-07-10 17:35 UTC (permalink / raw)
To: Kees Cook
Cc: linux-arch, linux-mips, Will Drewry, linux-security-module,
Linux API, x86@kernel.org, LKML, Andy Lutomirski, Daniel Borkmann,
Julien Tinnes, Michael Kerrisk (man-pages), Andrew Morton,
David Drysdale, linux-arm-kernel@lists.infradead.org,
Alexei Starovoitov
In-Reply-To: <CAGXu5jKNUn0OcXPyTmqbHwQ_GPMNTeajyrxpd2xAtzjTRFyhpg@mail.gmail.com>
On 07/10, Kees Cook wrote:
>
> On Thu, Jul 10, 2014 at 8:24 AM, Oleg Nesterov <oleg@redhat.com> wrote:
> >
> > Just to simplify. Suppose TIF_SECCOMP was set a long ago. This thread
> > has a single filter F1 and it enters seccomp_run_filters().
> >
> > Right before it does ACCESS_ONCE() to read the pointer, another thread
> > does seccomp_sync_threads() and sets .filter = F2.
> >
> > If ACCESS_ONCE() returns F1 - everything is fine. But it can see the new
> > pointer F2, and in this case we need a barrier to ensure that, say,
> > LOAD(F2->prog) will see all the preceding changes in this memory.
>
> And the rmb() isn't sufficient for that?
But it has no effect if the pointer was changed _after_ rmb() was already
called.
And, you need a barrier _after_ ACCESS_ONCE().
(Unless, again, we know that this is the first filter, but this is only
by accident).
> Is another barrier needed
> before assigning the filter pointer to make sure the contents it
> points to are flushed?
I think smp_store_release() should be moved from seccomp_attach_filter()
to seccomp_sync_threads(). Although probably it _should_ work either way,
but at least this looks confusing because a) "current" doesn't need a
barrier to serialize wuth itself, and b) it is not clear why it is safe
to change the pointer dereferenced by another thread without a barrier.
> What's the least time-consuming operation I can use in run_filters?
As I said smp_read_barrier_depends() (nop unless alpha) or
smp_load_acquire() which you used in the previous version.
And to remind, afaics smp_load_acquire() in put_filter() should die ;)
Oleg.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox