* Re: [PATCH v4 02/12] KVM: define common KVM_GUESTDBG_USE_SW/HW_BP bits
From: Christian Borntraeger @ 2015-05-15 15:58 UTC (permalink / raw)
To: Alex Bennée, kvm, linux-arm-kernel, kvmarm, christoffer.dall,
marc.zyngier, peter.maydell, agraf, drjones, pbonzini,
zhichao.huang
Cc: Tony Krowiak, Alexey Kardashevskiy, Benjamin Herrenschmidt,
Heiko Carstens, Paul Mackerras, H. Peter Anvin, linux-s390,
Michael Mueller, Nadav Amit, Michael Ellerman, x86, Ingo Molnar,
Jason J. Herne, jan.kiszka, bp, Gleb Natapov, r65777,
Cornelia Huck, Thomas Gleixner, Eric Farman, linux-api, open list,
Dominik Dingel, dahi, Martin Schwidefsky
In-Reply-To: <1431700035-23479-3-git-send-email-alex.bennee@linaro.org>
Am 15.05.2015 um 16:27 schrieb Alex Bennée:
> index ef1a5fc..aca4f86 100644
> --- a/arch/s390/include/uapi/asm/kvm.h
> +++ b/arch/s390/include/uapi/asm/kvm.h
> @@ -114,8 +114,6 @@ struct kvm_fpu {
> __u64 fprs[16];
> };
>
> -#define KVM_GUESTDBG_USE_HW_BP 0x00010000
> -
> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> index 70ac641..7c5dd11 100644
> --- a/include/uapi/linux/kvm.h
> +++ b/include/uapi/linux/kvm.h
[...]
> +#define KVM_GUESTDBG_USE_SW_BP (1 << 16)
> +#define KVM_GUESTDBG_USE_HW_BP (1 << 17)
This is a abi break for s390, no?
David do you remember why we did not use SW_BP?
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply
* Re: [PATCH 10/23] userfaultfd: add new syscall to provide memory externalization
From: Andrea Arcangeli @ 2015-05-15 16:04 UTC (permalink / raw)
To: Linus Torvalds
Cc: Andrew Morton, Linux Kernel Mailing List, linux-mm, qemu-devel,
KVM list, Linux API, Pavel Emelyanov, Sanidhya Kashyap,
zhang.zhanghailiang, Kirill A. Shutemov, Andres Lagar-Cavilla,
Dave Hansen, Paolo Bonzini, Rik van Riel, Mel Gorman,
Andy Lutomirski, Hugh Dickins, Peter Feiner,
Dr. David Alan Gilbert, Johannes Weiner, Huangpeng (Pet
In-Reply-To: <CA+55aFwCODeiXUPDR7-Y-=2xE2abmVuCnmVV=ezFqhO+JkaW=A@mail.gmail.com>
On Thu, May 14, 2015 at 10:49:06AM -0700, Linus Torvalds wrote:
> On Thu, May 14, 2015 at 10:31 AM, Andrea Arcangeli <aarcange@redhat.com> wrote:
> > +static __always_inline void wake_userfault(struct userfaultfd_ctx *ctx,
> > + struct userfaultfd_wake_range *range)
> > +{
> > + if (waitqueue_active(&ctx->fault_wqh))
> > + __wake_userfault(ctx, range);
> > +}
>
> Pretty much every single time people use this "if
> (waitqueue_active())" model, it tends to be a bug, because it means
> that there is zero serialization with people who are just about to go
> to sleep. It's fundamentally racy against all the "wait_event()" loops
> that carefully do memory barriers between testing conditions and going
> to sleep, because the memory barriers now don't exist on the waking
> side.
As far as 10/23 is concerned, the __wake_userfault taking the locks
would also ignore any "incoming" not yet "read(2)" userfault. "read(2)"
as in syscall read. So there was no race to worry about as "incoming"
userfaults would be ignored anyway by the wake.
The only case that had to be reliable in not missing wakeup events was
the "release" file operation. But that doesn't use waitqueue_active
and it relies on ctx->released combined with mmap_sem taken for writing.
http://git.kernel.org/cgit/linux/kernel/git/andrea/aa.git/tree/fs/userfaultfd.c?h=userfault&id=2f73ffa8267e41f04fe6b0d93d23feed45c0980a#n267
However later (after I started documenting what userland should do) I
started to question this old model of leaving the race handling to
userland. It's way too complex to leave the race handling to
userland. Furthermore it's inefficient because there would be lots of
spurious userfaults that we could have been waken up within the kernel
before userland could have a chance to read them.
So then I handled the race in the kernel in patch 17/23. That allowed
to drop hundres of line of locking code from qemu (two bits per page
and a mutex and lots of complexity) and then I didn't need to document
the complex rules described in this commit header:
http://git.kernel.org/cgit/linux/kernel/git/andrea/aa.git/commit/?h=userfault&id=5a2b3614e107482da9a1fcfcd120d30eb62d45dc
You can see from the patch it complicated the kernel by adding a
pagetable walk, but it's worth it because it's simpler than solving it
in userland, plus its solved at once for all users.
The only cons is that the old model would allow userland to be way
more consistent in enforcing asserts as it had the control.
With the current model POLLIN may be returned from poll() despite the
later read returns -EAGAIN, so it basically requires a non blocking
open if people uses poll(). (blocking reads without poll are still
fine)
Now back to your question: the waitqueue_active is still there in the
current model as well (since 17/23).
Since 17/23 losing a wakeup (as far as qemu is concerned) would mean
that qemu would read the address of the fault that didn't get waken up
because of the race, then it would send a page request to the source
node (it had no state in the destination node where the userfault runs
to know if it was a "dup"), which would discard the request (not
sending the page twice) noticing in its simple per-page bitmap that it
was already sent. So with the new model after 17/23 qemu, losing a
wakeup is a bug.
The wait_event is like this:
handle_userfault (wait_event)
-------
spin_lock(&ctx->fault_pending_wqh.lock);
/*
* After the __add_wait_queue the uwq is visible to userland
* through poll/read().
*/
__add_wait_queue(&ctx->fault_pending_wqh, &uwq.wq);
/*
* The smp_mb() after __set_current_state prevents the reads
* following the spin_unlock to happen before the list_add in
* __add_wait_queue.
*/
set_current_state(TASK_KILLABLE);
spin_unlock(&ctx->fault_pending_wqh.lock);
must_wait = userfaultfd_must_wait(ctx, address, flags, reason);
up_read(&mm->mmap_sem);
if (likely(must_wait && !ACCESS_ONCE(ctx->released) &&
!fatal_signal_pending(current))) {
wake_up_poll(&ctx->fd_wqh, POLLIN);
schedule();
The wakeup side is:
userfaultfd_copy
-----------
ret = mcopy_atomic(ctx->mm, uffdio_copy.dst, uffdio_copy.src,
uffdio_copy.len);
if (unlikely(put_user(ret, &user_uffdio_copy->copy)))
return -EFAULT;
if (ret < 0)
goto out;
BUG_ON(!ret);
/* len == 0 would wake all */
range.len = ret;
if (!(uffdio_copy.mode & UFFDIO_COPY_MODE_DONTWAKE)) {
range.start = uffdio_copy.dst;
wake_userfault(ctx, &range);
wake_userfault is the function that is using waitqueue_active so the
problem materializes if waitqueue_active moves before
mcopy_atomic. Precisely it should move before the set_pte_at below:
mcopy_atomic_pte
------
set_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte);
/* No need to invalidate - it was non-present before */
update_mmu_cache(dst_vma, dst_addr, dst_pte);
pte_unmap_unlock(dst_pte, ptl);
unlock would allow the read to be reordered before it even on
x86. There's an up_read as well in between but it has the same
problem. So you're right that theoretically we can miss a wakeup.
Practically it sounds unlikely because of the sheer size of
mcopy_atomic and we never experienced it but it's still a bug.
> So I'm making a new rule: if you use waitqueue_active(), I want an
> explanation for why it's not racy with the waiter. A big comment
> about the memory ordering, or about higher-level locks that are held
> by the caller, or something. Linus
To fix it I added this along a comment:
diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
index c89e96f..6be316b 100644
--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -593,6 +593,21 @@ static void __wake_userfault(struct userfaultfd_ctx *ctx,
static __always_inline void wake_userfault(struct userfaultfd_ctx *ctx,
struct userfaultfd_wake_range *range)
{
+ /*
+ * To be sure waitqueue_active() is not reordered by the CPU
+ * before the pagetable update, use an explicit SMP memory
+ * barrier here. PT lock release or up_read(mmap_sem) still
+ * have release semantics that can allow the
+ * waitqueue_active() to be reordered before the pte update.
+ */
+ smp_mb();
+
+ /*
+ * Use waitqueue_active because it's very frequent to
+ * change the address space atomically even if there are no
+ * userfaults yet. So we take the spinlock only when we're
+ * sure we've userfaults to wake.
+ */
if (waitqueue_active(&ctx->fault_pending_wqh) ||
waitqueue_active(&ctx->fault_wqh))
__wake_userfault(ctx, range);
The wait_event/handle_userfault side already has a smp_mb() to prevent
the lockless pagetable walk to be reordered before the list_add in
__add_wait_queue (needed as well precisely because of the
waitqueue_active optimization that I'd like to keep).
Thanks,
Andrea
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [PATCH v4 10/12] KVM: arm64: guest debug, HW assisted debug support
From: Alex Bennée @ 2015-05-15 16:16 UTC (permalink / raw)
To: Mark Rutland
Cc: kvm@vger.kernel.org, open list:DOCUMENTATION,
jan.kiszka@siemens.com, Will Deacon, Peter Zijlstra,
kvmarm@lists.cs.columbia.edu, Lorenzo Pieralisi, Russell King,
Jonathan Corbet, Ingo Molnar, Gleb Natapov,
zhichao.huang@linaro.org, Catalin Marinas, bp@suse.de,
Marc Zyngier, r65777@freescale.com,
linux-arm-kernel@lists.infradead.org, adead.org,
open list:ABI/API, open list
In-Reply-To: <20150515152346.GB23652@leverpostej>
Mark Rutland <mark.rutland@arm.com> writes:
> Hi Alex,
>
> On Fri, May 15, 2015 at 03:27:13PM +0100, Alex Bennée wrote:
>> This adds support for userspace to control the HW debug registers for
>> guest debug. In the debug ioctl we copy the IMPDEF defined number of
>> registers into a new register set called host_debug_state. There is now
>> a new vcpu parameter called debug_ptr which selects which register set
>> is to copied into the real registers when world switch occurs.
>>
>> I've moved some helper functions into the hw_breakpoint.h header for
>> re-use.
>>
>> As with single step we need to tweak the guest registers to enable the
>> exceptions so we need to save and restore those bits.
>>
>> Two new capabilities have been added to the KVM_EXTENSION ioctl to allow
>> userspace to query the number of hardware break and watch points
>> available on the host hardware.
>
> There's the unfortunate possibility that these could vary across cores
> in a big.LITTLE system (though we haven't seen that thus far). The
> kernel sanity checks should currently explode if such a case is
> encountered, but I don't know what we'd do were that to happen.
I suspect we would have to disable HW assisted breakpoints or return the
lowest common denominator if we can tell which cores we shall every be
scheduled on.
>
> This gets more fun when you consider the context-aware breakpoints are
> the highest numbered. So the set of (context-aware) breakpoints might
> not intersect across all CPUs.
I didn't see a reference to that in the ARM ARM. It seemed to imply any
breakpoint could be context aware is .BT was appropriately set and
linked to the VR.
As it happens the gdb stub interface in QEMU is fairly limited so while
I expose the full debug registers I don't think there is currently a way
to expose any of the fancy linking/context stuff to the userspace
debugger. However I did make the ABI pass full raw values in so this
could become a possibility later without having to expand the ABI.
> I'm not sure what the best thing to do is w.r.t. exposing that to
> userspace.
>
> Thanks,
> Mark.
--
Alex Bennée
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply
* Re: [PATCH v4 10/12] KVM: arm64: guest debug, HW assisted debug support
From: Mark Rutland @ 2015-05-15 17:01 UTC (permalink / raw)
To: Alex Bennée
Cc: kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
kvmarm@lists.cs.columbia.edu, christoffer.dall@linaro.org,
Marc Zyngier, peter.maydell@linaro.org, agraf@suse.de,
drjones@redhat.com, pbonzini@redhat.com, zhichao.huang@linaro.org,
Lorenzo Pieralisi, Russell King, Jonathan Corbet, Gleb Natapov,
jan.kiszka@siemens.com, open list:DOCUMENTATION, Will Deacon,
open list
In-Reply-To: <87y4kpke91.fsf@linaro.org>
> > This gets more fun when you consider the context-aware breakpoints are
> > the highest numbered. So the set of (context-aware) breakpoints might
> > not intersect across all CPUs.
>
> I didn't see a reference to that in the ARM ARM. It seemed to imply any
> breakpoint could be context aware is .BT was appropriately set and
> linked to the VR.
The existence of ID_AA64_DFR0_EL1.CTX_CMPs implies otherwise, though I
haven't dug much deeper.
Thanks,
Mark.
^ permalink raw reply
* Re: [PATCH v4 10/12] KVM: arm64: guest debug, HW assisted debug support
From: Peter Maydell @ 2015-05-15 17:09 UTC (permalink / raw)
To: Alex Bennée
Cc: kvm@vger.kernel.org, open list:DOCUMENTATION, Peter Zijlstra,
jan.kiszka@siemens.com, Will Deacon, kvmarm@lists.cs.columbia.edu,
Lorenzo Pieralisi, Russell King, Jonathan Corbet, Ingo Molnar,
Gleb Natapov, zhichao.huang@linaro.org, Catalin Marinas,
bp@suse.de, Marc Zyngier, r65777@freescale.com,
linux-arm-kernel@lists.infradead.org, open list:ABI/API,
open list, "dahi@linux.vnet.ibm.com" <da>
In-Reply-To: <87y4kpke91.fsf@linaro.org>
On 15 May 2015 at 17:16, Alex Bennée <alex.bennee@linaro.org> wrote:
> Mark Rutland <mark.rutland@arm.com> writes:
>> This gets more fun when you consider the context-aware breakpoints are
>> the highest numbered. So the set of (context-aware) breakpoints might
>> not intersect across all CPUs.
>
> I didn't see a reference to that in the ARM ARM. It seemed to imply any
> breakpoint could be context aware is .BT was appropriately set and
> linked to the VR.
No; see D2.9.2; there must be at least one context-aware
breakpoint, but no requirement for more than that.
-- PMM
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply
* Re: [PATCH v4 02/12] KVM: define common KVM_GUESTDBG_USE_SW/HW_BP bits
From: David Hildenbrand @ 2015-05-15 17:33 UTC (permalink / raw)
To: Christian Borntraeger
Cc: Alex Bennée, kvm-u79uwXL29TY76Z2rM5mHXA,
linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r,
kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg,
christoffer.dall-QSEj5FYQhm4dnm+yROfE0A, marc.zyngier-5wv7dgnIgG8,
peter.maydell-QSEj5FYQhm4dnm+yROfE0A, agraf-l3A5Bk7waGM,
drjones-H+wXaHxf7aLQT0dZR+AlfA, pbonzini-H+wXaHxf7aLQT0dZR+AlfA,
zhichao.huang-QSEj5FYQhm4dnm+yROfE0A,
jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, r65777-KZfg59tc24xl57MIdRCFDg,
bp-l3A5Bk7waGM, Benjamin Herrenschmidt, Paul Mackerras,
Michael Ellerman, Martin Schwidefsky, Heiko Carstens,
supporter:S390, Thomas Gleixner, Ingo Molnar, H. Peter Anvin,
maintainer:X86 ARCHITECTURE..., Gleb Natapov, Bharat Bhushan,
Alexey Kardashevskiy
In-Reply-To: <555613F2.9060204-tA70FqPdS9bQT0dZR+AlfA@public.gmane.org>
> Am 15.05.2015 um 16:27 schrieb Alex Bennée:
> > +++ b/arch/s390/include/uapi/asm/kvm.h
> > @@ -114,8 +114,6 @@ struct kvm_fpu {
> > __u64 fprs[16];
> > };
> >
> > -#define KVM_GUESTDBG_USE_HW_BP 0x00010000
> [...]
> > +++ b/include/uapi/linux/kvm.h
> [...]
> > +#define KVM_GUESTDBG_USE_SW_BP (1 << 16)
> > +#define KVM_GUESTDBG_USE_HW_BP (1 << 17)
>
> This is an ABI break for s390, no?
>
> David, do you remember why we do not use KVM_GUESTDBG_USE_SW_BP?
>
We never had to tell the kernel about software breakpoints as this is all
handled via 4 byte DIAG instructions until now. We don't have to turn this
mechanism on. QEMU can directly insert the desired DIAG instructions and gets
notified when they are about to get executed.
(But we still have 2 byte breakpoint support todo - still tbd how exactly this
will be realized - could be turned on via such a mechanism)
The problem is, that these bits are arch specific, now Alex wants to unify
them for all archs.
So yes, this is an ABI break for us and breaks hardware breakpoints.(I think
the first version of this patch didn't contain this ABI break when I had a look)
I wonder if it wouldn't make more sense to
- introduce new bits in the arch-unspecific section
- rework the existing implementers to accept both bits
Or to simply leave stuff as it is and handle it via arch specific bits.
David
^ permalink raw reply
* Re: [PATCH v4 02/12] KVM: define common KVM_GUESTDBG_USE_SW/HW_BP bits
From: Jan Kiszka @ 2015-05-15 17:46 UTC (permalink / raw)
To: David Hildenbrand, Christian Borntraeger, Alex Bennée
Cc: Tony Krowiak, kvm, supporter, Benjamin Herrenschmidt,
Heiko Carstens, Paul Mackerras, H. Peter Anvin, kvmarm, S390,
Michael Mueller, Nadav Amit, Michael Ellerman,
maintainer:X86 ARCHITECTURE..., Gleb Natapov, Ingo Molnar,
zhichao.huang, r65777, bp, Jason J. Herne, marc.zyngier, pbonzini,
Cornelia Huck, Thomas Gleixner, linux-arm-kernel, Eric Farman,
list
In-Reply-To: <20150515193315.6d3e82d2@thinkpad-w530>
On 2015-05-15 19:33, David Hildenbrand wrote:
>> Am 15.05.2015 um 16:27 schrieb Alex Bennée:
>>> +++ b/arch/s390/include/uapi/asm/kvm.h
>>> @@ -114,8 +114,6 @@ struct kvm_fpu {
>>> __u64 fprs[16];
>>> };
>>>
>>> -#define KVM_GUESTDBG_USE_HW_BP 0x00010000
>> [...]
>>> +++ b/include/uapi/linux/kvm.h
>> [...]
>>> +#define KVM_GUESTDBG_USE_SW_BP (1 << 16)
>>> +#define KVM_GUESTDBG_USE_HW_BP (1 << 17)
>>
>> This is an ABI break for s390, no?
>>
>> David, do you remember why we do not use KVM_GUESTDBG_USE_SW_BP?
>>
>
> We never had to tell the kernel about software breakpoints as this is all
> handled via 4 byte DIAG instructions until now. We don't have to turn this
> mechanism on. QEMU can directly insert the desired DIAG instructions and gets
> notified when they are about to get executed.
>
> (But we still have 2 byte breakpoint support todo - still tbd how exactly this
> will be realized - could be turned on via such a mechanism)
>
> The problem is, that these bits are arch specific, now Alex wants to unify
> them for all archs.
>
> So yes, this is an ABI break for us and breaks hardware breakpoints.(I think
> the first version of this patch didn't contain this ABI break when I had a look)
>
> I wonder if it wouldn't make more sense to
>
> - introduce new bits in the arch-unspecific section
> - rework the existing implementers to accept both bits
>
> Or to simply leave stuff as it is and handle it via arch specific bits.
With one arch proving the "all need this" theory wrong, just drop this
patch. Even quicker when it breaks an ABI.
Jan
--
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply
* Re: [PATCH 10/23] userfaultfd: add new syscall to provide memory externalization
From: Linus Torvalds @ 2015-05-15 18:22 UTC (permalink / raw)
To: Andrea Arcangeli
Cc: Andrew Morton, Linux Kernel Mailing List, linux-mm, qemu-devel,
KVM list, Linux API, Pavel Emelyanov, Sanidhya Kashyap,
zhang.zhanghailiang, Kirill A. Shutemov, Andres Lagar-Cavilla,
Dave Hansen, Paolo Bonzini, Rik van Riel, Mel Gorman,
Andy Lutomirski, Hugh Dickins, Peter Feiner,
Dr. David Alan Gilbert, Johannes Weiner, Huangpeng (Peter)
In-Reply-To: <20150515160426.GD19097@redhat.com>
On Fri, May 15, 2015 at 9:04 AM, Andrea Arcangeli <aarcange@redhat.com> wrote:
>
> To fix it I added this along a comment:
Ok, this looks good as a explanation/fix for the races (and also as an
example of my worry about waitqueue_active() use in general).
However, it now makes me suspect that the optimistic "let's check if
they are even active" may not be worth it any more. You're adding a
"smp_mb()" in order to avoid taking the real lock. Although I guess
there are two locks there (one for each wait-queue) so maybe it's
worth it.
Linus
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [GIT PULL] kselftest fixes for 4.1-rc4
From: Shuah Khan @ 2015-05-15 18:47 UTC (permalink / raw)
To: Linus Torvalds
Cc: Ingo Molnar, Andy Lutomirski, Tyler Baker,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA, Shuah Khan
Hi Linus,
Please pull the following Kselftest fixes for 4.1-rc4
thanks,
-- Shuah
The following changes since commit b787f68c36d49bb1d9236f403813641efa74a031:
Linux 4.1-rc1 (2015-04-26 17:59:10 -0700)
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
tags/linux-kselftest-4.1-rc4
for you to fetch changes up to e9886ace222eb48bb57bd541320056ca334bd3a0:
selftests, x86: Rework x86 target architecture detection (2015-05-12
20:02:40 -0600)
----------------------------------------------------------------
linux-kselftest-4.1-rc4
Urgent fix for Kselftest regression introduced in 4.1-rc1
by the new x86 test due to its hard dependency on 32-bit
build environment. A set of 5 patches fix the make kselftest
run and kselftest install.
----------------------------------------------------------------
Andy Lutomirski (2):
selftests, x86: Remove useless run_tests rule
selftests, x86: Rework x86 target architecture detection
Tyler Baker (3):
selftest/x86: build both bitnesses
selftest/x86: have no dependency on all when cross building
selftests/x86: install tests
tools/testing/selftests/x86/Makefile | 53
+++++++++++++---------
tools/testing/selftests/x86/check_cc.sh | 16 +++++++
tools/testing/selftests/x86/run_x86_tests.sh | 13 ------
.../testing/selftests/x86/trivial_32bit_program.c | 4 ++
.../testing/selftests/x86/trivial_64bit_program.c | 18 ++++++++
5 files changed, 69 insertions(+), 35 deletions(-)
create mode 100755 tools/testing/selftests/x86/check_cc.sh
delete mode 100644 tools/testing/selftests/x86/run_x86_tests.sh
create mode 100644 tools/testing/selftests/x86/trivial_64bit_program.c
--
Shuah Khan
Sr. Linux Kernel Developer
Open Source Innovation Group
Samsung Research America (Silicon Valley)
shuahkh-JPH+aEBZ4P+UEJcrhfAQsw@public.gmane.org | (970) 217-8978
^ permalink raw reply
* Re: [PATCH v8 12/16] ARM: Add STM32 family machine
From: Arnd Bergmann @ 2015-05-15 19:57 UTC (permalink / raw)
To: Maxime Coquelin
Cc: u.kleine-koenig-bIcnvbaLZ9MEGnE8C9+IrQ, afaerber-l3A5Bk7waGM,
geert-Td1EMuHUCqxL1ZNQvxDV9g, Rob Herring, Philipp Zabel,
Linus Walleij, stefan-XLVq0VzYD2Y, pmeerw-jW+XmwGofnusTnJN9+BGXg,
pebolle-IWqWACnzNjzz+pZb47iToQ,
peter-WaGBZJeGNqdsbIuE7sb01tBPR1lH4CV8,
andy.shevchenko-Re5JQEeQqe8AvxtiuMwx3w,
cw00.choi-Sze3O3UU22JBDgjK7y7TUQ, Russell King, Daniel Lezcano,
joe-6d6DIl74uiNBDgjK7y7TUQ, Vladimir Zapolskiy,
lee.jones-QSEj5FYQhm4dnm+yROfE0A, Daniel Thompson,
Jonathan Corbet, Pawel Moll, Mark Rutland, Ian Campbell,
Kumar Gala, Thomas Gleixner, Greg Kroah-Hartman, Jiri Slaby,
Andrew Morton
In-Reply-To: <1431158038-3813-13-git-send-email-mcoquelin.stm32-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
On Saturday 09 May 2015 09:53:54 Maxime Coquelin wrote:
> STMicrolectronics's STM32 series is a family of Cortex-M
> microcontrollers. It is used in various applications, and
> proposes a wide range of peripherals.
>
> Tested-by: Chanwoo Choi <cw00.choi-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
> Signed-off-by: Maxime Coquelin <mcoquelin.stm32-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Applied to next/soc, thanks
Arnd
^ permalink raw reply
* Re: [PATCH v8 16/16] MAINTAINERS: Add entry for STM32 MCUs
From: Arnd Bergmann @ 2015-05-15 19:58 UTC (permalink / raw)
To: Maxime Coquelin
Cc: u.kleine-koenig, afaerber, geert, Rob Herring, Philipp Zabel,
Linus Walleij, stefan, pmeerw, pebolle, peter, andy.shevchenko,
cw00.choi, Russell King, Daniel Lezcano, joe, Vladimir Zapolskiy,
lee.jones, Daniel Thompson, Jonathan Corbet, Pawel Moll,
Mark Rutland, Ian Campbell, Kumar Gala, Thomas Gleixner,
Greg Kroah-Hartman, Jiri Slaby, Andrew Morton
In-Reply-To: <1431158038-3813-17-git-send-email-mcoquelin.stm32@gmail.com>
On Saturday 09 May 2015 09:53:58 Maxime Coquelin wrote:
> Add a MAINTAINER entry covering all STM32 machine and drivers files.
>
> Signed-off-by: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> ---
> MAINTAINERS | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 2e5bbc0..858d821 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1479,6 +1479,14 @@ F: drivers/usb/host/ehci-st.c
> F: drivers/usb/host/ohci-st.c
> F: drivers/ata/ahci_st.c
>
> +ARM/STM32 ARCHITECTURE
> +M: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> +L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
> +S: Maintained
> +T: git git://git.kernel.org/pub/scm/linux/kernel/git/mcoquelin/stm32.git
> +N: stm32
> +F: drivers/clocksource/armv7m_systick.c
> +
> ARM/TECHNOLOGIC SYSTEMS TS7250 MACHINE SUPPORT
> M: Lennert Buytenhek <kernel@wantstofly.org>
> L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
>
Applied to next/soc, thanks!
Arnd
^ permalink raw reply
* Re: [PATCH v8 13/16] ARM: dts: Add ARM System timer as clocksource in armv7m
From: Arnd Bergmann @ 2015-05-15 19:59 UTC (permalink / raw)
To: Maxime Coquelin
Cc: u.kleine-koenig, afaerber, geert, Rob Herring, Philipp Zabel,
Linus Walleij, stefan, pmeerw, pebolle, peter, andy.shevchenko,
cw00.choi, Russell King, Daniel Lezcano, joe, Vladimir Zapolskiy,
lee.jones, Daniel Thompson, Jonathan Corbet, Pawel Moll,
Mark Rutland, Ian Campbell, Kumar Gala, Thomas Gleixner,
Greg Kroah-Hartman, Jiri Slaby, Andrew Morton
In-Reply-To: <1431158038-3813-14-git-send-email-mcoquelin.stm32@gmail.com>
On Saturday 09 May 2015 09:53:55 Maxime Coquelin wrote:
> Tested-by: Chanwoo Choi <cw00.choi@samsung.com>
> Signed-off-by: Maxime Coquelin <mcoquelin.stm32@gmail.com>
> ---
> arch/arm/boot/dts/armv7-m.dtsi | 6 ++++++
> 1 file changed, 6 insertions(+)
>
Applied to next/dt. I've left out patch 14 for now, until we have a new version.
Arnd
^ permalink raw reply
* Re: [PATCH v8 15/16] ARM: configs: Add STM32 defconfig
From: Arnd Bergmann @ 2015-05-15 20:03 UTC (permalink / raw)
To: Maxime Coquelin
Cc: u.kleine-koenig-bIcnvbaLZ9MEGnE8C9+IrQ, afaerber-l3A5Bk7waGM,
geert-Td1EMuHUCqxL1ZNQvxDV9g, Rob Herring, Philipp Zabel,
Linus Walleij, stefan-XLVq0VzYD2Y, pmeerw-jW+XmwGofnusTnJN9+BGXg,
pebolle-IWqWACnzNjzz+pZb47iToQ,
peter-WaGBZJeGNqdsbIuE7sb01tBPR1lH4CV8,
andy.shevchenko-Re5JQEeQqe8AvxtiuMwx3w,
cw00.choi-Sze3O3UU22JBDgjK7y7TUQ, Russell King, Daniel Lezcano,
joe-6d6DIl74uiNBDgjK7y7TUQ, Vladimir Zapolskiy,
lee.jones-QSEj5FYQhm4dnm+yROfE0A, Daniel Thompson,
Jonathan Corbet, Pawel Moll, Mark Rutland, Ian Campbell,
Kumar Gala, Thomas Gleixner, Greg Kroah-Hartman, Jiri Slaby,
Andrew Morton
In-Reply-To: <1431158038-3813-16-git-send-email-mcoquelin.stm32-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
On Saturday 09 May 2015 09:53:57 Maxime Coquelin wrote:
> This patch adds a new config for STM32 MCUs.
> STM32F429 Discovery board boots successfully with this config applied.
>
> Tested-by: Chanwoo Choi <cw00.choi-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
> Signed-off-by: Maxime Coquelin <mcoquelin.stm32-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
>
Applied to next/defconfig, thanks!
Arnd
^ permalink raw reply
* Re: [PATCH V6 05/10] audit: log creation and deletion of namespace instances
From: Paul Moore @ 2015-05-15 20:26 UTC (permalink / raw)
To: Richard Guy Briggs
Cc: Steve Grubb, containers, linux-kernel, linux-audit, eparis,
arozansk, ebiederm, serge, zohar, viro, linux-fsdevel, linux-api,
netdev
In-Reply-To: <20150515004855.GB10526@madcap2.tricolour.ca>
On Thursday, May 14, 2015 08:48:55 PM Richard Guy Briggs wrote:
> On 15/05/14, Steve Grubb wrote:
> > What they would want to know is what resources were assigned; if two
> > containers shared a resource, what resource and container was it shared
> > with; if two containers can communicate, we need to see or control
> > information flow when necessary; and we need to see termination and
> > release of resources.
>
> So, namespaces are a big part of this. I understand how they are
> spawned and potentially shared. I have a more vague idea about how
> cgroups contribute to this concept of a container. So far, I have very
> little idea how seccomp contributes, but I assume that it will also need
> to be part of this tracking.
It doesn't, really. We shouldn't worry about seccomp from a
namespace/container auditing perspective. The normal seccomp auditing should
be sufficient for namespaces/containers.
--
paul moore
security @ redhat
^ permalink raw reply
* Re: [PATCH V6 05/10] audit: log creation and deletion of namespace instances
From: Paul Moore @ 2015-05-15 20:42 UTC (permalink / raw)
To: Oren Laadan
Cc: Richard Guy Briggs, Steve Grubb, linux-api-u79uwXL29TY76Z2rM5mHXA,
Linux Containers, linux-kernel,
viro-RmSDqhL/yNMiFSDQTTA3OLVCufUGDwFn,
linux-audit-H+wXaHxf7aLQT0dZR+AlfA, netdev-u79uwXL29TY76Z2rM5mHXA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
eparis-FjpueFixGhCM4zKIHC2jIg,
zohar-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8, Eric Biederman
In-Reply-To: <CAA4jN2bgynVTwF+owtXgq06JMLQJpy_qokpD0mAguNYeDxmh1A-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
On Thursday, May 14, 2015 09:10:56 PM Oren Laadan wrote:
> [focusing on "containers id" - snipped the rest away]
>
> I am unfamiliar with the audit subsystem, but work with namespaces in other
> contexts. Perhaps the term "container" is overloaded here. The definition
> suggested by Steve in this thread makes sense to me: "a combination of
> namespaces". I imagine people may want to audit subsets of namespaces.
>
> For namespaces, can use a string like "A:B:C:D:E:F" as an identifier for a
> particular combination, where A-F are respective namespaces identifiers.
> (Can be taken for example from /proc/PID/ns/{mnt,uts,ipc,user,pid,net}).
> That will even be grep-able to locate records related to a particular
> subset
> of namespaces. So a "container" in the classic meaning would have all A-F
> unique and different from the init process, but processes separated only by
> e.g. mnt-ns and net-ns will differ from the init process in A and F.
>
> (If a string is a no go, then perhaps combine the IDs in a unique way into a
> super ID).
As has been mentioned in every other email in this thread, the kernel has no
concept of a container, it is a userspace idea and trying to generate a
meaningful value in the kernel is a mistake in my opinion. My current opinion
is that we allow userspace to set a container ID token as it sees fit and the
kernel will just use the value provided by userspace.
--
paul moore
security @ redhat
^ permalink raw reply
* Re: [PATCH V6 05/10] audit: log creation and deletion of namespace instances
From: Paul Moore @ 2015-05-15 21:01 UTC (permalink / raw)
To: Eric W. Biederman
Cc: Steve Grubb, Richard Guy Briggs, containers, linux-kernel,
linux-audit, eparis, arozansk, serge, zohar, viro, linux-fsdevel,
linux-api, netdev
In-Reply-To: <87bnhmbp8e.fsf@x220.int.ebiederm.org>
On Thursday, May 14, 2015 08:31:45 PM Eric W. Biederman wrote:
> Paul Moore <pmoore@redhat.com> writes:
> > As Eric, and others, have stated, the container concept is a userspace
> > idea, not a kernel idea; the kernel only knows, and cares about,
> > namespaces. This is unlikely to change.
> >
> > However, as Steve points out, there is precedence for the kernel to record
> > userspace tokens for the sake of audit. Personally I'm not a big fan of
> > this in general, but I do recognize that it does satisfy a legitimate
> > need. Think of things like auid and the sessionid as necessary evils;
> > audit is already chock full of evilness I doubt one more will doom us all
> > to hell.
> >
> > Moving forward, I'd like to see the following:
> >
> > * Create a container ID token (unsigned 32-bit integer?), similar to
> > auid/sessionid, that is set by userspace and carried by the kernel to be
> > used in audit records. I'd like to see some discussion on how we manage
> > this, e.g. how do handle container ID inheritance, how do we handle
> > nested containers (setting the containerid when it is already set), do we
> > care if multiple different containers share the same namespace config,
> > etc.?
> >
> >
> > Can we all live with this? If not, please suggest some alternate ideas;
> > simply shouting "IT'S ALL CRAP!" isn't helpful for anyone ... it may be
> > true, but it doesn't help us solve the problem ;)
>
> Without stopping and defining what someone means by container I think it
> is pretty much nonsense.
For what it is worth, I doubt we will ever arrive at a consistent definition
of a container. This is one of the reasons why I don't think we want the
kernel generating a container ID token, although I understand the real world
desire to have the kernel report such information back in the audit logs.
> Should every vsftp connection get a container every? Every chrome tab?
That's up to the individual system. I would argue that's a pretty silly
configuration, but one persons silliness is another's best practice. It's a
mad, mad world.
> At some of the connections per second numbers I have seen we might
> exhaust a 32bit number in an hour or two. Will any of that make sense
> to someone reading the audit logs?
If someone if going to spawn each process in a container then they will need
to live with the fallout of that decision.
Also, if folks thing 32-bits is too small, we can always do 64-bits, but I
don't think that was the point you were trying to make (I could be wrong).
> Without considerning that container creation is an unprivileged
> operation I think it is pretty much nonsense. Do I get to say I am any
> container I want? That would seem to invalidate the concept of
> userspace setting a container id.
>
> How does any of this interact with setns? AKA entering a container?
As I said in my email, I think we need some discussion around this; I don't
pretend to think we have this sorted at this point. I just want to make sure
were working towards some common ground instead of shouting the same stuff
back and forth at each other.
> I will go as far as looking at patches. If someone comes up with
> a mission statement about what they are actually trying to achieve and a
> mechanism that actually achieves that, and that allows for containers to
> nest we can talk about doing something like that.
I think Steve has posted some requirements that Richard is trying to satisfy
with these patches; we've also heard from at least one person who is looking
at how to deploy this in the Real World. Perhaps in the next round of patches
Richard can list the requirements in the 0/X patch and describe how they are
satisfied in the patchset.
Beyond that, and ignoring for a moment the whole "a container is not a
*thing*" argument, can I assume that the auditing of nested "containers" are
your main remaining concern at this point?
> But for right now I just hear proposals for things that make no sense
> and can not possibly work. Not least because it will require modifying
> every program that creates a container and who knows how many of them
> there are. Especially since you don't need to be root. Modifying
> /usr/bin/unshare seems a little far out to me.
I think it is very reasonable that there will be some container infrastructure
tools which would handle this, we're already seeing this happening now; asking
for minor changes to these infrastructure applications to support container
auditing doesn't seem like a significant ask to me. Also, to be perfectly
clear, if the applications aren't updated it isn't as if they will fail to
work, it is just that they won't be able to take advantage of the new
container auditing capabilities. That seems reasonable to me.
--
paul moore
security @ redhat
^ permalink raw reply
* Re: [PATCH V6 05/10] audit: log creation and deletion of namespace instances
From: Paul Moore @ 2015-05-15 21:05 UTC (permalink / raw)
To: Andy Lutomirski
Cc: Linux API, Linux Containers,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Al Viro,
linux-audit-H+wXaHxf7aLQT0dZR+AlfA, Eric W. Biederman,
Network Development, Linux FS Devel, Eric Paris, Steve Grubb,
Mimi Zohar
In-Reply-To: <CALCETrWzM4+Vs8OVJWBcWJfbR_DRSb+e7SmUyy6CS4sHQaTkRw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
On Thursday, May 14, 2015 11:23:09 PM Andy Lutomirski wrote:
> On Thu, May 14, 2015 at 7:32 PM, Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> > On 15/05/14, Paul Moore wrote:
> >> * Look at our existing audit records to determine which records should
> >> have
> >> namespace and container ID tokens added. We may only want to add the
> >> additional fields in the case where the namespace/container ID tokens are
> >> not the init namespace.
> >
> > If we have a record that ties a set of namespace IDs with a container
> > ID, then I expect we only need to list the containerID along with auid
> > and sessionID.
>
> The problem here is that the kernel has no concept of a "container", and I
> don't think it makes any sense to add one just for audit. "Container" is a
> marketing term used by some userspace tools.
>
> I can imagine that both audit could benefit from a concept of a
> namespace *path* that understands nesting (e.g. root/2/5/1 or
> something along those lines). Mapping these to "containers" belongs
> in userspace, I think.
It might be helpful to climb up a few levels in this thread ...
I think we all agree that containers are not a kernel construct. I further
believe that the kernel has no business generating container IDs, those should
come from userspace and will likely be different depending on how you define
"container". However, what is less clear to me at this point is how the
kernel should handle the setting, reporting, and general management of this
container ID token.
--
paul moore
security @ redhat
^ permalink raw reply
* [PATCH RFC v2 0/2] O_NOCMTIME protected by generic mount option
From: Zach Brown @ 2015-05-15 21:23 UTC (permalink / raw)
To: Sage Weil, linux-fsdevel, linux-kernel, linux-api
Here's a current draft of what is now the O_NOCMTIME series. It
implements the frequent suggestion to gate unprivileged O_NOCMTIME use
with a mount option.
This method has the advantage of being entirely runtime. There's no
persistence that'd require updating all the tools that deal with each
file system's format. It's also requested by writers as they open.
Writes to the file that know nothing of O_NOCMTIME will behave as
usual.
Another suggested method is to use inode attributes: require root to
set +nocmtime on a dir and inherit it down subdirs to new files. This
nicely solves the unprivilieged use problem without having to fiddle
with mount options but it requires touching all systems that support
it and would prevent cmtime updates on all writes to the inode. I
have a patch series that starts on this but haven't taken it very far.
Sage is working on spinning up some hardware to test the various dirty
inode avoidance methods at load and should have numbers soon. That'll
tell us if lazytime isn't good enough and if any of this is worth the
trouble.
- z
^ permalink raw reply
* [PATCH RFC v2 1/2] vfs: add generic nocmtime mount flag
From: Zach Brown @ 2015-05-15 21:23 UTC (permalink / raw)
To: Sage Weil, linux-fsdevel, linux-kernel, linux-api
In-Reply-To: <1431725028-24071-1-git-send-email-zab@zabbo.net>
Add the infrastructure to support a generic nocmtime mount flag. Like
MS_NOATIME/MNT_NOATIME this can be used to support the mount option in
file systems without having to touch each file system.
This will be used to provide a priviledged indication that unpriviledged
apps can safely use O_NOCMTIME to prevent cmtime updates without harm.
Signed-off-by: Zach Brown <zab@zabbo.net>
---
fs/namespace.c | 2 ++
fs/proc_namespace.c | 1 +
fs/statfs.c | 2 ++
include/linux/mount.h | 1 +
include/linux/statfs.h | 1 +
include/uapi/linux/fs.h | 1 +
6 files changed, 8 insertions(+)
diff --git a/fs/namespace.c b/fs/namespace.c
index 1b9e111..48be1f9 100644
--- a/fs/namespace.c
+++ b/fs/namespace.c
@@ -2652,6 +2652,8 @@ long do_mount(const char *dev_name, const char __user *dir_name,
mnt_flags |= MNT_NOEXEC;
if (flags & MS_NOATIME)
mnt_flags |= MNT_NOATIME;
+ if (flags & MS_NOCMTIME)
+ mnt_flags |= MNT_NOCMTIME;
if (flags & MS_NODIRATIME)
mnt_flags |= MNT_NODIRATIME;
if (flags & MS_STRICTATIME)
diff --git a/fs/proc_namespace.c b/fs/proc_namespace.c
index 8db932d..49d7839 100644
--- a/fs/proc_namespace.c
+++ b/fs/proc_namespace.c
@@ -64,6 +64,7 @@ static void show_mnt_opts(struct seq_file *m, struct vfsmount *mnt)
{ MNT_NODEV, ",nodev" },
{ MNT_NOEXEC, ",noexec" },
{ MNT_NOATIME, ",noatime" },
+ { MNT_NOCMTIME, ",nocmtime" },
{ MNT_NODIRATIME, ",nodiratime" },
{ MNT_RELATIME, ",relatime" },
{ 0, NULL }
diff --git a/fs/statfs.c b/fs/statfs.c
index 083dc0a..43d3de2 100644
--- a/fs/statfs.c
+++ b/fs/statfs.c
@@ -23,6 +23,8 @@ static int flags_by_mnt(int mnt_flags)
flags |= ST_NOEXEC;
if (mnt_flags & MNT_NOATIME)
flags |= ST_NOATIME;
+ if (mnt_flags & MNT_NOCMTIME)
+ flags |= ST_NOCMTIME;
if (mnt_flags & MNT_NODIRATIME)
flags |= ST_NODIRATIME;
if (mnt_flags & MNT_RELATIME)
diff --git a/include/linux/mount.h b/include/linux/mount.h
index f822c3c..deb458f 100644
--- a/include/linux/mount.h
+++ b/include/linux/mount.h
@@ -28,6 +28,7 @@ struct mnt_namespace;
#define MNT_NODIRATIME 0x10
#define MNT_RELATIME 0x20
#define MNT_READONLY 0x40 /* does the user want this to be r/o? */
+#define MNT_NOCMTIME 0x80 /* allow O_NOCMTIME to stop cmtime updates */
#define MNT_SHRINKABLE 0x100
#define MNT_WRITE_HOLD 0x200
diff --git a/include/linux/statfs.h b/include/linux/statfs.h
index 0166d32..bde224e 100644
--- a/include/linux/statfs.h
+++ b/include/linux/statfs.h
@@ -39,5 +39,6 @@ struct kstatfs {
#define ST_NOATIME 0x0400 /* do not update access times */
#define ST_NODIRATIME 0x0800 /* do not update directory access times */
#define ST_RELATIME 0x1000 /* update atime relative to mtime/ctime */
+#define ST_NOCMTIME 0x2000 /* allow O_NOCMTIME to stop cmtime updates */
#endif
diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h
index 9b964a5..af1131e 100644
--- a/include/uapi/linux/fs.h
+++ b/include/uapi/linux/fs.h
@@ -91,6 +91,7 @@ struct inodes_stat_t {
#define MS_I_VERSION (1<<23) /* Update inode I_version field */
#define MS_STRICTATIME (1<<24) /* Always perform atime updates */
#define MS_LAZYTIME (1<<25) /* Update the on-disk [acm]times lazily */
+#define MS_NOCMTIME (1<<26) /* allow O_NOCMTIME to stop cmtime updates */
/* These sb flags are internal to the kernel */
#define MS_NOSEC (1<<28)
--
2.1.0
^ permalink raw reply related
* [PATCH RFC v2 2/2] vfs: add O_NOCMTIME
From: Zach Brown @ 2015-05-15 21:23 UTC (permalink / raw)
To: Sage Weil, linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <1431725028-24071-1-git-send-email-zab-ugsP4Wv/S6ZeoWH0uzbU5w@public.gmane.org>
Add a O_NOCMTIME flag which prevents inode time updates on writes and
can greatly reduce the IO overhead of writes to allocated and
initialized regions of files.
ceph servers can have loads where they perform O_DIRECT overwrites of
allocated file data and then sync to make sure that the O_DIRECT writes
are flushed from write caches. If the writes dirty the inode with mtime
updates then the syncs also write out the metadata needed to track the
inodes which can add significant iop and latency overhead.
The ceph servers don't use mtime at all. They're using the local file
system as a backing store and any backups would be driven by their upper
level ceph metadata.
In simple tests a O_DIRECT|O_NOCMTIME overwriting write followed by a
sync went from 2 serial write round trips to 1 in XFS and from 4 serial
IO round trips to 1 in ext4.
file_update_time() is changed to call a file_is_nocmtime() helper which
tests the file flag in addition to testing the inode's S_NOCMTIME flag.
It doesn't check FMODE_NOCMTIME because that's only used by XFS to
trigger private flags which trigger private flags which do other things.
O_NOCMTIME can only be used if the mount has its MNT_NOCMTIME flag set.
This requires priviledged intervention to testify that mtime isn't
critical to, say, backup infrastructure or NFS server consistency
guarantees.
Signed-off-by: Zach Brown <zab-ugsP4Wv/S6ZeoWH0uzbU5w@public.gmane.org>
---
fs/fcntl.c | 30 +++++++++++++++++++++++-------
fs/inode.c | 2 +-
fs/namei.c | 3 +--
include/linux/fs.h | 7 +++++++
include/uapi/asm-generic/fcntl.h | 4 ++++
5 files changed, 36 insertions(+), 10 deletions(-)
diff --git a/fs/fcntl.c b/fs/fcntl.c
index ee85cd4..eaa5d1d 100644
--- a/fs/fcntl.c
+++ b/fs/fcntl.c
@@ -22,12 +22,30 @@
#include <linux/pid_namespace.h>
#include <linux/user_namespace.h>
#include <linux/shmem_fs.h>
+#include <linux/mount.h>
#include <asm/poll.h>
#include <asm/siginfo.h>
#include <asm/uaccess.h>
-#define SETFL_MASK (O_APPEND | O_NONBLOCK | O_NDELAY | O_DIRECT | O_NOATIME)
+#define SETFL_MASK (O_APPEND | O_NONBLOCK | O_NDELAY | O_DIRECT | O_NOATIME | \
+ O_NOCMTIME)
+
+/*
+ * O_NOATIME and O_NOCMTIME can only be set by the ownder or superuser.
+ * And O_NOCMTIME requires MNT_NOCMTIME.
+ */
+bool forbid_o_notime(struct inode *inode, struct vfsmount *mnt,
+ unsigned long flags)
+{
+ if ((flags & (O_NOATIME|O_NOCMTIME)) && !inode_owner_or_capable(inode))
+ return true;
+
+ if ((flags & O_NOCMTIME) && !(mnt->mnt_flags & MNT_NOCMTIME))
+ return true;
+
+ return false;
+}
static int setfl(int fd, struct file * filp, unsigned long arg)
{
@@ -41,10 +59,8 @@ static int setfl(int fd, struct file * filp, unsigned long arg)
if (((arg ^ filp->f_flags) & O_APPEND) && IS_APPEND(inode))
return -EPERM;
- /* O_NOATIME can only be set by the owner or superuser */
- if ((arg & O_NOATIME) && !(filp->f_flags & O_NOATIME))
- if (!inode_owner_or_capable(inode))
- return -EPERM;
+ if (forbid_o_notime(inode, filp->f_path.mnt, arg & ~filp->f_flags))
+ return -EPERM;
/* required for strict SunOS emulation */
if (O_NONBLOCK != O_NDELAY)
@@ -740,7 +756,7 @@ static int __init fcntl_init(void)
* Exceptions: O_NONBLOCK is a two bit define on parisc; O_NDELAY
* is defined as O_NONBLOCK on some platforms and not on others.
*/
- BUILD_BUG_ON(21 - 1 /* for O_RDONLY being 0 */ != HWEIGHT32(
+ BUILD_BUG_ON(22 - 1 /* for O_RDONLY being 0 */ != HWEIGHT32(
O_RDONLY | O_WRONLY | O_RDWR |
O_CREAT | O_EXCL | O_NOCTTY |
O_TRUNC | O_APPEND | /* O_NONBLOCK | */
@@ -748,7 +764,7 @@ static int __init fcntl_init(void)
O_DIRECT | O_LARGEFILE | O_DIRECTORY |
O_NOFOLLOW | O_NOATIME | O_CLOEXEC |
__FMODE_EXEC | O_PATH | __O_TMPFILE |
- __FMODE_NONOTIFY
+ __FMODE_NONOTIFY| O_NOCMTIME
));
fasync_cache = kmem_cache_create("fasync_cache",
diff --git a/fs/inode.c b/fs/inode.c
index ea37cd1..b643dd0 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -1721,7 +1721,7 @@ int file_update_time(struct file *file)
int ret;
/* First try to exhaust all avenues to not sync */
- if (IS_NOCMTIME(inode))
+ if (file_is_nocmtime(file))
return 0;
now = current_fs_time(inode->i_sb);
diff --git a/fs/namei.c b/fs/namei.c
index fe30d3b..8ecebca 100644
--- a/fs/namei.c
+++ b/fs/namei.c
@@ -2617,8 +2617,7 @@ static int may_open(struct path *path, int acc_mode, int flag)
return -EPERM;
}
- /* O_NOATIME can only be set by the owner or superuser */
- if (flag & O_NOATIME && !inode_owner_or_capable(inode))
+ if (forbid_o_notime(inode, path->mnt, flag))
return -EPERM;
return 0;
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 35ec87e..dd92eeb 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -1471,6 +1471,8 @@ static inline void sb_start_intwrite(struct super_block *sb)
extern bool inode_owner_or_capable(const struct inode *inode);
+extern bool forbid_o_notime(struct inode *inode, struct vfsmount *mnt,
+ unsigned long flags);
/*
* VFS helper functions..
@@ -2950,6 +2952,11 @@ static inline bool is_root_inode(struct inode *inode)
return inode == inode->i_sb->s_root->d_inode;
}
+static inline bool file_is_nocmtime(struct file *file)
+{
+ return IS_NOCMTIME(file_inode(file)) || (file->f_flags & O_NOCMTIME);
+}
+
static inline bool dir_emit(struct dir_context *ctx,
const char *name, int namelen,
u64 ino, unsigned type)
diff --git a/include/uapi/asm-generic/fcntl.h b/include/uapi/asm-generic/fcntl.h
index e063eff..ed7b2e1 100644
--- a/include/uapi/asm-generic/fcntl.h
+++ b/include/uapi/asm-generic/fcntl.h
@@ -88,6 +88,10 @@
#define __O_TMPFILE 020000000
#endif
+#ifndef O_NOCMTIME
+#define O_NOCMTIME 040000000
+#endif
+
/* a horrid kludge trying to make sure that this will fail on old kernels */
#define O_TMPFILE (__O_TMPFILE | O_DIRECTORY)
#define O_TMPFILE_MASK (__O_TMPFILE | O_DIRECTORY | O_CREAT)
--
2.1.0
^ permalink raw reply related
* Re: [PATCH 5/6] nohz: support PR_DATAPLANE_STRICT mode
From: Chris Metcalf @ 2015-05-15 21:25 UTC (permalink / raw)
To: Andy Lutomirski
Cc: Paul E. McKenney, Frederic Weisbecker,
linux-kernel@vger.kernel.org, Rik van Riel, Andrew Morton,
Linux API, Thomas Gleixner, Tejun Heo, Peter Zijlstra,
Steven Rostedt, linux-doc@vger.kernel.org, Christoph Lameter,
Gilad Ben Yossef, Ingo Molnar
In-Reply-To: <CALCETrXVLGi=DSHza2fCXDNbkh_MOAbxQANNg738=AWVMsYPrw@mail.gmail.com>
On 05/12/2015 06:23 PM, Andy Lutomirski wrote:
> On May 13, 2015 6:06 AM, "Chris Metcalf" <cmetcalf@ezchip.com> wrote:
>> On 05/11/2015 06:28 PM, Andy Lutomirski wrote:
>>> On Mon, May 11, 2015 at 12:13 PM, Chris Metcalf <cmetcalf@ezchip.com> wrote:
>>>> In this case, killing the task is appropriate, since that's exactly
>>>> the semantics that have been asked for - it's like on architectures
>>>> that don't natively support unaligned accesses, but fake it relatively
>>>> slowly in the kernel, and in development you just say "give me a
>>>> SIGBUS when that happens" and in production you might say
>>>> "fix it up and let's try to keep going".
>>> I think more control is needed. I also think that, if we go this
>>> route, we should distinguish syscalls, synchronous non-syscall
>>> entries, and asynchronous non-syscall entries. They're quite
>>> different.
>>
>> I don't think it's necessary to distinguish the types. As long as we
>> have a PC pointing to the instruction that triggered the problem,
>> we can see if it's a system call instruction, a memory write that
>> caused a page fault, a trap instruction, etc.
> Not true. PC right after a syscall insn could be any type of kernel
> entry, and you can't even reliably tell whether the syscall insn was
> executed or, on x86, whether it was a syscall at all. (x86 insns
> can't be reliably decided backwards.)
>
> PC pointing at a load could be a page fault or an IPI.
All that we are trying to do with this API, though, is distinguish
synchronous faults. So IPIs, etc., should not be happening
(they would be bugs), and hopefully we are mostly just
distinguishing different types of synchronous program entries.
That said, I did a si_info flag to differentiate syscalls from other
synchronous entries, and I'm open to looking at more such if
it seems useful.
> Again, though, I think we really do need to distinguish at least MCE
> and NMI (on x86) from the others.
Yes, those are both interesting cases, and I'm not entirely
sure what the right way to handle them is - for example,
likely disable STRICT if you are running with perf enabled.
I look forward to hearing more when you're back next week!
--
Chris Metcalf, EZChip Semiconductor
http://www.ezchip.com
^ permalink raw reply
* [PATCH v2 0/5] support "cpu_isolated" mode for nohz_full
From: Chris Metcalf @ 2015-05-15 21:26 UTC (permalink / raw)
To: Gilad Ben Yossef, Steven Rostedt, Ingo Molnar, Peter Zijlstra,
Andrew Morton, Rik van Riel, Tejun Heo, Frederic Weisbecker,
Thomas Gleixner, Paul E. McKenney, Christoph Lameter,
Viresh Kumar, linux-doc-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Cc: Chris Metcalf
In-Reply-To: <1431107927-13998-1-git-send-email-cmetcalf-d5a29ZRxExrQT0dZR+AlfA@public.gmane.org>
The existing nohz_full mode does a nice job of suppressing extraneous
kernel interrupts for cores that desire it. However, there is a need
for a more deterministic mode that rigorously disallows kernel
interrupts, even at a higher cost in user/kernel transition time:
for example, high-speed networking applications running userspace
drivers that will drop packets if they are ever interrupted.
These changes attempt to provide an initial draft of such a framework;
the changes do not add any overhead to the usual non-nohz_full mode,
and only very small overhead to the typical nohz_full mode. A prctl()
option (PR_SET_CPU_ISOLATED) is added to control whether processes have
requested this stricter semantics, and within that prctl() option we
provide a number of different bits for more precise control.
Additionally, we add a new command-line boot argument to facilitate
debugging where unexpected interrupts are being delivered from.
Code that is conceptually similar has been in use in Tilera's
Multicore Development Environment since 2008, known as Zero-Overhead
Linux, and has seen wide adoption by a range of customers. This patch
series represents the first serious attempt to upstream that
functionality. Although the current state of the kernel isn't quite
ready to run with absolutely no kernel interrupts (for example,
workqueues on cpu_isolated cores still remain to be dealt with), this
patch series provides a way to make dynamic tradeoffs between avoiding
kernel interrupts on the one hand, and making voluntary calls in and
out of the kernel more expensive, for tasks that want it.
The series (based currently on my arch/tile master tree for 4.2,
in turn based on 4.1-rc1) is available at:
git://git.kernel.org/pub/scm/linux/kernel/git/cmetcalf/linux-tile.git dataplane
v2:
rename "dataplane" to "cpu_isolated"
drop ksoftirqd suppression changes (believed no longer needed)
merge previous "QUIESCE" functionality into baseline functionality
explicitly track syscalls and exceptions for "STRICT" functionality
allow configuring a signal to be delivered for STRICT mode failures
move debug tracking to irq_enter(), not irq_exit()
Note: I have not yet removed the hack to disable the 1Hz timer tick
fallback that was nack'ed by PeterZ, pending a decision on that thread
as to what to do (https://lkml.org/lkml/2015/5/8/555).
Chris Metcalf (5):
nohz_full: add support for "cpu_isolated" mode
nohz: support PR_CPU_ISOLATED_STRICT mode
nohz: cpu_isolated strict mode configurable signal
nohz: add cpu_isolated_debug boot flag
nohz: cpu_isolated: allow tick to be fully disabled
Documentation/kernel-parameters.txt | 6 +++
arch/tile/kernel/ptrace.c | 6 ++-
arch/tile/mm/homecache.c | 5 +-
arch/x86/kernel/ptrace.c | 2 +
include/linux/context_tracking.h | 11 +++--
include/linux/sched.h | 3 ++
include/linux/tick.h | 28 +++++++++++
include/uapi/linux/prctl.h | 8 +++
kernel/context_tracking.c | 12 +++--
kernel/irq_work.c | 4 +-
kernel/sched/core.c | 18 +++++++
kernel/signal.c | 5 ++
kernel/smp.c | 4 ++
kernel/softirq.c | 6 +++
kernel/sys.c | 8 +++
kernel/time/tick-sched.c | 98 ++++++++++++++++++++++++++++++++++++-
16 files changed, 214 insertions(+), 10 deletions(-)
--
2.1.2
^ permalink raw reply
* [PATCH v2 1/5] nohz_full: add support for "cpu_isolated" mode
From: Chris Metcalf @ 2015-05-15 21:27 UTC (permalink / raw)
To: Gilad Ben Yossef, Steven Rostedt, Ingo Molnar, Peter Zijlstra,
Andrew Morton, Rik van Riel, Tejun Heo, Frederic Weisbecker,
Thomas Gleixner, Paul E. McKenney, Christoph Lameter,
Viresh Kumar, linux-doc, linux-api, linux-kernel
Cc: Chris Metcalf
In-Reply-To: <1431725178-20876-1-git-send-email-cmetcalf@ezchip.com>
The existing nohz_full mode makes tradeoffs to minimize userspace
interruptions while still attempting to avoid overheads in the
kernel entry/exit path, to provide 100% kernel semantics, etc.
However, some applications require a stronger commitment from the
kernel to avoid interruptions, in particular userspace device
driver style applications, such as high-speed networking code.
This change introduces a framework to allow applications to elect
to have the stronger semantics as needed, specifying
prctl(PR_SET_CPU_ISOLATED, PR_CPU_ISOLATED_ENABLE) to do so.
Subsequent commits will add additional flags and additional
semantics.
The "cpu_isolated" state is indicated by setting a new task struct
field, cpu_isolated_flags, to the value passed by prctl(). When the
_ENABLE bit is set for a task, and it is returning to userspace
on a nohz_full core, it calls the new tick_nohz_cpu_isolated_enter()
routine to take additional actions to help the task avoid being
interrupted in the future.
Initially, there are only two actions taken. First, the task
calls lru_add_drain() to prevent being interrupted by a subsequent
lru_add_drain_all() call on another core. Then, the code checks for
pending timer interrupts and quiesces until they are no longer pending.
As a result, sys calls (and page faults, etc.) can be inordinately slow.
However, this quiescing guarantees that no unexpected interrupts will
occur, even if the application intentionally calls into the kernel.
Signed-off-by: Chris Metcalf <cmetcalf@ezchip.com>
---
include/linux/sched.h | 3 +++
include/linux/tick.h | 10 +++++++++
include/uapi/linux/prctl.h | 5 +++++
kernel/context_tracking.c | 3 +++
kernel/sys.c | 8 ++++++++
kernel/time/tick-sched.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++
6 files changed, 80 insertions(+)
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 8222ae40ecb0..fb4ba400d7e1 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -1732,6 +1732,9 @@ struct task_struct {
#ifdef CONFIG_DEBUG_ATOMIC_SLEEP
unsigned long task_state_change;
#endif
+#ifdef CONFIG_NO_HZ_FULL
+ unsigned int cpu_isolated_flags;
+#endif
};
/* Future-safe accessor for struct task_struct's cpus_allowed. */
diff --git a/include/linux/tick.h b/include/linux/tick.h
index f8492da57ad3..ec1953474a65 100644
--- a/include/linux/tick.h
+++ b/include/linux/tick.h
@@ -10,6 +10,7 @@
#include <linux/context_tracking_state.h>
#include <linux/cpumask.h>
#include <linux/sched.h>
+#include <linux/prctl.h>
#ifdef CONFIG_GENERIC_CLOCKEVENTS
extern void __init tick_init(void);
@@ -134,11 +135,18 @@ static inline bool tick_nohz_full_cpu(int cpu)
return cpumask_test_cpu(cpu, tick_nohz_full_mask);
}
+static inline bool tick_nohz_is_cpu_isolated(void)
+{
+ return tick_nohz_full_cpu(smp_processor_id()) &&
+ (current->cpu_isolated_flags & PR_CPU_ISOLATED_ENABLE);
+}
+
extern void __tick_nohz_full_check(void);
extern void tick_nohz_full_kick(void);
extern void tick_nohz_full_kick_cpu(int cpu);
extern void tick_nohz_full_kick_all(void);
extern void __tick_nohz_task_switch(struct task_struct *tsk);
+extern void tick_nohz_cpu_isolated_enter(void);
#else
static inline bool tick_nohz_full_enabled(void) { return false; }
static inline bool tick_nohz_full_cpu(int cpu) { return false; }
@@ -147,6 +155,8 @@ static inline void tick_nohz_full_kick_cpu(int cpu) { }
static inline void tick_nohz_full_kick(void) { }
static inline void tick_nohz_full_kick_all(void) { }
static inline void __tick_nohz_task_switch(struct task_struct *tsk) { }
+static inline bool tick_nohz_is_cpu_isolated(void) { return false; }
+static inline void tick_nohz_cpu_isolated_enter(void) { }
#endif
static inline bool is_housekeeping_cpu(int cpu)
diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h
index 31891d9535e2..edb40b6b84db 100644
--- a/include/uapi/linux/prctl.h
+++ b/include/uapi/linux/prctl.h
@@ -190,4 +190,9 @@ struct prctl_mm_map {
# define PR_FP_MODE_FR (1 << 0) /* 64b FP registers */
# define PR_FP_MODE_FRE (1 << 1) /* 32b compatibility */
+/* Enable/disable or query cpu_isolated mode for NO_HZ_FULL kernels. */
+#define PR_SET_CPU_ISOLATED 47
+#define PR_GET_CPU_ISOLATED 48
+# define PR_CPU_ISOLATED_ENABLE (1 << 0)
+
#endif /* _LINUX_PRCTL_H */
diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index 72d59a1a6eb6..66739d7c1350 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -20,6 +20,7 @@
#include <linux/hardirq.h>
#include <linux/export.h>
#include <linux/kprobes.h>
+#include <linux/tick.h>
#define CREATE_TRACE_POINTS
#include <trace/events/context_tracking.h>
@@ -85,6 +86,8 @@ void context_tracking_enter(enum ctx_state state)
* on the tick.
*/
if (state == CONTEXT_USER) {
+ if (tick_nohz_is_cpu_isolated())
+ tick_nohz_cpu_isolated_enter();
trace_user_enter(0);
vtime_user_enter(current);
}
diff --git a/kernel/sys.c b/kernel/sys.c
index a4e372b798a5..3fd9e47f8fc8 100644
--- a/kernel/sys.c
+++ b/kernel/sys.c
@@ -2243,6 +2243,14 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3,
case PR_GET_FP_MODE:
error = GET_FP_MODE(me);
break;
+#ifdef CONFIG_NO_HZ_FULL
+ case PR_SET_CPU_ISOLATED:
+ me->cpu_isolated_flags = arg2;
+ break;
+ case PR_GET_CPU_ISOLATED:
+ error = me->cpu_isolated_flags;
+ break;
+#endif
default:
error = -EINVAL;
break;
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 914259128145..f1551c946c45 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -24,6 +24,7 @@
#include <linux/posix-timers.h>
#include <linux/perf_event.h>
#include <linux/context_tracking.h>
+#include <linux/swap.h>
#include <asm/irq_regs.h>
@@ -389,6 +390,56 @@ void __init tick_nohz_init(void)
pr_info("NO_HZ: Full dynticks CPUs: %*pbl.\n",
cpumask_pr_args(tick_nohz_full_mask));
}
+
+/*
+ * We normally return immediately to userspace.
+ *
+ * In "cpu_isolated" mode we wait until no more interrupts are
+ * pending. Otherwise we nap with interrupts enabled and wait for the
+ * next interrupt to fire, then loop back and retry.
+ *
+ * Note that if you schedule two "cpu_isolated" processes on the same
+ * core, neither will ever leave the kernel, and one will have to be
+ * killed manually. Otherwise in situations where another process is
+ * in the runqueue on this cpu, this task will just wait for that
+ * other task to go idle before returning to user space.
+ */
+void tick_nohz_cpu_isolated_enter(void)
+{
+ struct clock_event_device *dev =
+ __this_cpu_read(tick_cpu_device.evtdev);
+ struct task_struct *task = current;
+ unsigned long start = jiffies;
+ bool warned = false;
+
+ /* Drain the pagevecs to avoid unnecessary IPI flushes later. */
+ lru_add_drain();
+
+ while (ACCESS_ONCE(dev->next_event.tv64) != KTIME_MAX) {
+ if (!warned && (jiffies - start) >= (5 * HZ)) {
+ pr_warn("%s/%d: cpu %d: cpu_isolated task blocked for %ld jiffies\n",
+ task->comm, task->pid, smp_processor_id(),
+ (jiffies - start));
+ warned = true;
+ }
+ if (should_resched())
+ schedule();
+ if (test_thread_flag(TIF_SIGPENDING))
+ break;
+
+ /* Idle with interrupts enabled and wait for the tick. */
+ set_current_state(TASK_INTERRUPTIBLE);
+ arch_cpu_idle();
+ set_current_state(TASK_RUNNING);
+ }
+ if (warned) {
+ pr_warn("%s/%d: cpu %d: cpu_isolated task unblocked after %ld jiffies\n",
+ task->comm, task->pid, smp_processor_id(),
+ (jiffies - start));
+ dump_stack();
+ }
+}
+
#endif
/*
--
2.1.2
^ permalink raw reply related
* [PATCH v2 2/5] nohz: support PR_CPU_ISOLATED_STRICT mode
From: Chris Metcalf @ 2015-05-15 21:27 UTC (permalink / raw)
To: Gilad Ben Yossef, Steven Rostedt, Ingo Molnar, Peter Zijlstra,
Andrew Morton, Rik van Riel, Tejun Heo, Frederic Weisbecker,
Thomas Gleixner, Paul E. McKenney, Christoph Lameter,
Viresh Kumar, linux-doc, linux-api, linux-kernel
Cc: Chris Metcalf
In-Reply-To: <1431725251-20943-1-git-send-email-cmetcalf@ezchip.com>
With cpu_isolated mode, the task is in principle guaranteed not to be
interrupted by the kernel, but only if it behaves. In particular, if it
enters the kernel via system call, page fault, or any of a number of other
synchronous traps, it may be unexpectedly exposed to long latencies.
Add a simple flag that puts the process into a state where any such
kernel entry is fatal.
To allow the state to be entered and exited, we add an internal bit to
current->cpu_isolated_flags that is set when prctl() sets the flags.
We check the bit on syscall entry as well as on any exception_enter().
The prctl() syscall is ignored to allow clearing the bit again later,
and exit/exit_group are ignored to allow exiting the task without
a pointless signal killing you as you try to do so.
This change adds the syscall-detection hooks only for x86 and tile;
I am happy to try to add more for additional platforms in the final
version.
The signature of context_tracking_exit() changes to report whether
we, in fact, are exiting back to user space, so that we can track
user exceptions properly separately from other kernel entries.
Signed-off-by: Chris Metcalf <cmetcalf@ezchip.com>
---
arch/tile/kernel/ptrace.c | 6 +++++-
arch/x86/kernel/ptrace.c | 2 ++
include/linux/context_tracking.h | 11 ++++++++---
include/linux/tick.h | 16 ++++++++++++++++
include/uapi/linux/prctl.h | 1 +
kernel/context_tracking.c | 9 ++++++---
kernel/time/tick-sched.c | 38 ++++++++++++++++++++++++++++++++++++++
7 files changed, 76 insertions(+), 7 deletions(-)
diff --git a/arch/tile/kernel/ptrace.c b/arch/tile/kernel/ptrace.c
index f84eed8243da..d4e43a13bab1 100644
--- a/arch/tile/kernel/ptrace.c
+++ b/arch/tile/kernel/ptrace.c
@@ -259,8 +259,12 @@ int do_syscall_trace_enter(struct pt_regs *regs)
* If TIF_NOHZ is set, we are required to call user_exit() before
* doing anything that could touch RCU.
*/
- if (work & _TIF_NOHZ)
+ if (work & _TIF_NOHZ) {
user_exit();
+ if (tick_nohz_cpu_isolated_strict())
+ tick_nohz_cpu_isolated_syscall(
+ regs->regs[TREG_SYSCALL_NR]);
+ }
if (work & _TIF_SYSCALL_TRACE) {
if (tracehook_report_syscall_entry(regs))
diff --git a/arch/x86/kernel/ptrace.c b/arch/x86/kernel/ptrace.c
index a7bc79480719..7f784054ddea 100644
--- a/arch/x86/kernel/ptrace.c
+++ b/arch/x86/kernel/ptrace.c
@@ -1479,6 +1479,8 @@ unsigned long syscall_trace_enter_phase1(struct pt_regs *regs, u32 arch)
if (work & _TIF_NOHZ) {
user_exit();
work &= ~_TIF_NOHZ;
+ if (tick_nohz_cpu_isolated_strict())
+ tick_nohz_cpu_isolated_syscall(regs->orig_ax);
}
#ifdef CONFIG_SECCOMP
diff --git a/include/linux/context_tracking.h b/include/linux/context_tracking.h
index 2821838256b4..d042f4cda39d 100644
--- a/include/linux/context_tracking.h
+++ b/include/linux/context_tracking.h
@@ -3,6 +3,7 @@
#include <linux/sched.h>
#include <linux/vtime.h>
+#include <linux/tick.h>
#include <linux/context_tracking_state.h>
#include <asm/ptrace.h>
@@ -11,7 +12,7 @@
extern void context_tracking_cpu_set(int cpu);
extern void context_tracking_enter(enum ctx_state state);
-extern void context_tracking_exit(enum ctx_state state);
+extern bool context_tracking_exit(enum ctx_state state);
extern void context_tracking_user_enter(void);
extern void context_tracking_user_exit(void);
extern void __context_tracking_task_switch(struct task_struct *prev,
@@ -37,8 +38,12 @@ static inline enum ctx_state exception_enter(void)
return 0;
prev_ctx = this_cpu_read(context_tracking.state);
- if (prev_ctx != CONTEXT_KERNEL)
- context_tracking_exit(prev_ctx);
+ if (prev_ctx != CONTEXT_KERNEL) {
+ if (context_tracking_exit(prev_ctx)) {
+ if (tick_nohz_cpu_isolated_strict())
+ tick_nohz_cpu_isolated_exception();
+ }
+ }
return prev_ctx;
}
diff --git a/include/linux/tick.h b/include/linux/tick.h
index ec1953474a65..b7ffb10337ba 100644
--- a/include/linux/tick.h
+++ b/include/linux/tick.h
@@ -147,6 +147,8 @@ extern void tick_nohz_full_kick_cpu(int cpu);
extern void tick_nohz_full_kick_all(void);
extern void __tick_nohz_task_switch(struct task_struct *tsk);
extern void tick_nohz_cpu_isolated_enter(void);
+extern void tick_nohz_cpu_isolated_syscall(int nr);
+extern void tick_nohz_cpu_isolated_exception(void);
#else
static inline bool tick_nohz_full_enabled(void) { return false; }
static inline bool tick_nohz_full_cpu(int cpu) { return false; }
@@ -157,6 +159,8 @@ static inline void tick_nohz_full_kick_all(void) { }
static inline void __tick_nohz_task_switch(struct task_struct *tsk) { }
static inline bool tick_nohz_is_cpu_isolated(void) { return false; }
static inline void tick_nohz_cpu_isolated_enter(void) { }
+static inline void tick_nohz_cpu_isolated_syscall(int nr) { }
+static inline void tick_nohz_cpu_isolated_exception(void) { }
#endif
static inline bool is_housekeeping_cpu(int cpu)
@@ -189,4 +193,16 @@ static inline void tick_nohz_task_switch(struct task_struct *tsk)
__tick_nohz_task_switch(tsk);
}
+static inline bool tick_nohz_cpu_isolated_strict(void)
+{
+#ifdef CONFIG_NO_HZ_FULL
+ if (tick_nohz_full_cpu(smp_processor_id()) &&
+ (current->cpu_isolated_flags &
+ (PR_CPU_ISOLATED_ENABLE | PR_CPU_ISOLATED_STRICT)) ==
+ (PR_CPU_ISOLATED_ENABLE | PR_CPU_ISOLATED_STRICT))
+ return true;
+#endif
+ return false;
+}
+
#endif
diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h
index edb40b6b84db..0c11238a84fb 100644
--- a/include/uapi/linux/prctl.h
+++ b/include/uapi/linux/prctl.h
@@ -194,5 +194,6 @@ struct prctl_mm_map {
#define PR_SET_CPU_ISOLATED 47
#define PR_GET_CPU_ISOLATED 48
# define PR_CPU_ISOLATED_ENABLE (1 << 0)
+# define PR_CPU_ISOLATED_STRICT (1 << 1)
#endif /* _LINUX_PRCTL_H */
diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
index 66739d7c1350..c82509caa42e 100644
--- a/kernel/context_tracking.c
+++ b/kernel/context_tracking.c
@@ -131,15 +131,16 @@ NOKPROBE_SYMBOL(context_tracking_user_enter);
* This call supports re-entrancy. This way it can be called from any exception
* handler without needing to know if we came from userspace or not.
*/
-void context_tracking_exit(enum ctx_state state)
+bool context_tracking_exit(enum ctx_state state)
{
unsigned long flags;
+ bool from_user = false;
if (!context_tracking_is_enabled())
- return;
+ return false;
if (in_interrupt())
- return;
+ return false;
local_irq_save(flags);
if (__this_cpu_read(context_tracking.state) == state) {
@@ -150,6 +151,7 @@ void context_tracking_exit(enum ctx_state state)
*/
rcu_user_exit();
if (state == CONTEXT_USER) {
+ from_user = true;
vtime_user_exit(current);
trace_user_exit(0);
}
@@ -157,6 +159,7 @@ void context_tracking_exit(enum ctx_state state)
__this_cpu_write(context_tracking.state, CONTEXT_KERNEL);
}
local_irq_restore(flags);
+ return from_user;
}
NOKPROBE_SYMBOL(context_tracking_exit);
EXPORT_SYMBOL_GPL(context_tracking_exit);
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index f1551c946c45..273820cd484a 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -27,6 +27,7 @@
#include <linux/swap.h>
#include <asm/irq_regs.h>
+#include <asm/unistd.h>
#include "tick-internal.h"
@@ -440,6 +441,43 @@ void tick_nohz_cpu_isolated_enter(void)
}
}
+static void kill_cpu_isolated_strict_task(void)
+{
+ dump_stack();
+ current->cpu_isolated_flags &= ~PR_CPU_ISOLATED_ENABLE;
+ send_sig(SIGKILL, current, 1);
+}
+
+/*
+ * This routine is called from syscall entry (with the syscall number
+ * passed in) if the _STRICT flag is set.
+ */
+void tick_nohz_cpu_isolated_syscall(int syscall)
+{
+ /* Ignore prctl() syscalls or any task exit. */
+ switch (syscall) {
+ case __NR_prctl:
+ case __NR_exit:
+ case __NR_exit_group:
+ return;
+ }
+
+ pr_warn("%s/%d: cpu_isolated strict mode violated by syscall %d\n",
+ current->comm, current->pid, syscall);
+ kill_cpu_isolated_strict_task();
+}
+
+/*
+ * This routine is called from any userspace exception if the _STRICT
+ * flag is set.
+ */
+void tick_nohz_cpu_isolated_exception(void)
+{
+ pr_warn("%s/%d: cpu_isolated strict mode violated by exception\n",
+ current->comm, current->pid);
+ kill_cpu_isolated_strict_task();
+}
+
#endif
/*
--
2.1.2
^ permalink raw reply related
* [PATCH v2 3/5] nohz: cpu_isolated strict mode configurable signal
From: Chris Metcalf @ 2015-05-15 21:27 UTC (permalink / raw)
To: Gilad Ben Yossef, Steven Rostedt, Ingo Molnar, Peter Zijlstra,
Andrew Morton, Rik van Riel, Tejun Heo, Frederic Weisbecker,
Thomas Gleixner, Paul E. McKenney, Christoph Lameter,
Viresh Kumar, linux-doc-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
Cc: Chris Metcalf
In-Reply-To: <1431725251-20943-1-git-send-email-cmetcalf-d5a29ZRxExrQT0dZR+AlfA@public.gmane.org>
Allow userspace to override the default SIGKILL delivered
when a cpu_isolated process in STRICT mode does a syscall
or otherwise synchronously enters the kernel.
In addition to being able to set the signal, we now also
pass whether or not the interruption was from a syscall in
the si_code field of the siginfo.
Signed-off-by: Chris Metcalf <cmetcalf-d5a29ZRxExrQT0dZR+AlfA@public.gmane.org>
---
include/uapi/linux/prctl.h | 2 ++
kernel/time/tick-sched.c | 15 +++++++++++----
2 files changed, 13 insertions(+), 4 deletions(-)
diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h
index 0c11238a84fb..ab45bd3d5799 100644
--- a/include/uapi/linux/prctl.h
+++ b/include/uapi/linux/prctl.h
@@ -195,5 +195,7 @@ struct prctl_mm_map {
#define PR_GET_CPU_ISOLATED 48
# define PR_CPU_ISOLATED_ENABLE (1 << 0)
# define PR_CPU_ISOLATED_STRICT (1 << 1)
+# define PR_CPU_ISOLATED_SET_SIG(sig) (((sig) & 0x7f) << 8)
+# define PR_CPU_ISOLATED_GET_SIG(bits) (((bits) >> 8) & 0x7f)
#endif /* _LINUX_PRCTL_H */
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 273820cd484a..772be78f926c 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -441,11 +441,18 @@ void tick_nohz_cpu_isolated_enter(void)
}
}
-static void kill_cpu_isolated_strict_task(void)
+static void kill_cpu_isolated_strict_task(int is_syscall)
{
+ siginfo_t info = {};
+ int sig;
+
dump_stack();
current->cpu_isolated_flags &= ~PR_CPU_ISOLATED_ENABLE;
- send_sig(SIGKILL, current, 1);
+
+ sig = PR_CPU_ISOLATED_GET_SIG(current->cpu_isolated_flags) ?: SIGKILL;
+ info.si_signo = sig;
+ info.si_code = is_syscall;
+ send_sig_info(sig, &info, current);
}
/*
@@ -464,7 +471,7 @@ void tick_nohz_cpu_isolated_syscall(int syscall)
pr_warn("%s/%d: cpu_isolated strict mode violated by syscall %d\n",
current->comm, current->pid, syscall);
- kill_cpu_isolated_strict_task();
+ kill_cpu_isolated_strict_task(1);
}
/*
@@ -475,7 +482,7 @@ void tick_nohz_cpu_isolated_exception(void)
{
pr_warn("%s/%d: cpu_isolated strict mode violated by exception\n",
current->comm, current->pid);
- kill_cpu_isolated_strict_task();
+ kill_cpu_isolated_strict_task(0);
}
#endif
--
2.1.2
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox