* Re: [PATCH RFC] vfs: add a O_NOMTIME flag
From: Pavel Machek @ 2015-07-22 13:47 UTC (permalink / raw)
To: NeilBrown
Cc: NeilBrown, J. Bruce Fields, John Stoffel, Austin S Hemmelgarn,
Kevin Easton, Theodore Ts'o, Sage Weil, Trond Myklebust,
Dave Chinner, Zach Brown, Alexander Viro,
Linux FS-devel Mailing List, Linux Kernel Mailing List,
Linux API Mailing List
In-Reply-To: <20150715145456.0376796b@noble>
On Wed 2015-07-15 14:54:56, NeilBrown wrote:
> On Tue, 14 Jul 2015 15:13:00 +0200 Pavel Machek <pavel-+ZI9xUNit7I@public.gmane.org> wrote:
>
> > Hi!
> >
> > > BTW When you "swap" to a file the mtime doesn't get updated. No one seems to
> > > complain about that. I guess it is a rather narrow use-case though.
> >
> > Actually yes, I'd like to complain.
> >
> > It was not swap, it was mount -o loop, but I guess that's the same
> > case. Then rsync refused to work on that file... and being on slow ARM
> > system it took me a while to figure out WTF is going on.
> >
> > So yes, we have problems with mtime, and yes, they matter.
> > Pavel
>
> Odd...
> I assume you mean
> mount -o loop /some/file /mountpoint
>
> and then when you write to the filesystem on /mountpoint the mtime
> of /some/file doesn't get updated?
> I think it should.
> drivers/block/loop.c uses vfs_iter_write() to write to a file.
> That calls f_op->write_iter which will typically call
> generic_file_write_iter() which will call file_update_time() to update
> the time stamps.
Yes, that. I'm pretty sure I seen it, but it was probably on 2.6.X
kernel... Does it make sense to try to reproduce it on the old kernel?
> What filesystem was /some/file on?
Very probably VFAT.
> I just did some testing on ext4 and it seems to do the right thing
> mtime gets updated.
Yes, I tried here, and it seems to be ok.
Thanks,
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
^ permalink raw reply
* Re: [PATCH v4 1/5] nohz_full: add support for "cpu_isolated" mode
From: Christoph Lameter @ 2015-07-22 13:57 UTC (permalink / raw)
To: Paul E. McKenney
Cc: Andy Lutomirski, Chris Metcalf, Gilad Ben Yossef, Steven Rostedt,
Ingo Molnar, Peter Zijlstra, Andrew Morton, Rik van Riel,
Tejun Heo, Frederic Weisbecker, Thomas Gleixner, Viresh Kumar,
linux-doc@vger.kernel.org, Linux API,
linux-kernel@vger.kernel.org
In-Reply-To: <20150721203622.GX3717@linux.vnet.ibm.com>
On Tue, 21 Jul 2015, Paul E. McKenney wrote:
> > > In the current version, there's also the vmstat_update() that
> > > may schedule delayed work and interrupt the core again
> > > shortly before realizing that there are no more counter updates
> > > happening, at which point it quiesces. Currently we handle
> > > this in cpu_isolated mode simply by spinning and waiting for
> > > the timer interrupts to complete.
> >
> > Perhaps we should fix that?
>
> Didn't Christoph Lameter fix this? Or is this an additional problem?
Well the vmstat update must realize first that there are no outstanding
updates before switching itself off. So typically there is one extra tick.
But we could add another function that will simply fold the differential
immediately and turn the kworker task in the expectation that the
processor will stay quiet.
^ permalink raw reply
* Re: [PATCH V4 2/6] mm: mlock: Add new mlock, munlock, and munlockall system calls
From: Eric B Munson @ 2015-07-22 14:05 UTC (permalink / raw)
To: Vlastimil Babka
Cc: Andrew Morton, Michal Hocko, Heiko Carstens, Geert Uytterhoeven,
Catalin Marinas, Stephen Rothwell, Guenter Roeck, linux-alpha,
linux-kernel, linux-arm-kernel, adi-buildroot-devel,
linux-cris-kernel, linux-ia64, linux-m68k, linux-mips,
linux-am33-list, linux-parisc, linuxppc-dev, linux-s390, linux-sh,
sparclinux, linux-xtensa, linux-api, linux-arch, linux-mm
In-Reply-To: <55AF5F5A.3000707@suse.cz>
[-- Attachment #1: Type: text/plain, Size: 540 bytes --]
On Wed, 22 Jul 2015, Vlastimil Babka wrote:
> On 07/21/2015 09:59 PM, Eric B Munson wrote:
> >With the refactored mlock code, introduce new system calls for mlock,
> >munlock, and munlockall. The new calls will allow the user to specify
> >what lock states are being added or cleared. mlock2 and munlock2 are
> >trivial at the moment, but a follow on patch will add a new mlock state
> >making them useful.
> >
> >munlock2 addresses a limitation of the current implementation. If a
>
> ^ munlockall2?
Fixed, thanks.
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH V3 4/5] mm: mmap: Add mmap flag to request VM_LOCKONFAULT
From: Paul Gortmaker @ 2015-07-22 14:10 UTC (permalink / raw)
To: Eric B Munson
Cc: Chris Metcalf, Andrew Morton, Michal Hocko, Vlastimil Babka, LKML,
linux-mm, linux-arch, linux-api, linux-next@vger.kernel.org
In-Reply-To: <20150721153722.GB5411@akamai.com>
On 2015-07-21 11:37 AM, Eric B Munson wrote:
> On Mon, 20 Jul 2015, Chris Metcalf wrote:
>
>> On 07/18/2015 03:11 PM, Paul Gortmaker wrote:
>>> On Tue, Jul 7, 2015 at 1:03 PM, Eric B Munson<emunson@akamai.com> wrote:
>>>>> The cost of faulting in all memory to be locked can be very high when
>>>>> working with large mappings. If only portions of the mapping will be
>>>>> used this can incur a high penalty for locking.
>>>>>
>>>>> Now that we have the new VMA flag for the locked but not present state,
>>>>> expose it as an mmap option like MAP_LOCKED -> VM_LOCKED.
>>> An automatic bisection on arch/tile leads to this commit:
>>>
>>> 5a5656f2c9b61c74c15f9ef3fa2e6513b6c237bb is the first bad commit
>>> commit 5a5656f2c9b61c74c15f9ef3fa2e6513b6c237bb
>>> Author: Eric B Munson<emunson@akamai.com>
>>> Date: Thu Jul 16 10:09:22 2015 +1000
>>>
>>> mm: mmap: add mmap flag to request VM_LOCKONFAULT
>>
>> Eric, I'm happy to help with figuring out the tile issues.
>
> Thanks for the offer, I think I have is sorted in V4 (which I am
> checking one last time before I post).
Not quite sorted yet. Seems parisc fails on v4. It updated the
number of syscalls but did not update syscall_table.S causing:
arch/parisc/kernel/syscall_table.S:444: Error: size of syscall table does not fit value of __NR_Linux_syscalls
http://kisskb.ellerman.id.au/kisskb/buildresult/12468884/
Paul.
--
> Eric
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH V4 2/6] mm: mlock: Add new mlock, munlock, and munlockall system calls
From: Eric B Munson @ 2015-07-22 14:15 UTC (permalink / raw)
To: Michael Ellerman
Cc: Andrew Morton, linux-mips, linux-m68k, linux-ia64, linux-sh,
Catalin Marinas, Heiko Carstens, Michal Hocko, linux-mm,
sparclinux, linux-arch, Stephen Rothwell, linux-am33-list,
Geert Uytterhoeven, Vlastimil Babka, Guenter Roeck, linux-xtensa,
linux-s390, adi-buildroot-devel, linux-arm-kernel,
linux-cris-kernel, linux-parisc, linux-api, linux-kernel,
linux-alpha, linuxppc-dev
In-Reply-To: <1437528316.16792.7.camel@ellerman.id.au>
[-- Attachment #1: Type: text/plain, Size: 1353 bytes --]
On Wed, 22 Jul 2015, Michael Ellerman wrote:
> On Tue, 2015-07-21 at 13:44 -0700, Andrew Morton wrote:
> > On Tue, 21 Jul 2015 15:59:37 -0400 Eric B Munson <emunson@akamai.com> wrote:
> >
> > > With the refactored mlock code, introduce new system calls for mlock,
> > > munlock, and munlockall. The new calls will allow the user to specify
> > > what lock states are being added or cleared. mlock2 and munlock2 are
> > > trivial at the moment, but a follow on patch will add a new mlock state
> > > making them useful.
> > >
> > > munlock2 addresses a limitation of the current implementation. If a
> > > user calls mlockall(MCL_CURRENT | MCL_FUTURE) and then later decides
> > > that MCL_FUTURE should be removed, they would have to call munlockall()
> > > followed by mlockall(MCL_CURRENT) which could potentially be very
> > > expensive. The new munlockall2 system call allows a user to simply
> > > clear the MCL_FUTURE flag.
> >
> > This is hard. Maybe we shouldn't have wired up anything other than
> > x86. That's what we usually do with new syscalls.
>
> Yeah I think so.
>
> You haven't wired it up properly on powerpc, but I haven't mentioned it because
> I'd rather we did it.
>
> cheers
It looks like I will be spinning a V5, so I will drop all but the x86
system calls additions in that version.
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH V4 5/6] mm: mmap: Add mmap flag to request VM_LOCKONFAULT
From: Eric B Munson @ 2015-07-22 14:32 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Andrew Morton, Michal Hocko, Vlastimil Babka, Paul Gortmaker,
Chris Metcalf, Guenter Roeck, linux-alpha, linux-kernel,
linux-mips, linux-parisc, linuxppc-dev, sparclinux, linux-xtensa,
linux-mm, linux-arch, linux-api
In-Reply-To: <20150722112558.GC8630@node.dhcp.inet.fi>
[-- Attachment #1: Type: text/plain, Size: 760 bytes --]
On Wed, 22 Jul 2015, Kirill A. Shutemov wrote:
> On Tue, Jul 21, 2015 at 03:59:40PM -0400, Eric B Munson wrote:
> > The cost of faulting in all memory to be locked can be very high when
> > working with large mappings. If only portions of the mapping will be
> > used this can incur a high penalty for locking.
> >
> > Now that we have the new VMA flag for the locked but not present state,
> > expose it as an mmap option like MAP_LOCKED -> VM_LOCKED.
>
> What is advantage over mmap() + mlock(MLOCK_ONFAULT)?
There isn't one, it was added to maintain parity with the
mlock(MLOCK_LOCK) -> mmap(MAP_LOCKED) set. I think not having will lead
to confusion because we have MAP_LOCKED so why don't we support
LOCKONFAULT from mmap as well.
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH -mm v9 6/8] proc: add kpageidle file
From: Vladimir Davydov @ 2015-07-22 15:20 UTC (permalink / raw)
To: Andrew Morton
Cc: Andres Lagar-Cavilla, Minchan Kim, Raghavendra K T,
Johannes Weiner, Michal Hocko, Greg Thelen, Michel Lespinasse,
David Rientjes, Pavel Emelyanov, Cyrill Gorcunov, Jonathan Corbet,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-doc-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
cgroups-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20150721163452.c1e4075a2b193bcd325fad56-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>
On Tue, Jul 21, 2015 at 04:34:52PM -0700, Andrew Morton wrote:
> On Sun, 19 Jul 2015 15:31:15 +0300 Vladimir Davydov <vdavydov-bzQdu9zFT3WakBO8gow8eQ@public.gmane.org> wrote:
>
> > Knowing the portion of memory that is not used by a certain application
> > or memory cgroup (idle memory) can be useful for partitioning the system
> > efficiently, e.g. by setting memory cgroup limits appropriately.
> > Currently, the only means to estimate the amount of idle memory provided
> > by the kernel is /proc/PID/{clear_refs,smaps}: the user can clear the
> > access bit for all pages mapped to a particular process by writing 1 to
> > clear_refs, wait for some time, and then count smaps:Referenced.
> > However, this method has two serious shortcomings:
> >
> > - it does not count unmapped file pages
> > - it affects the reclaimer logic
> >
> > To overcome these drawbacks, this patch introduces two new page flags,
> > Idle and Young, and a new proc file, /proc/kpageidle. A page's Idle flag
> > can only be set from userspace by setting bit in /proc/kpageidle at the
> > offset corresponding to the page, and it is cleared whenever the page is
> > accessed either through page tables (it is cleared in page_referenced()
> > in this case) or using the read(2) system call (mark_page_accessed()).
> > Thus by setting the Idle flag for pages of a particular workload, which
> > can be found e.g. by reading /proc/PID/pagemap, waiting for some time to
> > let the workload access its working set, and then reading the kpageidle
> > file, one can estimate the amount of pages that are not used by the
> > workload.
> >
> > The Young page flag is used to avoid interference with the memory
> > reclaimer. A page's Young flag is set whenever the Access bit of a page
> > table entry pointing to the page is cleared by writing to kpageidle. If
> > page_referenced() is called on a Young page, it will add 1 to its return
> > value, therefore concealing the fact that the Access bit was cleared.
> >
> > Note, since there is no room for extra page flags on 32 bit, this
> > feature uses extended page flags when compiled on 32 bit.
> >
> > ...
> >
> >
> > ...
> >
> > +static void kpageidle_clear_pte_refs(struct page *page)
> > +{
> > + struct rmap_walk_control rwc = {
> > + .rmap_one = kpageidle_clear_pte_refs_one,
> > + .anon_lock = page_lock_anon_vma_read,
> > + };
>
> I think this can be static const, since `arg' is unused? That would
> save some cycles and stack.
Good catch, thanks.
>
> > + bool need_lock;
> > +
> > + if (!page_mapped(page) ||
> > + !page_rmapping(page))
> > + return;
> > +
> > + need_lock = !PageAnon(page) || PageKsm(page);
> > + if (need_lock && !trylock_page(page))
>
> Oh. So the feature is a bit unreliable.
>
> I'm not immediately seeing anything which would prevent us from using
> plain old lock_page() here. What's going on?
A page may be locked for quite a long period of time, e.g.
truncate_inode_pages_range() may wait until a page writeback finishes
under the page lock. Instead of stalling kpageidle scan, we'd better
move on to the next page. Of course, the result won't be 100% accurate.
In fact, it isn't accurate anyway, because we skip isolated pages,
neither can it possibly be 100% accurate, because the scan itself is not
instant so that while we are performing it the system usage pattern
might change. This new API is only supposed to give a good estimate of
memory usage pattern, which could be used as a hint for adjusting the
system configuration to improve performance.
>
> > + return;
> > +
> > + rmap_walk(page, &rwc);
> > +
> > + if (need_lock)
> > + unlock_page(page);
> > +}
> > +
> > +static ssize_t kpageidle_read(struct file *file, char __user *buf,
> > + size_t count, loff_t *ppos)
> > +{
> > + u64 __user *out = (u64 __user *)buf;
> > + struct page *page;
> > + unsigned long pfn, end_pfn;
> > + ssize_t ret = 0;
> > + u64 idle_bitmap = 0;
> > + int bit;
> > +
> > + if (*ppos & KPMMASK || count & KPMMASK)
> > + return -EINVAL;
>
> Interface requires 8-byte aligned offset and size.
>
> > + pfn = *ppos * BITS_PER_BYTE;
> > + if (pfn >= max_pfn)
> > + return 0;
> > +
> > + end_pfn = pfn + count * BITS_PER_BYTE;
> > + if (end_pfn > max_pfn)
> > + end_pfn = ALIGN(max_pfn, KPMBITS);
>
> So we lose up to 63 pages. Presumably max_pfn is well enough aligned
> for this to not matter, dunno.
ALIGN(x, a) resolves to ((x + a - 1) & ~(a - 1)), which is >= x, so we
shouldn't loose anything.
>
> > + for (; pfn < end_pfn; pfn++) {
> > + bit = pfn % KPMBITS;
> > + page = kpageidle_get_page(pfn);
> > + if (page) {
> > + if (page_is_idle(page)) {
> > + /*
> > + * The page might have been referenced via a
> > + * pte, in which case it is not idle. Clear
> > + * refs and recheck.
> > + */
> > + kpageidle_clear_pte_refs(page);
> > + if (page_is_idle(page))
> > + idle_bitmap |= 1ULL << bit;
>
> I don't understand what's going on here. More details, please?
The output is a bitmap, which is stored as an array of 8-byte elements,
where byte order within each word is native, i.e. if page at pfn #i is
idle we need to set bit #i%64 of element #i/64 of the array. I'll
reflect this in the documentation.
>
> > + }
> > + put_page(page);
> > + }
> > + if (bit == KPMBITS - 1) {
> > + if (put_user(idle_bitmap, out)) {
> > + ret = -EFAULT;
> > + break;
> > + }
> > + idle_bitmap = 0;
> > + out++;
> > + }
> > + }
> > +
> > + *ppos += (char __user *)out - buf;
> > + if (!ret)
> > + ret = (char __user *)out - buf;
> > + return ret;
> > +}
> > +
> > +static ssize_t kpageidle_write(struct file *file, const char __user *buf,
> > + size_t count, loff_t *ppos)
> > +{
> > + const u64 __user *in = (const u64 __user *)buf;
> > + struct page *page;
> > + unsigned long pfn, end_pfn;
> > + ssize_t ret = 0;
> > + u64 idle_bitmap = 0;
> > + int bit;
> > +
> > + if (*ppos & KPMMASK || count & KPMMASK)
> > + return -EINVAL;
> > +
> > + pfn = *ppos * BITS_PER_BYTE;
> > + if (pfn >= max_pfn)
> > + return -ENXIO;
> > +
> > + end_pfn = pfn + count * BITS_PER_BYTE;
> > + if (end_pfn > max_pfn)
> > + end_pfn = ALIGN(max_pfn, KPMBITS);
> > +
> > + for (; pfn < end_pfn; pfn++) {
> > + bit = pfn % KPMBITS;
> > + if (bit == 0) {
> > + if (get_user(idle_bitmap, in)) {
> > + ret = -EFAULT;
> > + break;
> > + }
> > + in++;
> > + }
> > + if (idle_bitmap >> bit & 1) {
>
> Hate it when I have to go look up a C precedence table. This is
>
> if ((idle_bitmap >> bit) & 1) {
Fixed.
Here goes the incremental patch with all the fixes:
---
diff --git a/fs/proc/page.c b/fs/proc/page.c
index 7ff7cba8617b..9daa6e92450f 100644
--- a/fs/proc/page.c
+++ b/fs/proc/page.c
@@ -362,7 +362,11 @@ static int kpageidle_clear_pte_refs_one(struct page *page,
static void kpageidle_clear_pte_refs(struct page *page)
{
- struct rmap_walk_control rwc = {
+ /*
+ * Since rwc.arg is unused, rwc is effectively immutable, so we
+ * can make it static const to save some cycles and stack.
+ */
+ static const struct rmap_walk_control rwc = {
.rmap_one = kpageidle_clear_pte_refs_one,
.anon_lock = page_lock_anon_vma_read,
};
@@ -376,7 +380,7 @@ static void kpageidle_clear_pte_refs(struct page *page)
if (need_lock && !trylock_page(page))
return;
- rmap_walk(page, &rwc);
+ rmap_walk(page, (struct rmap_walk_control *)&rwc);
if (need_lock)
unlock_page(page);
@@ -466,7 +470,7 @@ static ssize_t kpageidle_write(struct file *file, const char __user *buf,
}
in++;
}
- if (idle_bitmap >> bit & 1) {
+ if ((idle_bitmap >> bit) & 1) {
page = kpageidle_get_page(pfn);
if (page) {
kpageidle_clear_pte_refs(page);
^ permalink raw reply related
* Re: [PATCH V4 5/6] mm: mmap: Add mmap flag to request VM_LOCKONFAULT
From: Kirill A. Shutemov @ 2015-07-22 15:45 UTC (permalink / raw)
To: Eric B Munson
Cc: Andrew Morton, Michal Hocko, Vlastimil Babka, Paul Gortmaker,
Chris Metcalf, Guenter Roeck, linux-alpha, linux-kernel,
linux-mips, linux-parisc, linuxppc-dev, sparclinux, linux-xtensa,
linux-mm, linux-arch, linux-api
In-Reply-To: <20150722143220.GB3203@akamai.com>
On Wed, Jul 22, 2015 at 10:32:20AM -0400, Eric B Munson wrote:
> On Wed, 22 Jul 2015, Kirill A. Shutemov wrote:
>
> > On Tue, Jul 21, 2015 at 03:59:40PM -0400, Eric B Munson wrote:
> > > The cost of faulting in all memory to be locked can be very high when
> > > working with large mappings. If only portions of the mapping will be
> > > used this can incur a high penalty for locking.
> > >
> > > Now that we have the new VMA flag for the locked but not present state,
> > > expose it as an mmap option like MAP_LOCKED -> VM_LOCKED.
> >
> > What is advantage over mmap() + mlock(MLOCK_ONFAULT)?
>
> There isn't one, it was added to maintain parity with the
> mlock(MLOCK_LOCK) -> mmap(MAP_LOCKED) set. I think not having will lead
> to confusion because we have MAP_LOCKED so why don't we support
> LOCKONFAULT from mmap as well.
I don't think it's ia good idea to spend bits in flags unless we have a
reason for that.
BTW, you have typo on sparc: s/0x8000/0x80000/.
--
Kirill A. Shutemov
^ permalink raw reply
* Re: [RFC v3 1/4] fs: Add generic file system event notifications
From: Bartlomiej Zolnierkiewicz @ 2015-07-22 15:55 UTC (permalink / raw)
To: Beata Michalska
Cc: linux-kernel, linux-fsdevel, linux-api, greg, jack, tytso,
adilger.kernel, hughd, lczerner, hch, linux-ext4, linux-mm,
kyungmin.park, kmpark
In-Reply-To: <1434460173-18427-2-git-send-email-b.michalska@samsung.com>
Hi,
Some comments below.
On Tuesday, June 16, 2015 03:09:30 PM Beata Michalska wrote:
> Introduce configurable generic interface for file
> system-wide event notifications, to provide file
> systems with a common way of reporting any potential
> issues as they emerge.
>
> The notifications are to be issued through generic
> netlink interface by newly introduced multicast group.
>
> Threshold notifications have been included, allowing
> triggering an event whenever the amount of free space drops
> below a certain level - or levels to be more precise as two
> of them are being supported: the lower and the upper range.
> The notifications work both ways: once the threshold level
> has been reached, an event shall be generated whenever
> the number of available blocks goes up again re-activating
> the threshold.
>
> The interface has been exposed through a vfs. Once mounted,
> it serves as an entry point for the set-up where one can
> register for particular file system events.
>
> Signed-off-by: Beata Michalska <b.michalska@samsung.com>
> ---
> Documentation/filesystems/events.txt | 232 ++++++++++
> fs/Kconfig | 2 +
> fs/Makefile | 1 +
> fs/events/Kconfig | 7 +
> fs/events/Makefile | 5 +
> fs/events/fs_event.c | 809 ++++++++++++++++++++++++++++++++++
> fs/events/fs_event.h | 22 +
> fs/events/fs_event_netlink.c | 104 +++++
> fs/namespace.c | 1 +
> include/linux/fs.h | 6 +-
> include/linux/fs_event.h | 72 +++
> include/uapi/linux/Kbuild | 1 +
> include/uapi/linux/fs_event.h | 58 +++
> 13 files changed, 1319 insertions(+), 1 deletion(-)
> create mode 100644 Documentation/filesystems/events.txt
> create mode 100644 fs/events/Kconfig
> create mode 100644 fs/events/Makefile
> create mode 100644 fs/events/fs_event.c
> create mode 100644 fs/events/fs_event.h
> create mode 100644 fs/events/fs_event_netlink.c
> create mode 100644 include/linux/fs_event.h
> create mode 100644 include/uapi/linux/fs_event.h
>
> diff --git a/Documentation/filesystems/events.txt b/Documentation/filesystems/events.txt
> new file mode 100644
> index 0000000..c2e6227
> --- /dev/null
> +++ b/Documentation/filesystems/events.txt
> @@ -0,0 +1,232 @@
> +
> + Generic file system event notification interface
> +
> +Document created 23 April 2015 by Beata Michalska <b.michalska@samsung.com>
> +
> +1. The reason behind:
> +=====================
> +
> +There are many corner cases when things might get messy with the filesystems.
> +And it is not always obvious what and when went wrong. Sometimes you might
> +get some subtle hints that there is something going on - but by the time
> +you realise it, it might be too late as you are already out-of-space
> +or the filesystem has been remounted as read-only (i.e.). The generic
> +interface for the filesystem events fills the gap by providing a rather
> +easy way of real-time notifications triggered whenever something interesting
> +happens, allowing filesystems to report events in a common way, as they occur.
> +
> +2. How does it work:
> +====================
> +
> +The interface itself has been exposed as fstrace-type Virtual File System,
> +primarily to ease the process of setting up the configuration for the
> +notifications. So for starters, it needs to get mounted (obviously):
> +
> + mount -t fstrace none /sys/fs/events
> +
> +This will unveil the single fstrace filesystem entry - the 'config' file,
> +through which the notification are being set-up.
The patch creates a separate virtual filesystem for single file,
this is an overkill IMHO and a new sysfs or debugfs entry should
be sufficient.
> +
> +Activating notifications for particular filesystem is as straightforward
> +as writing into the 'config' file. Note that by default all events, despite
> +the actual filesystem type, are being disregarded.
> +
> +Synopsis of config:
> +------------------
> +
> + MOUNT EVENT_TYPE [L1] [L2]
OTOH Why not use the advantages of having a separate virtual
filesystem and create separate directories for each mount point
(+ maybe even extra parent directories for mount namespaces) and
put separate entries for each event type in these directories.
This would also allow usage of eventfd() notification interface
on such files.
Please take look at:
tools/cgroup/cgroup_event_listener.c
and
Documentation/cgroups/memcg_test.txt (point 9.10)
to see how much easier it is to observe memory usage thresholds
on memory cgroups compared to available blocks on filesystems
using fs events..
Also while at it please add your example user-space code (posted
on request in a some other mail) to tools/fs_events/ (preferably
in a separate patch).
> +
> + MOUNT : the filesystem's mount point
> + EVENT_TYPE : event types - currently two of them are being supported:
> +
> + * generic events ("G") covering most common warnings
> + and errors that might be reported by any filesystem;
> + this option does not take any arguments;
fs_event.h in uapi dir allows following events:
/*
* Supported set of FS events
*/
enum {
FS_EVENT_NONE,
FS_WARN_ENOSPC, /* No space left to reserve data blks */
FS_WARN_ENOSPC_META, /* No space left for metadata */
FS_THR_LRBELOW, /* The threshold lower range has been reached */
FS_THR_LRABOVE, /* The threshold lower range re-activcated*/
FS_THR_URBELOW,
FS_THR_URABOVE,
FS_ERR_REMOUNT_RO, /* The file system has been remounted as RO */
FS_ERR_CORRUPTED /* Critical error - fs corrupted */
};
For non-threshold related events the current interface allows
only configuration of all or none events to be anabled, i.e.
you cannot selectively enable notification on FS_WARN_ENOSPC
but not on FS_ERR_REMOUNT_RO.
I also think that configuration interface should be made to
match the notification interface when it comes to event types.
> +
> + * threshold notifications ("T") - events sent whenever
> + the amount of available space drops below certain level;
> + it is possible to specify two threshold levels though
> + only one is required to properly setup the notifications;
> + as those refer to the number of available blocks, the lower
> + level [L1] needs to be higher than the upper one [L2]
Why is there a limitation of only two thresholds?
It should be relatively easy to make the code support
unlimited number of thresholds.
> +
> +Sample request could look like the following:
> +
> + echo /sample/mount/point G T 710000 500000 > /sys/fs/events/config
> +
> +Multiple request might be specified provided they are separated with semicolon.
s/request/requests/
I think that allowing multiple event types and requests in one
configuration request is not a good idea. Currently parsing
code is relatively simple but once somebody decides to enhance
the interface with new event types the parsing code may get
complex & ugly.
> +
> +The configuration itself might be modified at any time. One can add/remove
> +particular event types for given fielsystem, modify the threshold levels,
s/fielsystem/filesystem/
> +and remove single or all entries from the 'config' file.
> +
> + - Adding new event type:
> +
> + $ echo MOUNT EVENT_TYPE > /sys/fs/events/config
> +
> +(Note that is is enough to provide the event type to be enabled without
s/is is/is/
> +the already set ones.)
> +
> + - Removing event type:
> +
> + $ echo '!MOUNT EVENT_TYPE' > /sys/fs/events/config
> +
> + - Updating threshold limits:
> +
> + $ echo MOUNT T L1 L2 > /sys/fs/events/config
> +
> + - Removing single entry:
> +
> + $ echo '!MOUNT' > /sys/fs/events/config
> +
> + - Removing all entries:
> +
> + $ echo > /sys/fs/events/config
> +
> +Reading the file will list all registered entries with their current set-up
> +along with some additional info like the filesystem type and the backing device
> +name if available.
> +
> +Final, though a very important note on the configuration: when and if the
> +actual events are being triggered falls way beyond the scope of the generic
> +filesystem events interface. It is up to a particular filesystem
> +implementation which events are to be supported - if any at all. So if
> +given filesystem does not support the event notifications, an attempt to
> +enable those through 'config' file will fail.
> +
> +
> +3. The generic netlink interface support:
> +=========================================
> +
> +Whenever an event notification is triggered (by given filesystem) the current
> +configuration is being validated to decide whether a userpsace notification
s/userpsace/userspace/
> +should be launched. If there has been no request (in a mean of 'config' file
> +entry) for given event, one will be silently disregarded. If, on the other
> +hand, someone is 'watching' given filesystem for specific events, a generic
> +netlink message will be sent. A dedicated multicast group has been provided
> +solely for this purpose so in order to receive such notifications, one should
> +subscribe to this new multicast group. As for now only the init network
> +namespace is being supported.
> +
> +3.1 Message format
> +
> +The FS_NL_C_EVENT shall be stored within the generic netlink message header
> +as the command field. The message payload will provide more detailed info:
> +the backing device major and minor numbers, the event code and the id of
> +the process which action led to the event occurrence. In case of threshold
> +notifications, the current number of available blocks will be included
> +in the payload as well.
> +
> +
> + 0 1 2 3
> + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> + | NETLINK MESSAGE HEADER |
> + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> + | GENERIC NETLINK MESSAGE HEADER |
> + | (with FS_NL_C_EVENT as genlmsghdr cdm field) |
> + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> + | Optional user specific message header |
> + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> + | GENERIC MESSAGE PAYLOAD: |
> + +---------------------------------------------------------------+
> + | FS_NL_A_EVENT_ID (NLA_U32) |
> + +---------------------------------------------------------------+
> + | FS_NL_A_DEV_MAJOR (NLA_U32) |
> + +---------------------------------------------------------------+
> + | FS_NL_A_DEV_MINOR (NLA_U32) |
> + +---------------------------------------------------------------+
> + | FS_NL_A_CAUSED_ID (NLA_U32) |
> + +---------------------------------------------------------------+
> + | FS_NL_A_DATA (NLA_U64) |
> + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> +
> +
> +The above figure is based on:
> + http://www.linuxfoundation.org/collaborate/workgroups/networking/generic_netlink_howto#Message_Format
> +
> +
> +4. API Reference:
> +=================
> +
> + 4.1 Generic file system event interface data & operations
> +
> + #include <linux/fs_event.h>
> +
> + struct fs_trace_info {
> + void __rcu *e_priv /* READ ONLY */
It should be marked as private for fs events core code and
not for use by filesystems' code. If possible it would be
the best to move it out of this struct,
> + unsigned int events_cap_mask; /* Supported notifications */
> + const struct fs_trace_operations *ops;
> + };
> +
> + struct fs_trace_operations {
> + void (*query)(struct super_block *, u64 *);
> + };
> +
> + In order to get the fireworks and stuff, each filesystem needs to setup
> + the events_cap_mask field of the fs_trace_info structure, which has been
> + embedded within the super_block structure. This should reflect the type of
> + events the filesystem wants to support. In case of threshold notifications,
> + apart from setting the FS_EVENT_THRESH flag, the 'query' callback should
> + be provided as this enables the events interface to get the up-to-date
> + state of the number of available blocks whenever those notifications are
> + being requested.
> +
> + The 'e_priv' field of the fs_trace_info structure should be completely ignored
> + as it's for INTERNAL USE ONLY. So don't even think of messing with it, if you
> + do not want to get yourself into some real trouble. If still, you are tempted
> + to do so - feel free, it's gonna be pure fun. Consider yourself warned.
> +
> +
> + 4.2 Event notification:
> +
> + #include <linux/fs_event.h>
> + void fs_event_notify(struct super_block *sb, unsigned int event_id);
> +
> + Notify the generic FS event interface of an occurring event.
> + This shall be used by any file system that wishes to inform any potential
> + listeners/watchers of a particular event.
> + - sb: the filesystem's super block
> + - event_id: an event identifier
> +
> + 4.3 Threshold notifications:
> +
> + #include <linux/fs_event.h>
> + void fs_event_alloc_space(struct super_block *sb, u64 ncount);
> + void fs_event_free_space(struct super_block *sb, u64 ncount);
> +
> + Each filesystme supporting the threshold notifications should call
> + fs_event_alloc_space/fs_event_free_space respectively whenever the
> + amount of available blocks changes.
> + - sb: the filesystem's super block
> + - ncount: number of blocks being acquired/released
> +
> + Note that to properly handle the threshold notifications the fs events
> + interface needs to be kept up to date by the filesystems. Each should
> + register fs_trace_operations to enable querying the current number of
> + available blocks.
> +
> + 4.4 Sending message through generic netlink interface
> +
> + #include <linux/fs_event.h>
> +
> + int fs_netlink_send_event(size_t size, unsigned int event_id,
> + int (*compose_msg)(struct sk_buff *skb, void *data), void *cbdata);
> +
> + Although the fs event interface is fully responsible for sending the messages
> + over the netlink, filesystems might use the FS_EVENT multicast group to send
> + their own custom messages.
> + - size: the size of the message payload
> + - event_id: the event identifier
> + - compose_msg: a callback responsible for filling-in the message payload
> + - cbdata: message custom data
> +
> + Calling fs_netlink_send_event will result in a message being sent by
> + the FS_EVENT multicast group. Note that the body of the message should be
> + prepared (set-up )by the caller - through compose_msg callback. The message's
(set-up)
> + sk_buff will be allocated on behalf of the caller (thus the size parameter).
> + The compose_msg should only fill the payload with proper data. Unless
> + the event id is specified as FS_EVENT_NONE, it's value shall be added
> + to the payload prior to calling the compose_msg.
> +
> +
> diff --git a/fs/Kconfig b/fs/Kconfig
> index ec35851..a89e678 100644
> --- a/fs/Kconfig
> +++ b/fs/Kconfig
> @@ -69,6 +69,8 @@ config FILE_LOCKING
> for filesystems like NFS and for the flock() system
> call. Disabling this option saves about 11k.
>
> +source "fs/events/Kconfig"
> +
> source "fs/notify/Kconfig"
>
> source "fs/quota/Kconfig"
> diff --git a/fs/Makefile b/fs/Makefile
> index a88ac48..bcb3048 100644
> --- a/fs/Makefile
> +++ b/fs/Makefile
> @@ -126,3 +126,4 @@ obj-y += exofs/ # Multiple modules
> obj-$(CONFIG_CEPH_FS) += ceph/
> obj-$(CONFIG_PSTORE) += pstore/
> obj-$(CONFIG_EFIVAR_FS) += efivarfs/
> +obj-$(CONFIG_FS_EVENTS) += events/
> diff --git a/fs/events/Kconfig b/fs/events/Kconfig
> new file mode 100644
> index 0000000..1c60195
> --- /dev/null
> +++ b/fs/events/Kconfig
> @@ -0,0 +1,7 @@
> +# Generic Files System events interface
> +config FS_EVENTS
> + bool "Generic filesystem events"
> + select NET
> + default y
Do we really want to default to yes?
[ If so then maybe we want to make the config option visible
only when EXPERT mode is enabled? ]
> + help
> + Enable generic filesystem events interface
Please enhance the help entry.
> diff --git a/fs/events/Makefile b/fs/events/Makefile
> new file mode 100644
> index 0000000..9c98337
> --- /dev/null
> +++ b/fs/events/Makefile
> @@ -0,0 +1,5 @@
> +#
> +# Makefile for the Linux Generic File System Event Interface
> +#
> +
> +obj-y := fs_event.o fs_event_netlink.o
> diff --git a/fs/events/fs_event.c b/fs/events/fs_event.c
> new file mode 100644
> index 0000000..1037311
> --- /dev/null
> +++ b/fs/events/fs_event.c
> @@ -0,0 +1,809 @@
> +/*
> + * Generic File System Evens Interface
> + *
> + * Copyright(c) 2015 Samsung Electronics. All rights reserved.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2.
> + *
> + * The full GNU General Public License is included in this distribution in the
> + * file called COPYING.
> + *
> + * This program is distributed in the hope that it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
> + * more details.
> + */
> +#include <linux/fs.h>
> +#include <linux/module.h>
> +#include <linux/mount.h>
> +#include <linux/namei.h>
> +#include <linux/nsproxy.h>
> +#include <linux/parser.h>
> +#include <linux/seq_file.h>
> +#include <linux/slab.h>
> +#include <linux/rcupdate.h>
> +#include <net/genetlink.h>
> +#include "../pnode.h"
> +#include "fs_event.h"
> +
> +static LIST_HEAD(fs_trace_list);
> +static DEFINE_MUTEX(fs_trace_lock);
> +
> +static struct kmem_cache *fs_trace_cachep __read_mostly;
> +
> +static atomic_t stray_traces = ATOMIC_INIT(0);
> +static DECLARE_WAIT_QUEUE_HEAD(trace_wq);
> +/*
> + * Threshold notification state bits.
> + * Note the reverse as this refers to the number
> + * of available blocks.
> + */
> +#define THRESH_LR_BELOW 0x0001 /* Falling below the lower range */
> +#define THRESH_LR_BEYOND 0x0002
> +#define THRESH_UR_BELOW 0x0004
> +#define THRESH_UR_BEYOND 0x0008 /* Going beyond the upper range */
> +
> +#define THRESH_LR_ON (THRESH_LR_BELOW | THRESH_LR_BEYOND)
> +#define THRESH_UR_ON (THRESH_UR_BELOW | THRESH_UR_BEYOND)
> +
> +#define FS_TRACE_ADD 0x100000
> +
> +struct fs_trace_entry {
> + struct kref count;
> + atomic_t active;
> + struct super_block *sb;
> + unsigned int notify;
> + struct path mnt_path;
> + struct list_head node;
> +
> + struct fs_event_thresh {
> + u64 avail_space;
> + u64 lrange;
> + u64 urange;
> + unsigned int state;
> + } th;
> + struct rcu_head rcu_head;
> + spinlock_t lock;
> +};
> +
> +static const match_table_t fs_etypes = {
> + { FS_EVENT_GENERIC, "G" },
> + { FS_EVENT_THRESH, "T" },
> + { 0, NULL },
> +};
> +
> +static inline int fs_trace_query_data(struct super_block *sb,
> + struct fs_trace_entry *en)
> +{
> + if (sb->s_etrace.ops && sb->s_etrace.ops->query) {
> + sb->s_etrace.ops->query(sb, &en->th.avail_space);
> + return 0;
> + }
> +
> + return -EINVAL;
> +}
> +
> +static inline void fs_trace_entry_free(struct fs_trace_entry *en)
I don't see a real need for this wrapper (it is used only once).
> +{
> + kmem_cache_free(fs_trace_cachep, en);
> +}
> +
> +static void fs_destroy_trace_entry(struct kref *en_ref)
> +{
> + struct fs_trace_entry *en = container_of(en_ref,
> + struct fs_trace_entry, count);
> +
> + /* Last reference has been dropped */
> + fs_trace_entry_free(en);
> + atomic_dec(&stray_traces);
> +}
> +
> +static void fs_trace_entry_put(struct fs_trace_entry *en)
> +{
> + kref_put(&en->count, fs_destroy_trace_entry);
> +}
> +
> +static void fs_release_trace_entry(struct rcu_head *rcu_head)
> +{
> + struct fs_trace_entry *en = container_of(rcu_head,
> + struct fs_trace_entry,
> + rcu_head);
> + /*
> + * As opposed to typical reference drop, this one is being
> + * called from the rcu callback. This is to make sure all
> + * readers have managed to safely grab the reference before
> + * the change to rcu pointer is visible to all and before
> + * the reference is dropped here.
> + */
> + fs_trace_entry_put(en);
> +}
> +
> +static void fs_drop_trace_entry(struct fs_trace_entry *en)
> +{
> + struct super_block *sb;
> +
> + lockdep_assert_held(&fs_trace_lock);
> + /*
> + * The trace entry might have already been removed
> + * from the list of active traces with the proper
> + * ref drop, though it was still in use handling
> + * one of the fs events. This means that the object
> + * has been already scheduled for being released.
> + * So leave...
> + */
> +
> + if (!atomic_add_unless(&en->active, -1, 0))
> + return;
> + /*
> + * At this point the trace entry is being marked as inactive
> + * so no new references will be allowed.
> + * Still it might be floating around somewhere
> + * so drop the reference when the rcu readers are done.
> + */
> + spin_lock(&en->lock);
> + list_del(&en->node);
> + sb = en->sb;
> + en->sb = NULL;
> + spin_unlock(&en->lock);
> +
> + rcu_assign_pointer(sb->s_etrace.e_priv, NULL);
> + call_rcu(&en->rcu_head, fs_release_trace_entry);
> + /* It's safe now to drop the reference to the super */
> + deactivate_super(sb);
> + atomic_inc(&stray_traces);
> +}
> +
> +static inline
> +struct fs_trace_entry *fs_trace_entry_get(struct fs_trace_entry *en)
> +{
> + if (en) {
> + if (!kref_get_unless_zero(&en->count))
> + return NULL;
> + /* Don't allow referencing inactive object */
> + if (!atomic_read(&en->active)) {
> + fs_trace_entry_put(en);
> + return NULL;
> + }
> + }
> + return en;
> +}
> +
> +static struct fs_trace_entry *fs_trace_entry_get_rcu(struct super_block *sb)
> +{
> + struct fs_trace_entry *en;
> +
> + if (!sb)
> + return NULL;
> +
> + rcu_read_lock();
> + en = rcu_dereference(sb->s_etrace.e_priv);
> + en = fs_trace_entry_get(en);
> + rcu_read_unlock();
> +
> + return en;
> +}
> +
> +static int fs_remove_trace_entry(struct super_block *sb)
> +{
> + struct fs_trace_entry *en;
> +
> + en = fs_trace_entry_get_rcu(sb);
> + if (!en)
> + return -EINVAL;
> +
> + mutex_lock(&fs_trace_lock);
> + fs_drop_trace_entry(en);
> + mutex_unlock(&fs_trace_lock);
> + fs_trace_entry_put(en);
> + return 0;
> +}
> +
> +static void fs_remove_all_traces(void)
> +{
> + struct fs_trace_entry *en, *guard;
> +
> + mutex_lock(&fs_trace_lock);
> + list_for_each_entry_safe(en, guard, &fs_trace_list, node)
> + fs_drop_trace_entry(en);
> + mutex_unlock(&fs_trace_lock);
> +}
> +
> +static int create_common_msg(struct sk_buff *skb, void *data)
> +{
> + struct fs_trace_entry *en = (struct fs_trace_entry *)data;
> + struct super_block *sb = en->sb;
> +
> + if (nla_put_u32(skb, FS_NL_A_DEV_MAJOR, MAJOR(sb->s_dev))
> + || nla_put_u32(skb, FS_NL_A_DEV_MINOR, MINOR(sb->s_dev)))
> + return -EINVAL;
> +
> + if (nla_put_u64(skb, FS_NL_A_CAUSED_ID, pid_vnr(task_pid(current))))
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static int create_thresh_msg(struct sk_buff *skb, void *data)
> +{
> + struct fs_trace_entry *en = (struct fs_trace_entry *)data;
> + int ret;
> +
> + ret = create_common_msg(skb, data);
> + if (!ret)
> + ret = nla_put_u64(skb, FS_NL_A_DATA, en->th.avail_space);
> + return ret;
> +}
> +
> +static void fs_event_send(struct fs_trace_entry *en, unsigned int event_id)
> +{
> + size_t size = nla_total_size(sizeof(u32)) * 2 +
> + nla_total_size(sizeof(u64));
> +
> + fs_netlink_send_event(size, event_id, create_common_msg, en);
> +}
> +
> +static void fs_event_send_thresh(struct fs_trace_entry *en,
> + unsigned int event_id)
> +{
> + size_t size = nla_total_size(sizeof(u32)) * 2 +
> + nla_total_size(sizeof(u64)) * 2;
> +
> + fs_netlink_send_event(size, event_id, create_thresh_msg, en);
> +}
> +
> +void fs_event_notify(struct super_block *sb, unsigned int event_id)
> +{
> + struct fs_trace_entry *en;
> +
> + en = fs_trace_entry_get_rcu(sb);
> + if (!en)
> + return;
> +
> + spin_lock(&en->lock);
> + if (atomic_read(&en->active) && (en->notify & FS_EVENT_GENERIC))
> + fs_event_send(en, event_id);
> + spin_unlock(&en->lock);
> + fs_trace_entry_put(en);
> +}
> +EXPORT_SYMBOL(fs_event_notify);
> +
> +void fs_event_alloc_space(struct super_block *sb, u64 ncount)
> +{
> + struct fs_trace_entry *en;
> + s64 count;
> +
> + en = fs_trace_entry_get_rcu(sb);
> + if (!en)
> + return;
> +
> + spin_lock(&en->lock);
> +
> + if (!atomic_read(&en->active) || !(en->notify & FS_EVENT_THRESH))
> + goto leave;
> + /*
> + * we shouldn't drop below 0 here,
> + * unless there is a sync issue somewhere (?)
> + */
> + count = en->th.avail_space - ncount;
> + en->th.avail_space = count < 0 ? 0 : count;
> +
> + if (en->th.avail_space > en->th.lrange)
> + /* Not 'even' close - leave */
> + goto leave;
> +
> + if (en->th.avail_space > en->th.urange) {
> + /* Close enough - the lower range has been reached */
> + if (!(en->th.state & THRESH_LR_BEYOND)) {
> + /* Send notification */
> + fs_event_send_thresh(en, FS_THR_LRBELOW);
> + en->th.state &= ~THRESH_LR_BELOW;
> + en->th.state |= THRESH_LR_BEYOND;
> + }
> + goto leave;
> + }
> + if (!(en->th.state & THRESH_UR_BEYOND)) {
> + fs_event_send_thresh(en, FS_THR_URBELOW);
> + en->th.state &= ~THRESH_UR_BELOW;
> + en->th.state |= THRESH_UR_BEYOND;
> + }
> +
> +leave:
> + spin_unlock(&en->lock);
> + fs_trace_entry_put(en);
> +}
> +EXPORT_SYMBOL(fs_event_alloc_space);
> +
> +void fs_event_free_space(struct super_block *sb, u64 ncount)
> +{
> + struct fs_trace_entry *en;
> +
> + en = fs_trace_entry_get_rcu(sb);
> + if (!en)
> + return;
> +
> + spin_lock(&en->lock);
> +
> + if (!atomic_read(&en->active) || !(en->notify & FS_EVENT_THRESH))
> + goto leave;
> +
> + en->th.avail_space += ncount;
> +
> + if (en->th.avail_space > en->th.lrange) {
> + if (!(en->th.state & THRESH_LR_BELOW)
> + && en->th.state & THRESH_LR_BEYOND) {
> + /* Send notification */
> + fs_event_send_thresh(en, FS_THR_LRABOVE);
> + en->th.state &= ~(THRESH_LR_BEYOND|THRESH_UR_BEYOND);
> + en->th.state |= THRESH_LR_BELOW;
> + goto leave;
> + }
> + }
> + if (en->th.avail_space > en->th.urange) {
> + if (!(en->th.state & THRESH_UR_BELOW)
> + && en->th.state & THRESH_UR_BEYOND) {
> + /* Notify */
> + fs_event_send_thresh(en, FS_THR_URABOVE);
> + en->th.state &= ~THRESH_UR_BEYOND;
> + en->th.state |= THRESH_UR_BELOW;
> + }
> + }
> +leave:
> + spin_unlock(&en->lock);
> + fs_trace_entry_put(en);
> +}
> +EXPORT_SYMBOL(fs_event_free_space);
> +
> +void fs_event_mount_dropped(struct vfsmount *mnt)
> +{
> + /*
> + * The mount is dropped but the super might not get released
> + * at once so there is very small chance some notifications
> + * will come through.
> + * Note that the mount being dropped here might belong to a different
> + * namespace - if this is the case, just ignore it.
> + */
> + struct fs_trace_entry *en = fs_trace_entry_get_rcu(mnt->mnt_sb);
> + struct vfsmount *en_mnt;
> +
> + if (!en || !atomic_read(&en->active))
> + return;
> + /*
> + * The entry once set, does not change the mountpoint it's being
> + * pinned to, so no need to take the lock here.
> + */
> + en_mnt = en->mnt_path.mnt;
> + if (!(real_mount(mnt)->mnt_ns != (real_mount(en_mnt))->mnt_ns))
> + fs_remove_trace_entry(mnt->mnt_sb);
> + fs_trace_entry_put(en);
> +}
> +
> +static int fs_new_trace_entry(struct path *path, struct fs_event_thresh *thresh,
> + unsigned int nmask)
> +{
> + struct fs_trace_entry *en;
> + struct super_block *sb;
> + struct mount *r_mnt;
> +
> + en = kmem_cache_zalloc(fs_trace_cachep, GFP_KERNEL);
> + if (unlikely(!en))
> + return -ENOMEM;
> + /*
> + * Note that no reference is being taken here for the path as it would
> + * make the unmount unnecessarily puzzling (due to an extra 'valid'
> + * reference for the mnt).
> + * This is *rather* safe as the notification on mount being dropped
> + * will get called prior to releasing the super block - so right
> + * in time to perform appropriate clean-up
> + */
> + r_mnt = real_mount(path->mnt);
> +
> + en->mnt_path.dentry = r_mnt->mnt.mnt_root;
> + en->mnt_path.mnt = &r_mnt->mnt;
> +
> + sb = path->mnt->mnt_sb;
> + en->sb = sb;
> + /*
> + * Increase the refcount for sb to mark it's being relied on.
> + * Note that the reference to path is taken by the caller, so it
> + * is safe to assume there is at least single active reference
> + * to super as well.
> + */
> + atomic_inc(&sb->s_active);
> +
> + nmask &= sb->s_etrace.events_cap_mask;
> + if (!nmask)
> + goto leave;
> +
> + spin_lock_init(&en->lock);
> + INIT_LIST_HEAD(&en->node);
> +
> + en->notify = nmask;
> + memcpy(&en->th, thresh, offsetof(struct fs_event_thresh, state));
> + if (nmask & FS_EVENT_THRESH)
> + fs_trace_query_data(sb, en);
> +
> + kref_init(&en->count);
> +
> + if (rcu_access_pointer(sb->s_etrace.e_priv) != NULL) {
> + struct fs_trace_entry *prev_en;
> +
> + prev_en = fs_trace_entry_get_rcu(sb);
> + if (prev_en) {
> + WARN_ON(prev_en);
> + fs_trace_entry_put(prev_en);
> + goto leave;
> + }
> + }
> + atomic_set(&en->active, 1);
> +
> + mutex_lock(&fs_trace_lock);
> + list_add(&en->node, &fs_trace_list);
> + mutex_unlock(&fs_trace_lock);
> +
> + rcu_assign_pointer(sb->s_etrace.e_priv, en);
> + synchronize_rcu();
> +
> + return 0;
> +leave:
> + deactivate_super(sb);
> + kmem_cache_free(fs_trace_cachep, en);
> + return -EINVAL;
> +}
> +
> +static int fs_update_trace_entry(struct path *path,
> + struct fs_event_thresh *thresh,
> + unsigned int nmask)
> +{
> + struct fs_trace_entry *en;
> + struct super_block *sb;
> + int extend = nmask & FS_TRACE_ADD;
> + int ret = -EINVAL;
> +
> + en = fs_trace_entry_get_rcu(path->mnt->mnt_sb);
> + if (!en)
> + return (extend) ? fs_new_trace_entry(path, thresh, nmask)
> + : -EINVAL;
> +
> + if (!atomic_read(&en->active))
> + return -EINVAL;
> +
> + nmask &= ~FS_TRACE_ADD;
> +
> + spin_lock(&en->lock);
> + sb = en->sb;
> + if (!sb || !(nmask & sb->s_etrace.events_cap_mask))
> + goto leave;
> +
> + if (nmask & FS_EVENT_THRESH) {
> + if (extend) {
> + /* Get the current state */
> + if (!(en->notify & FS_EVENT_THRESH))
> + if (fs_trace_query_data(sb, en))
> + goto leave;
> +
> + if (thresh->state & THRESH_LR_ON) {
> + en->th.lrange = thresh->lrange;
> + en->th.state &= ~THRESH_LR_ON;
> + }
> +
> + if (thresh->state & THRESH_UR_ON) {
> + en->th.urange = thresh->urange;
> + en->th.state &= ~THRESH_UR_ON;
> + }
> + } else {
> + memset(&en->th, 0, sizeof(en->th));
> + }
> + }
> +
> + if (extend)
> + en->notify |= nmask;
> + else
> + en->notify &= ~nmask;
> + ret = 0;
> +leave:
> + spin_unlock(&en->lock);
> + fs_trace_entry_put(en);
> + return ret;
> +}
> +
> +static int fs_parse_trace_request(int argc, char **argv)
> +{
> + struct fs_event_thresh thresh = {0};
> + struct path path;
> + substring_t args[MAX_OPT_ARGS];
> + unsigned int nmask = FS_TRACE_ADD;
> + int token;
> + char *s;
> + int ret = -EINVAL;
> +
> + if (!argc) {
> + fs_remove_all_traces();
> + return 0;
> + }
> +
> + s = *(argv);
> + if (*s == '!') {
> + /* Clear the trace entry */
> + nmask &= ~FS_TRACE_ADD;
> + ++s;
> + }
> +
> + if (kern_path_mountpoint(AT_FDCWD, s, &path, LOOKUP_FOLLOW))
> + return -EINVAL;
> +
> + if (!(--argc)) {
> + if (!(nmask & FS_TRACE_ADD))
> + ret = fs_remove_trace_entry(path.mnt->mnt_sb);
> + goto leave;
> + }
> +
> +repeat:
> + args[0].to = args[0].from = NULL;
> + token = match_token(*(++argv), fs_etypes, args);
> + if (!token && !nmask)
> + goto leave;
> +
> + nmask |= token & FS_EVENTS_ALL;
> + --argc;
> + if ((token & FS_EVENT_THRESH) && (nmask & FS_TRACE_ADD)) {
> + /*
> + * Get the threshold config data:
> + * lower range
> + * upper range
> + */
> + if (!argc)
> + goto leave;
> +
> + ret = kstrtoull(*(++argv), 10, &thresh.lrange);
> + if (ret)
> + goto leave;
> + thresh.state |= THRESH_LR_ON;
> + if ((--argc)) {
> + ret = kstrtoull(*(++argv), 10, &thresh.urange);
> + if (ret)
> + goto leave;
> + thresh.state |= THRESH_UR_ON;
> + --argc;
> + }
> + /* The thresholds are based on number of available blocks */
> + if (thresh.lrange < thresh.urange) {
> + ret = -EINVAL;
> + goto leave;
> + }
> + }
> + if (argc)
> + goto repeat;
> +
> + ret = fs_update_trace_entry(&path, &thresh, nmask);
> +leave:
> + path_put(&path);
> + return ret;
> +}
> +
> +#define DEFAULT_BUF_SIZE PAGE_SIZE
> +
> +static ssize_t fs_trace_write(struct file *file, const char __user *buffer,
> + size_t count, loff_t *ppos)
> +{
> + char **argv;
> + char *kern_buf, *next, *cfg;
> + size_t size, dcount = 0;
> + int argc;
> +
> + if (!count)
> + return 0;
> +
> + kern_buf = kmalloc(DEFAULT_BUF_SIZE, GFP_KERNEL);
> + if (!kern_buf)
> + return -ENOMEM;
> +
> + while (dcount < count) {
> +
> + size = count - dcount;
> + if (size >= DEFAULT_BUF_SIZE)
> + size = DEFAULT_BUF_SIZE - 1;
> + if (copy_from_user(kern_buf, buffer + dcount, size)) {
> + dcount = -EINVAL;
> + goto leave;
> + }
> +
> + kern_buf[size] = '\0';
> +
> + next = cfg = kern_buf;
> +
> + do {
> + next = strchr(cfg, ';');
> + if (next)
> + *next = '\0';
> +
> + argv = argv_split(GFP_KERNEL, cfg, &argc);
> + if (!argv) {
> + dcount = -ENOMEM;
> + goto leave;
> + }
> +
> + if (fs_parse_trace_request(argc, argv)) {
> + dcount = -EINVAL;
> + argv_free(argv);
> + goto leave;
> + }
> +
> + argv_free(argv);
> + if (next)
> + cfg = ++next;
> +
> + } while (next);
> + dcount += size;
> + }
> +leave:
> + kfree(kern_buf);
> + return dcount;
> +}
> +
> +static void *fs_trace_seq_start(struct seq_file *m, loff_t *pos)
> +{
> + mutex_lock(&fs_trace_lock);
> + return seq_list_start(&fs_trace_list, *pos);
> +}
> +
> +static void *fs_trace_seq_next(struct seq_file *m, void *v, loff_t *pos)
> +{
> + return seq_list_next(v, &fs_trace_list, pos);
> +}
> +
> +static void fs_trace_seq_stop(struct seq_file *m, void *v)
> +{
> + mutex_unlock(&fs_trace_lock);
> +}
> +
> +static int fs_trace_seq_show(struct seq_file *m, void *v)
> +{
> + struct fs_trace_entry *en;
> + struct super_block *sb;
> + struct mount *r_mnt;
> + const struct match_token *match;
> + unsigned int nmask;
> +
> + en = list_entry(v, struct fs_trace_entry, node);
> + /* Do not show the entries outside current mount namespace */
> + r_mnt = real_mount(en->mnt_path.mnt);
> + if (r_mnt->mnt_ns != current->nsproxy->mnt_ns) {
> + if (!__is_local_mountpoint(r_mnt->mnt_mountpoint))
> + return 0;
> + }
> +
> + sb = en->sb;
> +
> + seq_path(m, &en->mnt_path, "\t\n\\");
> + seq_putc(m, ' ');
> +
> + seq_escape(m, sb->s_type->name, " \t\n\\");
> + if (sb->s_subtype && sb->s_subtype[0]) {
> + seq_putc(m, '.');
> + seq_escape(m, sb->s_subtype, " \t\n\\");
> + }
> +
> + seq_putc(m, ' ');
> + if (sb->s_op->show_devname) {
> + sb->s_op->show_devname(m, en->mnt_path.mnt->mnt_root);
> + } else {
> + seq_escape(m, r_mnt->mnt_devname ? r_mnt->mnt_devname : "none",
> + " \t\n\\");
> + }
> + seq_puts(m, " (");
> +
> + nmask = en->notify;
> + for (match = fs_etypes; match->pattern; ++match) {
> + if (match->token & nmask) {
> + seq_puts(m, match->pattern);
> + nmask &= ~match->token;
> + if (nmask)
> + seq_putc(m, ',');
> + }
> + }
> + seq_printf(m, " %llu %llu", en->th.lrange, en->th.urange);
> + seq_puts(m, ")\n");
> + return 0;
> +}
> +
> +static const struct seq_operations fs_trace_seq_ops = {
> + .start = fs_trace_seq_start,
> + .next = fs_trace_seq_next,
> + .stop = fs_trace_seq_stop,
> + .show = fs_trace_seq_show,
> +};
> +
> +static int fs_trace_open(struct inode *inode, struct file *file)
> +{
> + return seq_open(file, &fs_trace_seq_ops);
> +}
> +
> +static const struct file_operations fs_trace_fops = {
> + .owner = THIS_MODULE,
> + .open = fs_trace_open,
> + .write = fs_trace_write,
> + .read = seq_read,
> + .llseek = seq_lseek,
> + .release = seq_release,
> +};
> +
> +static int fs_trace_init(void)
> +{
> + fs_trace_cachep = KMEM_CACHE(fs_trace_entry, 0);
> + if (!fs_trace_cachep)
> + return -EINVAL;
> + init_waitqueue_head(&trace_wq);
> + return 0;
> +}
> +
> +/* VFS support */
> +static int fs_trace_fill_super(struct super_block *sb, void *data, int silen)
> +{
> + int ret;
> + static struct tree_descr desc[] = {
> + [2] = {
> + .name = "config",
> + .ops = &fs_trace_fops,
> + .mode = S_IWUSR | S_IRUGO,
> + },
> + {""},
> + };
> +
> + ret = simple_fill_super(sb, 0x7246332, desc);
Please use a define for a magic number.
> + return !ret ? fs_trace_init() : ret;
> +}
> +
> +static struct dentry *fs_trace_do_mount(struct file_system_type *fs_type,
> + int ntype, const char *dev_name, void *data)
> +{
> + return mount_single(fs_type, ntype, data, fs_trace_fill_super);
> +}
> +
> +static void fs_trace_kill_super(struct super_block *sb)
> +{
> + /*
> + * The rcu_barrier here will/should make sure all call_rcu
> + * callbacks are completed - still there might be some active
> + * trace objects in use which can make calling the
> + * kmem_cache_destroy unsafe. So we wait until all traces
> + * are finally released.
> + */
> + fs_remove_all_traces();
> + rcu_barrier();
> + wait_event(trace_wq, !atomic_read(&stray_traces));
> +
> + kmem_cache_destroy(fs_trace_cachep);
> + kill_litter_super(sb);
> +}
> +
> +static struct kset *fs_trace_kset;
> +
> +static struct file_system_type fs_trace_fstype = {
> + .name = "fstrace",
> + .mount = fs_trace_do_mount,
> + .kill_sb = fs_trace_kill_super,
> +};
> +
> +static void __init fs_trace_vfs_init(void)
> +{
> + fs_trace_kset = kset_create_and_add("events", NULL, fs_kobj);
> +
> + if (!fs_trace_kset)
> + return;
> +
> + if (!register_filesystem(&fs_trace_fstype)) {
> + if (!fs_event_netlink_register())
> + return;
> + unregister_filesystem(&fs_trace_fstype);
> + }
> + kset_unregister(fs_trace_kset);
> +}
> +
> +static int __init fs_trace_evens_init(void)
> +{
> + fs_trace_vfs_init();
> + return 0;
> +};
> +module_init(fs_trace_evens_init);
> +
> diff --git a/fs/events/fs_event.h b/fs/events/fs_event.h
> new file mode 100644
> index 0000000..23f24c8
> --- /dev/null
> +++ b/fs/events/fs_event.h
> @@ -0,0 +1,22 @@
> +/*
> + * Copyright(c) 2015 Samsung Electronics. All rights reserved.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2.
> + *
> + * The full GNU General Public License is included in this distribution in the
> + * file called COPYING.
> + *
> + * This program is distributed in the hope that it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
> + * more details.
> + */
> +
> +#ifndef __GENERIC_FS_EVENTS_H
> +#define __GENERIC_FS_EVENTS_H
> +
> +int fs_event_netlink_register(void);
> +void fs_event_netlink_unregister(void);
> +
> +#endif /* __GENERIC_FS_EVENTS_H */
> diff --git a/fs/events/fs_event_netlink.c b/fs/events/fs_event_netlink.c
> new file mode 100644
> index 0000000..0c97eb7
> --- /dev/null
> +++ b/fs/events/fs_event_netlink.c
> @@ -0,0 +1,104 @@
> +/*
> + * Copyright(c) 2015 Samsung Electronics. All rights reserved.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2.
> + *
> + * The full GNU General Public License is included in this distribution in the
> + * file called COPYING.
> + *
> + * This program is distributed in the hope that it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
> + * more details.
> + */
> +#include <linux/fs.h>
> +#include <linux/init.h>
> +#include <linux/kernel.h>
> +#include <linux/sched.h>
> +#include <linux/slab.h>
> +#include <net/netlink.h>
> +#include <net/genetlink.h>
> +#include "fs_event.h"
> +
> +static const struct genl_multicast_group fs_event_mcgroups[] = {
> + { .name = FS_EVENTS_MCAST_GRP_NAME, },
> +};
> +
> +static struct genl_family fs_event_family = {
> + .id = GENL_ID_GENERATE,
> + .name = FS_EVENTS_FAMILY_NAME,
> + .version = 1,
> + .maxattr = FS_NL_A_MAX,
> + .mcgrps = fs_event_mcgroups,
> + .n_mcgrps = ARRAY_SIZE(fs_event_mcgroups),
> +};
> +
> +int fs_netlink_send_event(size_t size, unsigned int event_id,
> + int (*compose_msg)(struct sk_buff *skb, void *data),
> + void *cbdata)
> +{
> + static atomic_t seq;
> + struct sk_buff *skb;
> + void *msg_head;
> + int ret = 0;
> +
> + if (!size || !compose_msg)
> + return -EINVAL;
> +
> + /* Skip if there are no listeners */
> + if (!genl_has_listeners(&fs_event_family, &init_net, 0))
> + return 0;
> +
> + if (event_id != FS_EVENT_NONE)
> + size += nla_total_size(sizeof(u32));
> + size += nla_total_size(sizeof(u64));
> + skb = genlmsg_new(size, GFP_NOWAIT);
> +
> + if (!skb) {
> + pr_debug("Failed to allocate new FS generic netlink message\n");
> + return -ENOMEM;
> + }
> +
> + msg_head = genlmsg_put(skb, 0, atomic_add_return(1, &seq),
> + &fs_event_family, 0, FS_NL_C_EVENT);
> + if (!msg_head)
> + goto cleanup;
> +
> + if (event_id != FS_EVENT_NONE)
> + if (nla_put_u32(skb, FS_NL_A_EVENT_ID, event_id))
> + goto cancel;
> +
> + ret = compose_msg(skb, cbdata);
> + if (ret)
> + goto cancel;
> +
> + genlmsg_end(skb, msg_head);
> + ret = genlmsg_multicast(&fs_event_family, skb, 0, 0, GFP_NOWAIT);
> + if (ret && ret != -ENOBUFS && ret != -ESRCH)
> + goto cleanup;
> +
> + return ret;
> +
> +cancel:
> + genlmsg_cancel(skb, msg_head);
> +cleanup:
> + nlmsg_free(skb);
> + return ret;
> +}
> +EXPORT_SYMBOL(fs_netlink_send_event);
> +
> +int fs_event_netlink_register(void)
> +{
> + int ret;
> +
> + ret = genl_register_family(&fs_event_family);
> + if (ret)
> + pr_err("Failed to register FS netlink interface\n");
> + return ret;
> +}
> +
> +void fs_event_netlink_unregister(void)
> +{
> + genl_unregister_family(&fs_event_family);
> +}
> diff --git a/fs/namespace.c b/fs/namespace.c
> index 82ef140..ec6e2ef 100644
> --- a/fs/namespace.c
> +++ b/fs/namespace.c
> @@ -1031,6 +1031,7 @@ static void cleanup_mnt(struct mount *mnt)
> if (unlikely(mnt->mnt_pins.first))
> mnt_pin_kill(mnt);
> fsnotify_vfsmount_delete(&mnt->mnt);
> + fs_event_mount_dropped(&mnt->mnt);
> dput(mnt->mnt.mnt_root);
> deactivate_super(mnt->mnt.mnt_sb);
> mnt_free_id(mnt);
> diff --git a/include/linux/fs.h b/include/linux/fs.h
> index b4d71b5..b7dadd9 100644
> --- a/include/linux/fs.h
> +++ b/include/linux/fs.h
> @@ -263,6 +263,10 @@ struct iattr {
> * Includes for diskquotas.
> */
> #include <linux/quota.h>
> +/*
> + * Include for Generic File System Events Interface
> + */
> +#include <linux/fs_event.h>
>
> /*
> * Maximum number of layers of fs stack. Needs to be limited to
> @@ -1253,7 +1257,7 @@ struct super_block {
> struct hlist_node s_instances;
> unsigned int s_quota_types; /* Bitmask of supported quota types */
> struct quota_info s_dquot; /* Diskquota specific options */
> -
> + struct fs_trace_info s_etrace;
> struct sb_writers s_writers;
>
> char s_id[32]; /* Informational name */
> diff --git a/include/linux/fs_event.h b/include/linux/fs_event.h
> new file mode 100644
> index 0000000..83e22dd
> --- /dev/null
> +++ b/include/linux/fs_event.h
> @@ -0,0 +1,72 @@
> +/*
> + * Generic File System Events Interface
> + *
> + * Copyright(c) 2015 Samsung Electronics. All rights reserved.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2.
> + *
> + * The full GNU General Public License is included in this distribution in the
> + * file called COPYING.
> + *
> + * This program is distributed in the hope that it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
> + * more details.
> + */
> +#ifndef _LINUX_GENERIC_FS_EVETS_
> +#define _LINUX_GENERIC_FS_EVETS_
EVETS?
Also the define name usually corresponds to the header filename.
> +#include <net/netlink.h>
> +#include <uapi/linux/fs_event.h>
> +
> +/*
> + * Currently supported event types
> + */
> +#define FS_EVENT_GENERIC 0x001
> +#define FS_EVENT_THRESH 0x002
> +
> +#define FS_EVENTS_ALL (FS_EVENT_GENERIC | FS_EVENT_THRESH)
> +
> +struct fs_trace_operations {
> + void (*query)(struct super_block *, u64 *);
> +};
> +
> +struct fs_trace_info {
> + void __rcu *e_priv; /* READ ONLY */
> + unsigned int events_cap_mask; /* Supported notifications */
> + const struct fs_trace_operations *ops;
> +};
> +
> +#ifdef CONFIG_FS_EVENTS
> +
> +void fs_event_notify(struct super_block *sb, unsigned int event_id);
> +void fs_event_alloc_space(struct super_block *sb, u64 ncount);
> +void fs_event_free_space(struct super_block *sb, u64 ncount);
> +void fs_event_mount_dropped(struct vfsmount *mnt);
> +
> +int fs_netlink_send_event(size_t size, unsigned int event_id,
> + int (*compose_msg)(struct sk_buff *skb, void *data),
> + void *cbdata);
> +
> +#else /* CONFIG_FS_EVENTS */
> +
> +static inline
> +void fs_event_notify(struct super_block *sb, unsigned int event_id) {};
> +static inline
> +void fs_event_alloc_space(struct super_block *sb, u64 ncount) {};
> +static inline
> +void fs_event_free_space(struct super_block *sb, u64 ncount) {};
> +static inline
> +void fs_event_mount_dropped(struct vfsmount *mnt) {};
> +
> +static inline
> +int fs_netlink_send_event(size_t size, unsigned int event_id,
> + int (*compose_msig)(struct sk_buff *skb, void *data),
> + void *cbdata)
> +{
> + return -ENOSYS;
> +}
> +#endif /* CONFIG_FS_EVENTS */
> +
> +#endif /* _LINUX_GENERIC_FS_EVENTS_ */
> +
> diff --git a/include/uapi/linux/Kbuild b/include/uapi/linux/Kbuild
> index 68ceb97..dae0fab 100644
> --- a/include/uapi/linux/Kbuild
> +++ b/include/uapi/linux/Kbuild
> @@ -129,6 +129,7 @@ header-y += firewire-constants.h
> header-y += flat.h
> header-y += fou.h
> header-y += fs.h
> +header-y += fs_event.h
> header-y += fsl_hypervisor.h
> header-y += fuse.h
> header-y += futex.h
> diff --git a/include/uapi/linux/fs_event.h b/include/uapi/linux/fs_event.h
> new file mode 100644
> index 0000000..d8b07da
> --- /dev/null
> +++ b/include/uapi/linux/fs_event.h
> @@ -0,0 +1,58 @@
> +/*
> + * Generic netlink support for Generic File System Events Interface
> + *
> + * Copyright(c) 2015 Samsung Electronics. All rights reserved.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License version 2.
> + *
> + * The full GNU General Public License is included in this distribution in the
> + * file called COPYING.
> + *
> + * This program is distributed in the hope that it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
> + * more details.
> + */
> +#ifndef _UAPI_LINUX_GENERIC_FS_EVENTS_
> +#define _UAPI_LINUX_GENERIC_FS_EVENTS_
Define name usually corresponds to the header filename.
> +#define FS_EVENTS_FAMILY_NAME "fs_event"
> +#define FS_EVENTS_MCAST_GRP_NAME "fs_event_mc_grp"
> +
> +/*
> + * Generic netlink attribute types
> + */
> +enum {
> + FS_NL_A_NONE,
> + FS_NL_A_EVENT_ID,
> + FS_NL_A_DEV_MAJOR,
> + FS_NL_A_DEV_MINOR,
> + FS_NL_A_CAUSED_ID,
> + FS_NL_A_DATA,
> + __FS_NL_A_MAX,
> +};
> +#define FS_NL_A_MAX (__FS_NL_A_MAX - 1)
> +/*
> + * Generic netlink commands
> + */
> +#define FS_NL_C_EVENT 1
> +
> +/*
> + * Supported set of FS events
> + */
> +enum {
> + FS_EVENT_NONE,
> + FS_WARN_ENOSPC, /* No space left to reserve data blks */
> + FS_WARN_ENOSPC_META, /* No space left for metadata */
> + FS_THR_LRBELOW, /* The threshold lower range has been reached */
> + FS_THR_LRABOVE, /* The threshold lower range re-activcated*/
> + FS_THR_URBELOW,
> + FS_THR_URABOVE,
> + FS_ERR_REMOUNT_RO, /* The file system has been remounted as RO */
> + FS_ERR_CORRUPTED /* Critical error - fs corrupted */
> +
> +};
> +
> +#endif /* _UAPI_LINUX_GENERIC_FS_EVENTS_ */
> +
Best regards,
--
Bartlomiej Zolnierkiewicz
Samsung R&D Institute Poland
Samsung Electronics
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH v5 39/39] nfs: Add support for the v4.1 dacl attribute
From: Anna Schumaker @ 2015-07-22 15:57 UTC (permalink / raw)
To: Andreas Gruenbacher, linux-kernel
Cc: linux-fsdevel, linux-nfs, linux-api, samba-technical,
linux-security-module, Andreas Gruenbacher
In-Reply-To: <1437570209-29832-40-git-send-email-andreas.gruenbacher@gmail.com>
Hi Andreas,
On 07/22/2015 09:03 AM, Andreas Gruenbacher wrote:
> From: Andreas Gruenbacher <agruenba@redhat.com>
>
> The dacl attribute includes Automatic Inheritance flags not supported by the
> acl attribute. it is only supported in NFS version 4.1 and higher. On systems
> where NFS version 4.0 is still the default, an additional mount option is
> needed:
>
> mount -t nfs4 -o minorversion=1 [...]
Discussions around deprecating or removing the minorversion= argument have come up recently. The preferred way of mounting NFS v4.1 these days is with: "-o vers=4.1". Can you update your patch description so we don't confuse people? :)
Thanks,
Anna
>
> Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
> ---
> fs/nfs/nfs4proc.c | 2 +-
> fs/nfs/nfs4xdr.c | 55 ++++++++++++++++++++++++++++++++++++++++++-------
> include/linux/nfs_xdr.h | 2 +-
> 3 files changed, 50 insertions(+), 9 deletions(-)
>
> diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c
> index 5ba49c3..39f5548 100644
> --- a/fs/nfs/nfs4proc.c
> +++ b/fs/nfs/nfs4proc.c
> @@ -4469,7 +4469,7 @@ static struct richacl *__nfs4_get_acl_uncached(struct inode *inode)
> struct nfs_server *server = NFS_SERVER(inode);
> struct page *pages[DIV_ROUND_UP(NFS4ACL_SIZE_MAX, PAGE_SIZE)] = {};
> struct nfs_getaclargs args = {
> - .fh = NFS_FH(inode),
> + .inode = inode,
> .acl_pages = pages,
> .acl_len = ARRAY_SIZE(pages) * PAGE_SIZE,
> };
> diff --git a/fs/nfs/nfs4xdr.c b/fs/nfs/nfs4xdr.c
> index 85270df..06f5759 100644
> --- a/fs/nfs/nfs4xdr.c
> +++ b/fs/nfs/nfs4xdr.c
> @@ -1649,9 +1649,16 @@ encode_setacl(struct xdr_stream *xdr, struct nfs_setaclargs *arg, struct compoun
> encode_nfs4_stateid(xdr, &zero_stateid);
>
> /* Encode attribute bitmap. */
> - p = reserve_space(xdr, 2*4);
> - *p++ = cpu_to_be32(1);
> - *p = cpu_to_be32(FATTR4_WORD0_ACL);
> + if (arg->server->attr_bitmask[1] & FATTR4_WORD1_DACL) {
> + p = reserve_space(xdr, 3*4);
> + *p++ = cpu_to_be32(2);
> + *p++ = 0;
> + *p = cpu_to_be32(FATTR4_WORD1_DACL);
> + } else {
> + p = reserve_space(xdr, 2*4);
> + *p++ = cpu_to_be32(1);
> + *p = cpu_to_be32(FATTR4_WORD0_ACL);
> + }
>
> attrlen_offset = xdr->buf->len;
> xdr_reserve_space(xdr, 4); /* to be backfilled later */
> @@ -2486,9 +2493,12 @@ static void nfs4_xdr_enc_getacl(struct rpc_rqst *req, struct xdr_stream *xdr,
>
> encode_compound_hdr(xdr, req, &hdr);
> encode_sequence(xdr, &args->seq_args, &hdr);
> - encode_putfh(xdr, args->fh, &hdr);
> + encode_putfh(xdr, NFS_FH(args->inode), &hdr);
> replen = hdr.replen + op_decode_hdr_maxsz + 1;
> - encode_getattr_two(xdr, FATTR4_WORD0_ACL, FATTR4_WORD1_MODE, &hdr);
> + if (NFS_SERVER(args->inode)->attr_bitmask[1] & FATTR4_WORD1_DACL)
> + encode_getattr_two(xdr, 0, FATTR4_WORD1_MODE | FATTR4_WORD1_DACL, &hdr);
> + else
> + encode_getattr_two(xdr, FATTR4_WORD0_ACL, FATTR4_WORD1_MODE, &hdr);
>
> xdr_inline_pages(&req->rq_rcv_buf, replen << 2,
> args->acl_pages, 0, args->acl_len);
> @@ -5376,14 +5386,28 @@ static int decode_getacl(struct xdr_stream *xdr, struct rpc_rqst *req,
>
> if (unlikely(bitmap[0] & (FATTR4_WORD0_ACL - 1U)))
> return -EIO;
> - if (likely(bitmap[0] & FATTR4_WORD0_ACL)) {
> +
> + if (bitmap[0] & FATTR4_WORD0_ACL) {
> + struct richace *ace;
> +
> + if (bitmap[1] & FATTR4_WORD1_DACL)
> + return -EIO;
> +
> acl = decode_acl_entries(xdr, res->server);
> status = PTR_ERR(acl);
> if (IS_ERR(acl))
> goto out;
> +
> + status = -EIO;
> + richacl_for_each_entry(ace, acl) {
> + if (ace->e_flags & RICHACE_INHERITED_ACE)
> + goto out;
> + }
> bitmap[0] &= ~FATTR4_WORD0_ACL;
> - } else
> + } else if (!(bitmap[1] & FATTR4_WORD1_DACL)) {
> status = -EOPNOTSUPP;
> + goto out;
> + }
>
> status = -EIO;
> if (unlikely(bitmap[0]))
> @@ -5391,6 +5415,23 @@ static int decode_getacl(struct xdr_stream *xdr, struct rpc_rqst *req,
>
> if ((status = decode_attr_mode(xdr, bitmap, &res->mode)) < 0)
> goto out;
> + if (bitmap[1] & FATTR4_WORD1_DACL) {
> + unsigned int flags;
> + __be32 *p;
> +
> + p = xdr_inline_decode(xdr, 4);
> + status = -EIO;
> + if (unlikely(!p))
> + goto out;
> + flags = be32_to_cpup(p);
> +
> + acl = decode_acl_entries(xdr, res->server);
> + status = PTR_ERR(acl);
> + if (IS_ERR(acl))
> + goto out;
> + acl->a_flags = flags;
> + bitmap[1] &= ~FATTR4_WORD1_DACL;
> + }
> status = 0;
>
> out:
> diff --git a/include/linux/nfs_xdr.h b/include/linux/nfs_xdr.h
> index f1c8f93..abe0b66 100644
> --- a/include/linux/nfs_xdr.h
> +++ b/include/linux/nfs_xdr.h
> @@ -695,7 +695,7 @@ struct nfs_setaclres {
>
> struct nfs_getaclargs {
> struct nfs4_sequence_args seq_args;
> - struct nfs_fh * fh;
> + struct inode * inode;
> size_t acl_len;
> struct page ** acl_pages;
> };
>
^ permalink raw reply
* Re: [PATCH v5 39/39] nfs: Add support for the v4.1 dacl attribute
From: Andreas Grünbacher @ 2015-07-22 16:08 UTC (permalink / raw)
To: Anna Schumaker
Cc: Linux Kernel Mailing List, Linux FS-devel Mailing List,
Linux NFS Mailing List, Linux API Mailing List, samba-technical,
linux-security-module, Andreas Gruenbacher
In-Reply-To: <55AFBD84.5040904@Netapp.com>
Hi Anna,
2015-07-22 17:57 GMT+02:00 Anna Schumaker <Anna.Schumaker@netapp.com>:
> Discussions around deprecating or removing the minorversion= argument have come up recently.
> The preferred way of mounting NFS v4.1 these days is with: "-o vers=4.1". Can you update your
> patch description so we don't confuse people? :)
sure, that was easy :)
Thanks,
Andreas
^ permalink raw reply
* Re: [PATCH -mm v9 0/8] idle memory tracking
From: Vladimir Davydov @ 2015-07-22 16:23 UTC (permalink / raw)
To: Andrew Morton
Cc: Andres Lagar-Cavilla, Minchan Kim, Raghavendra K T,
Johannes Weiner, Michal Hocko, Greg Thelen, Michel Lespinasse,
David Rientjes, Pavel Emelyanov, Cyrill Gorcunov, Jonathan Corbet,
linux-api, linux-doc, linux-mm, cgroups, linux-kernel, Kees Cook
In-Reply-To: <20150721163402.43ad2527d9b8caa476a1c9e1@linux-foundation.org>
On Tue, Jul 21, 2015 at 04:34:02PM -0700, Andrew Morton wrote:
> On Sun, 19 Jul 2015 15:31:09 +0300 Vladimir Davydov <vdavydov@parallels.com> wrote:
>
> > Hi,
> >
> > This patch set introduces a new user API for tracking user memory pages
> > that have not been used for a given period of time. The purpose of this
> > is to provide the userspace with the means of tracking a workload's
> > working set, i.e. the set of pages that are actively used by the
> > workload. Knowing the working set size can be useful for partitioning
> > the system more efficiently, e.g. by tuning memory cgroup limits
> > appropriately, or for job placement within a compute cluster.
> >
> > It is based on top of v4.2-rc2-mmotm-2015-07-15-16-46
> > It applies without conflicts to v4.2-rc2-mmotm-2015-07-17-16-04 as well
> >
> > ---- USE CASES ----
> >
> > The unified cgroup hierarchy has memory.low and memory.high knobs, which
> > are defined as the low and high boundaries for the workload working set
> > size. However, the working set size of a workload may be unknown or
> > change in time. With this patch set, one can periodically estimate the
> > amount of memory unused by each cgroup and tune their memory.low and
> > memory.high parameters accordingly, therefore optimizing the overall
> > memory utilization.
> >
> > Another use case is balancing workloads within a compute cluster.
> > Knowing how much memory is not really used by a workload unit may help
> > take a more optimal decision when considering migrating the unit to
> > another node within the cluster.
> >
> > Also, as noted by Minchan, this would be useful for per-process reclaim
> > (https://lwn.net/Articles/545668/). With idle tracking, we could reclaim idle
> > pages only by smart user memory manager.
> >
> > ---- USER API ----
> >
> > The user API consists of two new proc files:
> >
> > * /proc/kpageidle. This file implements a bitmap where each bit corresponds
> > to a page, indexed by PFN.
>
> What are the bit mappings? If I read the first byte of /proc/kpageidle
> I get PFN #0 in bit zero of that byte? And the second byte of
> /proc/kpageidle contains PFN #8 in its LSB, etc?
The bit mapping is an array of u64 elements. Page at pfn #i corresponds
to bit #i%64 of element #i/64. Byte order is native.
Will add this to docs.
>
> Maybe this is covered in the documentation file.
>
> > When the bit is set, the corresponding page is
> > idle. A page is considered idle if it has not been accessed since it was
> > marked idle.
>
> Perhaps we can spell out in some detail what "accessed" means? I see
> you've hooked into mark_page_accessed(), so a read from disk is an
> access. What about a write to disk? And what about a page being
> accessed from some random device (could hook into get_user_pages()?) Is
> getting written to swap an access? When a dirty pagecache page is
> written out by kswapd or direct reclaim?
>
> This also should be in the permanent documentation.
OK, will add.
>
> > To mark a page idle one should set the bit corresponding to the
> > page by writing to the file. A value written to the file is OR-ed with the
> > current bitmap value. Only user memory pages can be marked idle, for other
> > page types input is silently ignored. Writing to this file beyond max PFN
> > results in the ENXIO error. Only available when CONFIG_IDLE_PAGE_TRACKING is
> > set.
> >
> > This file can be used to estimate the amount of pages that are not
> > used by a particular workload as follows:
> >
> > 1. mark all pages of interest idle by setting corresponding bits in the
> > /proc/kpageidle bitmap
> > 2. wait until the workload accesses its working set
> > 3. read /proc/kpageidle and count the number of bits set
>
> Security implications. This interface could be used to learn about a
> sensitive application by poking data at it and then observing its
> memory access patterns. Perhaps this is why the proc files are
> root-only (whcih I assume is sufficient).
That's one point. Another point is that if we allow unprivileged users
to access it, they may interfere with the system-wide daemon doing the
regular scan and estimating the system wss.
> Some words here about the security side of things and the reasoning
> behind the chosen permissions would be good to have.
>
> > * /proc/kpagecgroup. This file contains a 64-bit inode number of the
> > memory cgroup each page is charged to, indexed by PFN.
>
> Actually "closest online ancestor". This also should be in the
> interface documentation.
Actually, the userspace knows nothing about online/offline cgroups,
because all cgroups used to be online and charge re-parenting was used
to forcibly empty a memcg on deletion. Anyways, I'll add a note.
>
> > Only available when CONFIG_MEMCG is set.
>
> CONFIG_MEMCG and CONFIG_IDLE_PAGE_TRACKING I assume?
No, it's present iff CONFIG_PROC_PAGE_MONITOR && CONFIG_MEMCG, because
it might be useful even w/o CONFIG_IDLE_PAGE_TRACKING, e.g. in order to
find out which memcg pages of a particular process are accounted to.
>
> >
> > This file can be used to find all pages (including unmapped file
> > pages) accounted to a particular cgroup. Using /proc/kpageidle, one
> > can then estimate the cgroup working set size.
> >
> > For an example of using these files for estimating the amount of unused
> > memory pages per each memory cgroup, please see the script attached
> > below.
>
> Why were these put in /proc anyway? Rather than under /sys/fs/cgroup
> somewhere? Presumably because /proc/kpageidle is useful in non-memcg
> setups.
Yes, one might use it for estimating active wss of a single process or
the whole system.
>
> > ---- PERFORMANCE EVALUATION ----
>
> "^___" means "end of changelog". Perhaps that should have been
> "^---\n" - unclear.
Sorry :-/
>
> > Documentation/vm/pagemap.txt | 22 ++-
>
> I think we'll need quite a lot more than this to fully describe the
> interface?
Agree, the documentation sucks :-( Will try to forge something more
thorough.
Thanks,
Vladimir
^ permalink raw reply
* Re: [PATCH -mm v9 7/8] proc: export idle flag via kpageflags
From: Vladimir Davydov @ 2015-07-22 16:25 UTC (permalink / raw)
To: Andrew Morton
Cc: Andres Lagar-Cavilla, Minchan Kim, Raghavendra K T,
Johannes Weiner, Michal Hocko, Greg Thelen, Michel Lespinasse,
David Rientjes, Pavel Emelyanov, Cyrill Gorcunov, Jonathan Corbet,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-doc-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
cgroups-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20150721163500.528bd39bbbc71abc3c8d429b-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>
On Tue, Jul 21, 2015 at 04:35:00PM -0700, Andrew Morton wrote:
> On Sun, 19 Jul 2015 15:31:16 +0300 Vladimir Davydov <vdavydov-bzQdu9zFT3WakBO8gow8eQ@public.gmane.org> wrote:
>
> > As noted by Minchan, a benefit of reading idle flag from
> > /proc/kpageflags is that one can easily filter dirty and/or unevictable
> > pages while estimating the size of unused memory.
> >
> > Note that idle flag read from /proc/kpageflags may be stale in case the
> > page was accessed via a PTE, because it would be too costly to iterate
> > over all page mappings on each /proc/kpageflags read to provide an
> > up-to-date value. To make sure the flag is up-to-date one has to read
> > /proc/kpageidle first.
>
> Is there any value in teaching the regular old page scanner to update
> these flags? If it's doing an rmap scan anyway...
I don't understand what you mean by "regular old page scanner". Could
you please elaborate?
Thanks,
Vladimir
^ permalink raw reply
* Re: [PATCH -mm v9 5/8] mmu-notifier: add clear_young callback
From: Vladimir Davydov @ 2015-07-22 16:33 UTC (permalink / raw)
To: Andrew Morton
Cc: Andres Lagar-Cavilla, Minchan Kim, Raghavendra K T,
Johannes Weiner, Michal Hocko, Greg Thelen, Michel Lespinasse,
David Rientjes, Pavel Emelyanov, Cyrill Gorcunov, Jonathan Corbet,
linux-api, linux-doc, linux-mm, cgroups, linux-kernel
In-Reply-To: <20150721085108.GA18673@esperanza>
Hi Andrew,
Would you mind merging this incremental patch to the original one? Or
should I better resubmit the whole series with all the fixes?
On Tue, Jul 21, 2015 at 11:51:08AM +0300, Vladimir Davydov wrote:
> On Mon, Jul 20, 2015 at 11:34:21AM -0700, Andres Lagar-Cavilla wrote:
> > On Sun, Jul 19, 2015 at 5:31 AM, Vladimir Davydov <vdavydov@parallels.com>
> [...]
> > > +static int kvm_mmu_notifier_clear_young(struct mmu_notifier *mn,
> > > + struct mm_struct *mm,
> > > + unsigned long start,
> > > + unsigned long end)
> > > +{
> > > + struct kvm *kvm = mmu_notifier_to_kvm(mn);
> > > + int young, idx;
> > > +
> > >
> > If you need to cut out another version please add comments as to the two
> > issues raised:
> > - This doesn't proactively flush TLBs -- not obvious if it should.
> > - This adversely affects performance in Pre_haswell Intel EPT.
>
> Oops, I stopped reading your e-mail in reply to the previous version of
> this patch as soon as I saw the Reviewed-by tag, so I missed your
> request for the comment, sorry about that.
>
> Here it goes (incremental):
> ---
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index ff4173ce6924..e69a5cb99571 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -397,6 +397,19 @@ static int kvm_mmu_notifier_clear_young(struct mmu_notifier *mn,
>
> idx = srcu_read_lock(&kvm->srcu);
> spin_lock(&kvm->mmu_lock);
> + /*
> + * Even though we do not flush TLB, this will still adversely
> + * affect performance on pre-Haswell Intel EPT, where there is
> + * no EPT Access Bit to clear so that we have to tear down EPT
> + * tables instead. If we find this unacceptable, we can always
> + * add a parameter to kvm_age_hva so that it effectively doesn't
> + * do anything on clear_young.
> + *
> + * Also note that currently we never issue secondary TLB flushes
> + * from clear_young, leaving this job up to the regular system
> + * cadence. If we find this inaccurate, we might come up with a
> + * more sophisticated heuristic later.
> + */
> young = kvm_age_hva(kvm, start, end);
> spin_unlock(&kvm->mmu_lock);
> srcu_read_unlock(&kvm->srcu, idx);
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCHv1 0/8] CGroup Namespaces
From: Vincent Batts @ 2015-07-22 18:10 UTC (permalink / raw)
To: Eric W. Biederman
Cc: Aditya Kali, linux-api-u79uwXL29TY76Z2rM5mHXA, Linux Containers,
serge.hallyn-GeWIH/nMZzLQT0dZR+AlfA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, luto-kltTT9wpgjJwATOyAt5JVQ,
tj-DgEjT+Ai2ygdnm+yROfE0A, cgroups-u79uwXL29TY76Z2rM5mHXA,
mingo-H+wXaHxf7aLQT0dZR+AlfA
In-Reply-To: <87k33wpsl3.fsf-JOvCrm2gF+uungPnsOpG7nhyD016LWXt@public.gmane.org>
Has there been further movement on CLONE_NEWCGROUP outside of this?
vb
On Sun, Oct 19, 2014 at 12:54 AM, Eric W. Biederman
<ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org> wrote:
> Aditya Kali <adityakali-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> writes:
>
>> Second take at the Cgroup Namespace patch-set.
>>
>> Major changes form RFC (V0):
>> 1. setns support for cgroupns
>> 2. 'mount -t cgroup cgroup <mntpt>' from inside a cgroupns now
>> mounts the cgroup hierarcy with cgroupns-root as the filesystem root.
>> 3. writes to cgroup files outside of cgroupns-root are not allowed
>> 4. visibility of /proc/<pid>/cgroup is further restricted by not showing
>> anything if the <pid> is in a sibling cgroupns and its cgroup falls outside
>> your cgroupns-root.
>>
>> More details in the writeup below.
>
> This definitely looks like the right direction to go, and something that
> in some form or another I had been asking for since cgroups were merged.
> So I am very glad to see this work moving forward.
>
> I had hoped that we might just be able to be clever with remounting
> cgroupfs but 2 things stand in the way.
> 1) /proc/<pid>/cgroups (but proc could capture that).
> 2) providing a hard guarnatee that tasks stay within a subset of the
> cgroup hierarchy.
>
> So I think this clearly meets the requirements for a new namespace.
>
> We need to have the discussion on chmod of files on cgroupfs. There is
> a notion that has floated around that only systemd or only root (with
> the appropriate capabilities) should be allowed to set resource limits
> in cgroupfs. In a practical reality that is nonsense. If an atribute
> is properly bound in it's hiearchy it should be safe to change.
>
> Not all attributes are properly bound to hierarchy and some are or at
> least were dangerous for anyone except root to set. So I suggest that a
> CFTYPE flag perhaps CFTYPE_UNPRIV be added for attributes that are safe
> to allow anyone to set, and require CFTYPE_UNPRIV be set before we chmod
> a cgroup attribute from root.
>
> That would be complimentary work, and not strictly tied the cgroup
> namespaces but unprivileged cgroup namespaces don't make much sense
> without that work.
>
> Eric
>
>> Background
>> Cgroups and Namespaces are used together to create “virtual”
>> containers that isolates the host environment from the processes
>> running in container. But since cgroups themselves are not
>> “virtualized”, the task is always able to see global cgroups view
>> through cgroupfs mount and via /proc/self/cgroup file.
>>
>> $ cat /proc/self/cgroup
>> 0:cpuset,cpu,cpuacct,memory,devices,freezer,hugetlb:/batchjobs/c_job_id1
>>
>> This exposure of cgroup names to the processes running inside a
>> container results in some problems:
>> (1) The container names are typically host-container-management-agent
>> (systemd, docker/libcontainer, etc.) data and leaking its name (or
>> leaking the hierarchy) reveals too much information about the host
>> system.
>> (2) It makes the container migration across machines (CRIU) more
>> difficult as the container names need to be unique across the
>> machines in the migration domain.
>> (3) It makes it difficult to run container management tools (like
>> docker/libcontainer, lmctfy, etc.) within virtual containers
>> without adding dependency on some state/agent present outside the
>> container.
>>
>> Note that the feature proposed here is completely different than the
>> “ns cgroup” feature which existed in the linux kernel until recently.
>> The ns cgroup also attempted to connect cgroups and namespaces by
>> creating a new cgroup every time a new namespace was created. It did
>> not solve any of the above mentioned problems and was later dropped
>> from the kernel. Incidentally though, it used the same config option
>> name CONFIG_CGROUP_NS as used in my prototype!
>>
>> Introducing CGroup Namespaces
>> With unified cgroup hierarchy
>> (Documentation/cgroups/unified-hierarchy.txt), the containers can now
>> have a much more coherent cgroup view and its easy to associate a
>> container with a single cgroup. This also allows us to virtualize the
>> cgroup view for tasks inside the container.
>>
>> The new CGroup Namespace allows a process to “unshare” its cgroup
>> hierarchy starting from the cgroup its currently in.
>> For Ex:
>> $ cat /proc/self/cgroup
>> 0:cpuset,cpu,cpuacct,memory,devices,freezer,hugetlb:/batchjobs/c_job_id1
>> $ ls -l /proc/self/ns/cgroup
>> lrwxrwxrwx 1 root root 0 2014-07-15 10:37 /proc/self/ns/cgroup -> cgroup:[4026531835]
>> $ ~/unshare -c # calls unshare(CLONE_NEWCGROUP) and exec’s /bin/bash
>> [ns]$ ls -l /proc/self/ns/cgroup
>> lrwxrwxrwx 1 root root 0 2014-07-15 10:35 /proc/self/ns/cgroup ->
>> cgroup:[4026532183]
>> # From within new cgroupns, process sees that its in the root cgroup
>> [ns]$ cat /proc/self/cgroup
>> 0:cpuset,cpu,cpuacct,memory,devices,freezer,hugetlb:/
>>
>> # From global cgroupns:
>> $ cat /proc/<pid>/cgroup
>> 0:cpuset,cpu,cpuacct,memory,devices,freezer,hugetlb:/batchjobs/c_job_id1
>>
>> # Unshare cgroupns along with userns and mountns
>> # Following calls unshare(CLONE_NEWCGROUP|CLONE_NEWUSER|CLONE_NEWNS), then
>> # sets up uid/gid map and exec’s /bin/bash
>> $ ~/unshare -c -u -m
>>
>> # Originally, we were in /batchjobs/c_job_id1 cgroup. Mount our own cgroup
>> # hierarchy.
>> [ns]$ mount -t cgroup cgroup /tmp/cgroup
>> [ns]$ ls -l /tmp/cgroup
>> total 0
>> -r--r--r-- 1 root root 0 2014-10-13 09:32 cgroup.controllers
>> -r--r--r-- 1 root root 0 2014-10-13 09:32 cgroup.populated
>> -rw-r--r-- 1 root root 0 2014-10-13 09:25 cgroup.procs
>> -rw-r--r-- 1 root root 0 2014-10-13 09:32 cgroup.subtree_control
>>
>> The cgroupns-root (/batchjobs/c_job_id1 in above example) becomes the
>> filesystem root for the namespace specific cgroupfs mount.
>>
>> The virtualization of /proc/self/cgroup file combined with restricting
>> the view of cgroup hierarchy by namespace-private cgroupfs mount
>> should provide a completely isolated cgroup view inside the container.
>>
>> In its current form, the cgroup namespaces patcheset provides following
>> behavior:
>>
>> (1) The “root” cgroup for a cgroup namespace is the cgroup in which
>> the process calling unshare is running.
>> For ex. if a process in /batchjobs/c_job_id1 cgroup calls unshare,
>> cgroup /batchjobs/c_job_id1 becomes the cgroupns-root.
>> For the init_cgroup_ns, this is the real root (“/”) cgroup
>> (identified in code as cgrp_dfl_root.cgrp).
>>
>> (2) The cgroupns-root cgroup does not change even if the namespace
>> creator process later moves to a different cgroup.
>> $ ~/unshare -c # unshare cgroupns in some cgroup
>> [ns]$ cat /proc/self/cgroup
>> 0:cpuset,cpu,cpuacct,memory,devices,freezer,hugetlb:/
>> [ns]$ mkdir sub_cgrp_1
>> [ns]$ echo 0 > sub_cgrp_1/cgroup.procs
>> [ns]$ cat /proc/self/cgroup
>> 0:cpuset,cpu,cpuacct,memory,devices,freezer,hugetlb:/sub_cgrp_1
>>
>> (3) Each process gets its CGROUPNS specific view of
>> /proc/<pid>/cgroup.
>> (a) Processes running inside the cgroup namespace will be able to see
>> cgroup paths (in /proc/self/cgroup) only inside their root cgroup
>> [ns]$ sleep 100000 & # From within unshared cgroupns
>> [1] 7353
>> [ns]$ echo 7353 > sub_cgrp_1/cgroup.procs
>> [ns]$ cat /proc/7353/cgroup
>> 0:cpuset,cpu,cpuacct,memory,devices,freezer,hugetlb:/sub_cgrp_1
>>
>> (b) From global cgroupns, the real cgroup path will be visible:
>> $ cat /proc/7353/cgroup
>> 0:cpuset,cpu,cpuacct,memory,devices,freezer,hugetlb:/batchjobs/c_job_id1/sub_cgrp_1
>>
>> (c) From a sibling cgroupns (cgroupns root-ed at a sibling cgroup), no cgroup
>> path will be visible:
>> # ns2's cgroupns-root is at '/batchjobs/c_job_id2'
>> [ns2]$ cat /proc/7353/cgroup
>> [ns2]$
>> This is same as when cgroup hierarchy is not mounted at all.
>> (In correct container setup though, it should not be possible to
>> access PIDs in another container in the first place.)
>>
>> (4) Processes inside a cgroupns are not allowed to move out of the
>> cgroupns-root. This is true even if a privileged process in global
>> cgroupns tries to move the process out of its cgroupns-root.
>>
>> # From global cgroupns
>> $ cat /proc/7353/cgroup
>> 0:cpuset,cpu,cpuacct,memory,devices,freezer,hugetlb:/batchjobs/c_job_id1/sub_cgrp_1
>> # cgroupns-root for 7353 is /batchjobs/c_job_id1
>> $ echo 7353 > batchjobs/c_job_id2/cgroup.procs
>> -bash: echo: write error: Operation not permitted
>>
>> (5) Setns to another cgroup namespace is allowed only when:
>> (a) process has CAP_SYS_ADMIN in its current userns
>> (b) process has CAP_SYS_ADMIN in the target cgroupns' userns
>> (c) the process's current cgroup is a descendant cgroupns-root of the
>> target namespace.
>> (d) the target cgroupns-root is descendant of current cgroupns-root..
>> The last check (d) prevents processes from escaping their cgroupns-root by
>> attaching to parent cgroupns. Thus, setns is allowed only when the process
>> is trying to restrict itself to a deeper cgroup hierarchy.
>>
>> (6) When some thread from a multi-threaded process unshares its
>> cgroup-namespace, the new cgroupns gets applied to the entire
>> process (all the threads). This should be OK since
>> unified-hierarchy only allows process-level containerization. So
>> all the threads in the process will have the same cgroup. And both
>> - changing cgroups and unsharing namespaces - are protected under
>> threadgroup_lock(task).
>>
>> (7) The cgroup namespace is alive as long as there is atleast 1
>> process inside it. When the last process exits, the cgroup
>> namespace is destroyed. The cgroupns-root and the actual cgroups
>> remain though.
>>
>> (8) 'mount -t cgroup cgroup <mntpt>' when called from within cgroupns mounts
>> the unified cgroup hierarchy with cgroupns-root as the filesystem root.
>> The process needs CAP_SYS_ADMIN in its userns and mntns. This allows the
>> container management tools to be run inside the containers transparently.
>>
>> Implementation
>> The current patch-set is based on top of Tejun Heo's cgroup tree (for-next
>> branch). Its fairly non-intrusive and provides above mentioned
>> features.
>>
>> Possible extensions of CGROUPNS:
>> (1) The Documentation/cgroups/unified-hierarchy.txt mentions use of
>> capabilities to restrict cgroups to administrative users. CGroup
>> namespaces could be of help here. With cgroup namespaces, it might
>> be possible to delegate administration of sub-cgroups under a
>> cgroupns-root to the cgroupns owner.
>
>
>
>
>> ---
>> fs/kernfs/dir.c | 53 +++++++++---
>> fs/kernfs/mount.c | 48 +++++++++++
>> fs/proc/namespaces.c | 3 +
>> include/linux/cgroup.h | 41 +++++++++-
>> include/linux/cgroup_namespace.h | 62 +++++++++++++++
>> include/linux/kernfs.h | 5 ++
>> include/linux/nsproxy.h | 2 +
>> include/linux/proc_ns.h | 4 +
>> include/uapi/linux/sched.h | 3 +-
>> init/Kconfig | 9 +++
>> kernel/Makefile | 1 +
>> kernel/cgroup.c | 139 ++++++++++++++++++++++++++------
>> kernel/cgroup_namespace.c | 168 +++++++++++++++++++++++++++++++++++++++
>> kernel/fork.c | 2 +-
>> kernel/nsproxy.c | 19 ++++-
>> 15 files changed, 518 insertions(+), 41 deletions(-)
>> create mode 100644 include/linux/cgroup_namespace.h
>> create mode 100644 kernel/cgroup_namespace.c
>>
>> [PATCHv1 1/8] kernfs: Add API to generate relative kernfs path
>> [PATCHv1 2/8] sched: new clone flag CLONE_NEWCGROUP for cgroup
>> [PATCHv1 3/8] cgroup: add function to get task's cgroup on default
>> [PATCHv1 4/8] cgroup: export cgroup_get() and cgroup_put()
>> [PATCHv1 5/8] cgroup: introduce cgroup namespaces
>> [PATCHv1 6/8] cgroup: restrict cgroup operations within task's cgroupns
>> [PATCHv1 7/8] cgroup: cgroup namespace setns support
>> [PATCHv1 8/8] cgroup: mount cgroupns-root when inside non-init cgroupns
>> _______________________________________________
>> Containers mailing list
>> Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
>> https://lists.linuxfoundation.org/mailman/listinfo/containers
> _______________________________________________
> Containers mailing list
> Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
> https://lists.linuxfoundation.org/mailman/listinfo/containers
^ permalink raw reply
* Re: [PATCH V4 4/6] mm: mlock: Introduce VM_LOCKONFAULT and add mlock flags to enable it
From: Eric B Munson @ 2015-07-22 18:43 UTC (permalink / raw)
To: Vlastimil Babka
Cc: Andrew Morton, Michal Hocko, Jonathan Corbet, linux-alpha,
linux-kernel, linux-mips, linux-parisc, linuxppc-dev, sparclinux,
linux-xtensa, dri-devel, linux-mm, linux-arch, linux-api
In-Reply-To: <55AF6A73.1080500@suse.cz>
[-- Attachment #1: Type: text/plain, Size: 5437 bytes --]
On Wed, 22 Jul 2015, Vlastimil Babka wrote:
> On 07/21/2015 09:59 PM, Eric B Munson wrote:
> >The cost of faulting in all memory to be locked can be very high when
> >working with large mappings. If only portions of the mapping will be
> >used this can incur a high penalty for locking.
> >
> >For the example of a large file, this is the usage pattern for a large
> >statical language model (probably applies to other statical or graphical
> >models as well). For the security example, any application transacting
> >in data that cannot be swapped out (credit card data, medical records,
> >etc).
> >
> >This patch introduces the ability to request that pages are not
> >pre-faulted, but are placed on the unevictable LRU when they are finally
> >faulted in. This can be done area at a time via the
> >mlock2(MLOCK_ONFAULT) or the mlockall(MCL_ONFAULT) system calls. These
> >calls can be undone via munlock2(MLOCK_ONFAULT) or
> >munlockall2(MCL_ONFAULT).
> >
> >Applying the VM_LOCKONFAULT flag to a mapping with pages that are
> >already present required the addition of a function in gup.c to pin all
> >pages which are present in an address range. It borrows heavily from
> >__mm_populate().
> >
> >To keep accounting checks out of the page fault path, users are billed
> >for the entire mapping lock as if MLOCK_LOCKED was used.
>
> Hi,
>
> I think you should include a complete description of which
> transitions for vma states and mlock2/munlock2 flags applied on them
> are valid and what they do. It will also help with the manpages.
> You explained some to Jon in the last thread, but I think there
> should be a canonical description in changelog (if not also
> Documentation, if mlock is covered there).
>
> For example the scenario Jon asked, what happens after a
> mlock2(MLOCK_ONFAULT) followed by mlock2(MLOCK_LOCKED), and that the
> answer is "nothing". Your promised code comment for
> apply_vma_flags() doesn't suffice IMHO (and I'm not sure it's there,
> anyway?).
I missed adding that comment to the code, will be there in V5 along with
the description in the changelog.
>
> But the more I think about the scenario and your new VM_LOCKONFAULT
> vma flag, it seems awkward to me. Why should munlocking at all care
> if the vma was mlocked with MLOCK_LOCKED or MLOCK_ONFAULT? In either
> case the result is that all pages currently populated are munlocked.
> So the flags for munlock2 should be unnecessary.
Say a user has a large area of interleaved MLOCK_LOCK and MLOCK_ONFAULT
mappings and they want to unlock only the ones with MLOCK_LOCK. With
the current implementation, this is possible in a single system call
that spans the entire region. With your suggestion, the user would have
to know what regions where locked with MLOCK_LOCK and call munlock() on
each of them. IMO, the way munlock2() works better mirrors the way
munlock() currently works when called on a large area of interleaved
locked and unlocked areas.
>
> I also think VM_LOCKONFAULT is unnecessary. VM_LOCKED should be
> enough - see how you had to handle the new flag in all places that
> had to handle the old flag? I think the information whether mlock
> was supposed to fault the whole vma is obsolete at the moment mlock
> returns. VM_LOCKED should be enough for both modes, and the flag to
> mlock2 could just control whether the pre-faulting is done.
>
> So what should be IMHO enough:
> - munlock can stay without flags
> - mlock2 has only one new flag MLOCK_ONFAULT. If specified,
> pre-faulting is not done, just set VM_LOCKED and mlock pages already
> present.
> - same with mmap(MAP_LOCKONFAULT) (need to define what happens when
> both MAP_LOCKED and MAP_LOCKONFAULT are specified).
>
> Now mlockall(MCL_FUTURE) muddles the situation in that it stores the
> information for future VMA's in current->mm->def_flags, and this
> def_flags would need to distinguish VM_LOCKED with population and
> without. But that could be still solvable without introducing a new
> vma flag everywhere.
With you right up until that last paragraph. I have been staring at
this a while and I cannot come up a way to handle the
mlockall(MCL_ONFAULT) without introducing a new vm flag. It doesn't
have to be VM_LOCKONFAULT, we could use the model that Michal Hocko
suggested with something like VM_FAULTPOPULATE. However, we can't
really use this flag anywhere except the mlock code becuase we have to
be able to distinguish a caller that wants to use MLOCK_LOCK with
whatever control VM_FAULTPOPULATE might grant outside of mlock and a
caller that wants MLOCK_ONFAULT. That was a long way of saying we need
an extra vma flag regardless. However, if that flag only controls if
mlock pre-populates it would work and it would do away with most of the
places I had to touch to handle VM_LOCKONFAULT properly.
I picked VM_LOCKONFAULT because it is explicit about what it is for and
there is little risk of someone coming along in 5 years and saying "why
not overload this flag to do this other thing completely unrelated to
mlock?". A flag for controling speculative population is more likely to
be overloaded outside of mlock().
If you have a sane way of handling mlockall(MCL_ONFAULT) without a new
VMA flag, I am happy to give it a try, but I haven't been able to come
up with one that doesn't have its own gremlins.
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH v4 1/5] nohz_full: add support for "cpu_isolated" mode
From: Paul E. McKenney @ 2015-07-22 19:28 UTC (permalink / raw)
To: Christoph Lameter
Cc: Andy Lutomirski, Chris Metcalf, Gilad Ben Yossef, Steven Rostedt,
Ingo Molnar, Peter Zijlstra, Andrew Morton, Rik van Riel,
Tejun Heo, Frederic Weisbecker, Thomas Gleixner, Viresh Kumar,
linux-doc-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Linux API,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
In-Reply-To: <alpine.DEB.2.11.1507220856030.17411-wcBtFHqTun5QOdAKl3ChDw@public.gmane.org>
On Wed, Jul 22, 2015 at 08:57:45AM -0500, Christoph Lameter wrote:
> On Tue, 21 Jul 2015, Paul E. McKenney wrote:
>
> > > > In the current version, there's also the vmstat_update() that
> > > > may schedule delayed work and interrupt the core again
> > > > shortly before realizing that there are no more counter updates
> > > > happening, at which point it quiesces. Currently we handle
> > > > this in cpu_isolated mode simply by spinning and waiting for
> > > > the timer interrupts to complete.
> > >
> > > Perhaps we should fix that?
> >
> > Didn't Christoph Lameter fix this? Or is this an additional problem?
>
> Well the vmstat update must realize first that there are no outstanding
> updates before switching itself off. So typically there is one extra tick.
> But we could add another function that will simply fold the differential
> immediately and turn the kworker task in the expectation that the
> processor will stay quiet.
Got it, thank you!
Thanx, Paul
^ permalink raw reply
* Re: [PATCH -mm v9 7/8] proc: export idle flag via kpageflags
From: Andrew Morton @ 2015-07-22 19:44 UTC (permalink / raw)
To: Vladimir Davydov
Cc: Andres Lagar-Cavilla, Minchan Kim, Raghavendra K T,
Johannes Weiner, Michal Hocko, Greg Thelen, Michel Lespinasse,
David Rientjes, Pavel Emelyanov, Cyrill Gorcunov, Jonathan Corbet,
linux-api-u79uwXL29TY76Z2rM5mHXA,
linux-doc-u79uwXL29TY76Z2rM5mHXA, linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
cgroups-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20150722162528.GN23374@esperanza>
On Wed, 22 Jul 2015 19:25:28 +0300 Vladimir Davydov <vdavydov-bzQdu9zFT3WakBO8gow8eQ@public.gmane.org> wrote:
> On Tue, Jul 21, 2015 at 04:35:00PM -0700, Andrew Morton wrote:
> > On Sun, 19 Jul 2015 15:31:16 +0300 Vladimir Davydov <vdavydov-bzQdu9zFT3WakBO8gow8eQ@public.gmane.org> wrote:
> >
> > > As noted by Minchan, a benefit of reading idle flag from
> > > /proc/kpageflags is that one can easily filter dirty and/or unevictable
> > > pages while estimating the size of unused memory.
> > >
> > > Note that idle flag read from /proc/kpageflags may be stale in case the
> > > page was accessed via a PTE, because it would be too costly to iterate
> > > over all page mappings on each /proc/kpageflags read to provide an
> > > up-to-date value. To make sure the flag is up-to-date one has to read
> > > /proc/kpageidle first.
> >
> > Is there any value in teaching the regular old page scanner to update
> > these flags? If it's doing an rmap scan anyway...
>
> I don't understand what you mean by "regular old page scanner". Could
> you please elaborate?
Whenever kswapd or direct reclaim perform an rmap scan, take that as an
opportunity to also update PageIdle().
^ permalink raw reply
* [RFC v3 0/1] userio (originally ps2emu) - virtual serio device module
From: Stephen Chandler Paul @ 2015-07-22 19:52 UTC (permalink / raw)
To: Dmitry Torokhov, Andrew Morton, Mauro Carvalho Chehab, Greg KH,
Arnd Bergmann, Joe Perches, Jiri Slaby, Vishnu Patekar,
Sebastian Ott, linux-doc, linux-kernel, linux-input, linux-api
Cc: Benjamin Tissoires, Hans de Goede
Here's the latest revision of the userio patch with all of the suggested edits
made, along with a couple of extras I picked up on the way.
Cheers,
Stephen Chandler Paul
Stephen Chandler Paul (1):
Input: Add userio module
Documentation/input/userio.txt | 70 +++++++++++
MAINTAINERS | 6 +
drivers/input/serio/Kconfig | 11 ++
drivers/input/serio/Makefile | 1 +
drivers/input/serio/userio.c | 261 +++++++++++++++++++++++++++++++++++++++++
include/uapi/linux/userio.h | 42 +++++++
6 files changed, 391 insertions(+)
create mode 100644 Documentation/input/userio.txt
create mode 100644 drivers/input/serio/userio.c
create mode 100644 include/uapi/linux/userio.h
--
2.4.3
^ permalink raw reply
* [RFC v3 1/1] Input: Add userio module
From: Stephen Chandler Paul @ 2015-07-22 19:52 UTC (permalink / raw)
To: Dmitry Torokhov, Andrew Morton, Mauro Carvalho Chehab, Greg KH,
Arnd Bergmann, Joe Perches, Jiri Slaby, Vishnu Patekar,
Sebastian Ott, linux-doc, linux-kernel, linux-input, linux-api
Cc: Benjamin Tissoires, Hans de Goede
In-Reply-To: <1437594773-22274-1-git-send-email-cpaul@redhat.com>
Debugging input devices, specifically laptop touchpads, can be tricky
without having the physical device handy. Here we try to remedy that
with userio. This module allows an application to connect to a character
device provided by the kernel, and emulate any serio device. In
combination with userspace programs that can record PS/2 devices and
replay them through the /dev/userio device, this allows developers to
debug driver issues on the PS/2 level with devices simply by requesting
a recording from the user experiencing the issue without having to have
the physical hardware in front of them.
Signed-off-by: Stephen Chandler Paul <cpaul@redhat.com>
---
Documentation/input/userio.txt | 70 +++++++++++
MAINTAINERS | 6 +
drivers/input/serio/Kconfig | 11 ++
drivers/input/serio/Makefile | 1 +
drivers/input/serio/userio.c | 261 +++++++++++++++++++++++++++++++++++++++++
include/uapi/linux/userio.h | 42 +++++++
6 files changed, 391 insertions(+)
create mode 100644 Documentation/input/userio.txt
create mode 100644 drivers/input/serio/userio.c
create mode 100644 include/uapi/linux/userio.h
diff --git a/Documentation/input/userio.txt b/Documentation/input/userio.txt
new file mode 100644
index 0000000..0880c0f
--- /dev/null
+++ b/Documentation/input/userio.txt
@@ -0,0 +1,70 @@
+ The userio Protocol
+ (c) 2015 Stephen Chandler Paul <thatslyude@gmail.com>
+ Sponsored by Red Hat
+--------------------------------------------------------------------------------
+
+1. Introduction
+~~~~~~~~~~~~~~~
+ This module is intended to try to make the lives of input driver developers
+easier by allowing them to test various serio devices (mainly the various
+touchpads found on laptops) without having to have the physical device in front
+of them. userio accomplishes this by allowing any privileged userspace program
+to directly interact with the kernel's serio driver and control a virtual serio
+port from there.
+
+2. Usage overview
+~~~~~~~~~~~~~~~~~
+ In order to interact with the userio kernel module, one simply opens the
+/dev/userio character device in their applications. Commands are sent to the
+kernel module by writing to the device, and any data received from the serio
+driver is read as-is from the /dev/userio device. All of the structures and
+macros you need to interact with the device are defined in <linux/userio.h> and
+<linux/serio.h>.
+
+3. Command Structure
+~~~~~~~~~~~~~~~~~~~~
+ The struct used for sending commands to /dev/userio is as follows:
+
+ struct userio_cmd {
+ __u8 type;
+ __u8 data;
+ };
+
+ "type" describes the type of command that is being sent. This can be any one
+of the USERIO_CMD macros defined in <linux/userio.h>. "data" is the argument
+that goes along with the command. In the event that the command doesn't have an
+argument, this field can be left untouched and will be ignored by the kernel.
+Each command should be sent by writing the struct directly to the character
+device. In the event that the command you send is invalid, an error will be
+returned by the character device and a more descriptive error will be printed
+to the kernel log. Only one command can be sent at a time, any additional data
+written to the character device after the initial command will be ignored.
+ To close the virtual serio port, just close /dev/userio.
+
+4. Commands
+~~~~~~~~~~~
+
+4.1 USERIO_CMD_REGISTER
+~~~~~~~~~~~~~~~~~~~~~~~
+ Registers the port with the serio driver and begins transmitting data back and
+forth. Registration can only be performed once a port type is set with
+USERIO_CMD_SET_PORT_TYPE. Has no argument.
+
+4.2 USERIO_CMD_SET_PORT_TYPE
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ Sets the type of port we're emulating, where "data" is the port type being
+set. Can be any of the macros from <linux/serio.h>. For example: SERIO_8042
+would set the port type to be a normal PS/2 port.
+
+4.3 USERIO_CMD_SEND_INTERRUPT
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ Sends an interrupt through the virtual serio port to the serio driver, where
+"data" is the interrupt data being sent.
+
+5. Userspace tools
+~~~~~~~~~~~~~~~~~~
+ The userio userspace tools are able to record PS/2 devices using some of the
+debugging information from i8042, and play back the devices on /dev/userio. The
+latest version of these tools can be found at:
+
+ https://github.com/Lyude/ps2emu
diff --git a/MAINTAINERS b/MAINTAINERS
index a226416..68a0977 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -10877,6 +10877,12 @@ S: Maintained
F: drivers/media/v4l2-core/videobuf2-*
F: include/media/videobuf2-*
+VIRTUAL PS/2 DEVICE DRIVER
+M: Stephen Chandler Paul <thatslyude@gmail.com>
+S: Maintained
+F: drivers/input/serio/ps2emu.c
+F: include/uapi/linux/ps2emu.h
+
VIRTIO CONSOLE DRIVER
M: Amit Shah <amit.shah@redhat.com>
L: virtualization@lists.linux-foundation.org
diff --git a/drivers/input/serio/Kconfig b/drivers/input/serio/Kconfig
index 200841b..22b8b58 100644
--- a/drivers/input/serio/Kconfig
+++ b/drivers/input/serio/Kconfig
@@ -292,4 +292,15 @@ config SERIO_SUN4I_PS2
To compile this driver as a module, choose M here: the
module will be called sun4i-ps2.
+config USERIO
+ tristate "Virtual serio device support"
+ help
+ Say Y here if you want to emulate serio devices using the userio
+ kernel module.
+
+ To compile this driver as a module, choose M here: the module will be
+ called userio.
+
+ If you are unsure, say N.
+
endif
diff --git a/drivers/input/serio/Makefile b/drivers/input/serio/Makefile
index c600089..2374ef9 100644
--- a/drivers/input/serio/Makefile
+++ b/drivers/input/serio/Makefile
@@ -30,3 +30,4 @@ obj-$(CONFIG_SERIO_APBPS2) += apbps2.o
obj-$(CONFIG_SERIO_OLPC_APSP) += olpc_apsp.o
obj-$(CONFIG_HYPERV_KEYBOARD) += hyperv-keyboard.o
obj-$(CONFIG_SERIO_SUN4I_PS2) += sun4i-ps2.o
+obj-$(CONFIG_USERIO) += userio.o
diff --git a/drivers/input/serio/userio.c b/drivers/input/serio/userio.c
new file mode 100644
index 0000000..578b107
--- /dev/null
+++ b/drivers/input/serio/userio.c
@@ -0,0 +1,261 @@
+/*
+ * userio kernel serio device emulation module
+ * Copyright (C) 2015 Red Hat
+ * Copyright (C) 2015 Stephen Chandler Paul <thatslyude@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU Lesser General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ * FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more
+ * details.
+ */
+#include <linux/circ_buf.h>
+#include <linux/mutex.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/kernel.h>
+#include <linux/serio.h>
+#include <linux/slab.h>
+#include <linux/fs.h>
+#include <linux/miscdevice.h>
+#include <linux/sched.h>
+#include <linux/poll.h>
+#include <linux/spinlock.h>
+#include <uapi/linux/userio.h>
+
+#define USERIO_NAME "userio"
+#define USERIO_BUFSIZE 16
+
+static const struct file_operations userio_fops;
+static struct miscdevice userio_misc;
+
+struct userio_device {
+ struct serio *serio;
+
+ bool running;
+
+ u8 head;
+ u8 tail;
+
+ /*
+ * Since we need to copy from userspace when modifying the tail and we
+ * don't want to lock up the serio driver during these operations, we
+ * use two different locks
+ */
+ spinlock_t head_lock;
+ struct mutex tail_lock;
+
+ unsigned char buf[USERIO_BUFSIZE];
+
+ wait_queue_head_t waitq;
+};
+
+/**
+ * userio_device_write - Write data from serio to a userio device in userspace
+ * @id: The serio port for the userio device
+ * @val: The data to write to the device
+ */
+static int userio_device_write(struct serio *id, unsigned char val)
+{
+ struct userio_device *userio = id->port_data;
+ unsigned long flags;
+
+ if (!userio)
+ return -1;
+
+ spin_lock_irqsave(&userio->head_lock, flags);
+
+ userio->buf[userio->head] = val;
+ userio->head = (userio->head + 1) % USERIO_BUFSIZE;
+
+ spin_unlock_irqrestore(&userio->head_lock, flags);
+
+ if (userio->head == userio->tail)
+ dev_warn(userio_misc.this_device,
+ "Buffer overflowed, userio client isn't keeping up");
+
+ wake_up_interruptible(&userio->waitq);
+
+ return 0;
+}
+
+static int userio_char_open(struct inode *inode, struct file *file)
+{
+ struct userio_device *userio;
+
+ userio = devm_kzalloc(userio_misc.this_device,
+ sizeof(struct userio_device), GFP_KERNEL);
+ if (!userio)
+ return -ENOMEM;
+
+ spin_lock_init(&userio->head_lock);
+ mutex_init(&userio->tail_lock);
+ init_waitqueue_head(&userio->waitq);
+
+ userio->serio = kzalloc(sizeof(struct serio), GFP_KERNEL);
+ if (!userio->serio) {
+ devm_kfree(userio_misc.this_device, userio);
+ return -ENOMEM;
+ }
+
+ userio->serio->write = userio_device_write;
+ userio->serio->port_data = userio;
+
+ file->private_data = userio;
+
+ return 0;
+}
+
+static int userio_char_release(struct inode *inode, struct file *file)
+{
+ struct userio_device *userio = file->private_data;
+
+ if (userio->serio) {
+ userio->serio->port_data = NULL;
+
+ if (userio->running)
+ serio_unregister_port(userio->serio);
+ else
+ kfree(userio->serio);
+ }
+
+ devm_kfree(userio_misc.this_device, userio);
+
+ return 0;
+}
+
+static ssize_t userio_char_read(struct file *file, char __user *buffer,
+ size_t count, loff_t *ppos)
+{
+ struct userio_device *userio = file->private_data;
+ int ret;
+ size_t nonwrap_len, copylen;
+
+ if (!count)
+ return 0;
+
+ if (file->f_flags & O_NONBLOCK && userio->head == userio->tail)
+ return -EAGAIN;
+ else {
+ ret = wait_event_interruptible(userio->waitq,
+ userio->head != userio->tail);
+
+ if (ret)
+ return ret;
+ }
+
+ mutex_lock(&userio->tail_lock);
+
+ nonwrap_len = CIRC_CNT_TO_END(userio->head, userio->tail,
+ USERIO_BUFSIZE);
+ copylen = min(nonwrap_len, count);
+
+ if (copy_to_user(buffer, &userio->buf[userio->tail], copylen)) {
+ mutex_unlock(&userio->tail_lock);
+ return -EFAULT;
+ }
+
+ userio->tail = (userio->tail + copylen) % USERIO_BUFSIZE;
+
+ mutex_unlock(&userio->tail_lock);
+
+ return copylen;
+}
+
+static ssize_t userio_char_write(struct file *file, const char __user *buffer,
+ size_t count, loff_t *ppos)
+{
+ struct userio_device *userio = file->private_data;
+ struct userio_cmd cmd;
+
+ if (count < sizeof(cmd))
+ return -EINVAL;
+
+ if (copy_from_user(&cmd, buffer, sizeof(cmd)))
+ return -EFAULT;
+
+ switch (cmd.type) {
+ case USERIO_CMD_REGISTER:
+ if (!userio->serio->id.type) {
+ dev_warn(userio_misc.this_device,
+ "No port type given on /dev/userio\n");
+
+ return -EINVAL;
+ }
+ if (userio->running) {
+ dev_warn(userio_misc.this_device,
+ "Begin command sent, but we're already running\n");
+
+ return -EINVAL;
+ }
+
+ userio->running = true;
+ serio_register_port(userio->serio);
+ break;
+
+ case USERIO_CMD_SET_PORT_TYPE:
+ if (userio->running) {
+ dev_warn(userio_misc.this_device,
+ "Can't change port type on an already running userio instance\n");
+
+ return -EINVAL;
+ }
+
+ userio->serio->id.type = cmd.data;
+ break;
+
+ case USERIO_CMD_SEND_INTERRUPT:
+ if (!userio->running) {
+ dev_warn(userio_misc.this_device,
+ "The device must be registered before sending interrupts\n");
+
+ return -EINVAL;
+ }
+
+ serio_interrupt(userio->serio, cmd.data, 0);
+ break;
+
+ default:
+ return -EINVAL;
+ }
+
+ return sizeof(cmd);
+}
+
+static unsigned int userio_char_poll(struct file *file, poll_table *wait)
+{
+ struct userio_device *userio = file->private_data;
+
+ poll_wait(file, &userio->waitq, wait);
+
+ if (userio->head != userio->tail)
+ return POLLIN | POLLRDNORM;
+
+ return 0;
+}
+
+static const struct file_operations userio_fops = {
+ .owner = THIS_MODULE,
+ .open = userio_char_open,
+ .release = userio_char_release,
+ .read = userio_char_read,
+ .write = userio_char_write,
+ .poll = userio_char_poll,
+ .llseek = no_llseek,
+};
+
+static struct miscdevice userio_misc = {
+ .fops = &userio_fops,
+ .minor = MISC_DYNAMIC_MINOR,
+ .name = USERIO_NAME,
+};
+
+MODULE_AUTHOR("Stephen Chandler Paul <thatslyude@gmail.com>");
+MODULE_DESCRIPTION("userio");
+MODULE_LICENSE("GPL");
+
+module_driver(userio_misc, misc_register, misc_deregister);
diff --git a/include/uapi/linux/userio.h b/include/uapi/linux/userio.h
new file mode 100644
index 0000000..da0a3d6
--- /dev/null
+++ b/include/uapi/linux/userio.h
@@ -0,0 +1,42 @@
+/*
+ * userio.h
+ * Copyright (C) 2015 Red Hat
+ * Copyright (C) 2015 Lyude (Stephen Chandler Paul) <cpaul@redhat.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU Lesser General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ * FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more
+ * details.
+ *
+ * This is the public header used for user-space communication with the userio
+ * driver. __attribute__((__packed__)) is used for all structs to keep ABI
+ * compatibility between all architectures.
+ */
+
+#ifndef _USERIO_H
+#define _USERIO_H
+
+#include <linux/types.h>
+
+#define USERIO_CMD_REGISTER 0
+#define USERIO_CMD_SET_PORT_TYPE 1
+#define USERIO_CMD_SEND_INTERRUPT 2
+
+/*
+ * userio Commands
+ * All commands sent to /dev/userio are encoded using this structure. The type
+ * field should contain a USERIO_CMD* value that indicates what kind of command
+ * is being sent to userio. The data field should contain the accompanying
+ * argument for the command, if there is one.
+ */
+struct userio_cmd {
+ __u8 type;
+ __u8 data;
+} __attribute__((__packed__));
+
+#endif /* !_USERIO_H */
--
2.4.3
^ permalink raw reply related
* Re: [PATCH v4 1/5] nohz_full: add support for "cpu_isolated" mode
From: Christoph Lameter @ 2015-07-22 20:02 UTC (permalink / raw)
To: Paul E. McKenney
Cc: Andy Lutomirski, Chris Metcalf, Gilad Ben Yossef, Steven Rostedt,
Ingo Molnar, Peter Zijlstra, Andrew Morton, Rik van Riel,
Tejun Heo, Frederic Weisbecker, Thomas Gleixner, Viresh Kumar,
linux-doc@vger.kernel.org, Linux API,
linux-kernel@vger.kernel.org
In-Reply-To: <20150722192834.GA32511@linux.vnet.ibm.com>
On Wed, 22 Jul 2015, Paul E. McKenney wrote:
> > > Didn't Christoph Lameter fix this? Or is this an additional problem?
> >
> > Well the vmstat update must realize first that there are no outstanding
> > updates before switching itself off. So typically there is one extra tick.
> > But we could add another function that will simply fold the differential
> > immediately and turn the kworker task in the expectation that the
> > processor will stay quiet.
>
> Got it, thank you!
>
> Thanx, Paul
Ok here is a function that quiets down the vmstat kworkers.
Subject: vmstat: provide a function to quiet down the diff processing
quiet_vmstat() can be called in anticipation of a OS "quiet" period
where no tick processing should be triggered. quiet_vmstat() will fold
all pending differentials into the global counters and disable the
vmstat_worker processing.
Note that the shepherd thread will continue scanning the differentials
from another processor and will reenable the vmstat workers if it
detects any changes.
Signed-off-by: Christoph Lameter <cl@linux.com>
Index: linux/mm/vmstat.c
===================================================================
--- linux.orig/mm/vmstat.c
+++ linux/mm/vmstat.c
@@ -1394,6 +1394,20 @@ static void vmstat_update(struct work_st
}
/*
+ * Switch off vmstat processing and then fold all the remaining differentials
+ * until the diffs stay at zero. The function is used by NOHZ and can only be
+ * invoked when tick processing is not active.
+ */
+void quiet_vmstat(void)
+{
+ do {
+ if (!cpumask_test_and_set_cpu(smp_processor_id(), cpu_stat_off))
+ cancel_delayed_work(this_cpu_ptr(&vmstat_work));
+
+ } while (refresh_cpu_vm_stats());
+}
+
+/*
* Check if the diffs for a certain cpu indicate that
* an update is needed.
*/
Index: linux/include/linux/vmstat.h
===================================================================
--- linux.orig/include/linux/vmstat.h
+++ linux/include/linux/vmstat.h
@@ -211,6 +211,7 @@ extern void __inc_zone_state(struct zone
extern void dec_zone_state(struct zone *, enum zone_stat_item);
extern void __dec_zone_state(struct zone *, enum zone_stat_item);
+void quiet_vmstat(void);
void cpu_vm_stats_fold(int cpu);
void refresh_zone_stat_thresholds(void);
@@ -272,6 +273,7 @@ static inline void __dec_zone_page_state
static inline void refresh_cpu_vm_stats(int cpu) { }
static inline void refresh_zone_stat_thresholds(void) { }
static inline void cpu_vm_stats_fold(int cpu) { }
+static inline void quiet_vmstat(void) { }
static inline void drain_zonestat(struct zone *zone,
struct per_cpu_pageset *pset) { }
^ permalink raw reply
* Re: [RFC v3 1/1] Input: Add userio module
From: Stephen Chandler Paul @ 2015-07-22 20:16 UTC (permalink / raw)
To: Dmitry Torokhov, Andrew Morton, Mauro Carvalho Chehab, Greg KH,
Arnd Bergmann, Joe Perches, Jiri Slaby, Vishnu Patekar,
Sebastian Ott, linux-doc, linux-kernel, linux-input, linux-api
Cc: Benjamin Tissoires, Hans de Goede
In-Reply-To: <1437594773-22274-2-git-send-email-cpaul@redhat.com>
Forgot to add the changes I made onto here, whoops!
* Allocate the serio struct separately, we don't have control over
when it's freed
* Use locking when working with the head or when working with the
tail. Use two seperate locks so that we only risk making the
userspace threads wait, not the kernel threads
* Use modulus operations when incrementing the head and the tail
* Check for read()s with length 0, return immediately if we find one
* When doing ps2emu_device_write(), check to make sure that the ps2emu
struct hasn't been set for destruction (being set to NULL), if it
has just abort the write
* Don't assign ps2emu to NULL, kzalloc will set it to NULL if
allocation fails
* Get rid of some ugly whitespace
* Remove an unused include header
* Allow any port types to be used
* Rename the module from ps2emu to userio
* Change documentation in Kconfig to say "using the userio kernel
module" as opposed to "using the userio tools.", since it's possible
there may be tools other then the ps2emu tools used with userio in
the future.
* Mention the requirement for <linux/serio.h>
On Wed, 2015-07-22 at 15:52 -0400, Stephen Chandler Paul wrote:
> Debugging input devices, specifically laptop touchpads, can be tricky
> without having the physical device handy. Here we try to remedy that
> with userio. This module allows an application to connect to a
> character
> device provided by the kernel, and emulate any serio device. In
> combination with userspace programs that can record PS/2 devices and
> replay them through the /dev/userio device, this allows developers to
> debug driver issues on the PS/2 level with devices simply by
> requesting
> a recording from the user experiencing the issue without having to
> have
> the physical hardware in front of them.
>
> Signed-off-by: Stephen Chandler Paul <cpaul@redhat.com>
> ---
> Documentation/input/userio.txt | 70 +++++++++++
> MAINTAINERS | 6 +
> drivers/input/serio/Kconfig | 11 ++
> drivers/input/serio/Makefile | 1 +
> drivers/input/serio/userio.c | 261
> +++++++++++++++++++++++++++++++++++++++++
> include/uapi/linux/userio.h | 42 +++++++
> 6 files changed, 391 insertions(+)
> create mode 100644 Documentation/input/userio.txt
> create mode 100644 drivers/input/serio/userio.c
> create mode 100644 include/uapi/linux/userio.h
>
> diff --git a/Documentation/input/userio.txt
> b/Documentation/input/userio.txt
> new file mode 100644
> index 0000000..0880c0f
> --- /dev/null
> +++ b/Documentation/input/userio.txt
> @@ -0,0 +1,70 @@
> + The userio Protocol
> + (c) 2015 Stephen Chandler Paul <thatslyude@gmail.com>
> + Sponsored by Red Hat
> +--------------------------------------------------------------------
> ------------
> +
> +1. Introduction
> +~~~~~~~~~~~~~~~
> + This module is intended to try to make the lives of input driver
> developers
> +easier by allowing them to test various serio devices (mainly the
> various
> +touchpads found on laptops) without having to have the physical
> device in front
> +of them. userio accomplishes this by allowing any privileged
> userspace program
> +to directly interact with the kernel's serio driver and control a
> virtual serio
> +port from there.
> +
> +2. Usage overview
> +~~~~~~~~~~~~~~~~~
> + In order to interact with the userio kernel module, one simply
> opens the
> +/dev/userio character device in their applications. Commands are
> sent to the
> +kernel module by writing to the device, and any data received from
> the serio
> +driver is read as-is from the /dev/userio device. All of the
> structures and
> +macros you need to interact with the device are defined in
> <linux/userio.h> and
> +<linux/serio.h>.
> +
> +3. Command Structure
> +~~~~~~~~~~~~~~~~~~~~
> + The struct used for sending commands to /dev/userio is as follows:
> +
> + struct userio_cmd {
> + __u8 type;
> + __u8 data;
> + };
> +
> + "type" describes the type of command that is being sent. This can
> be any one
> +of the USERIO_CMD macros defined in <linux/userio.h>. "data" is the
> argument
> +that goes along with the command. In the event that the command
> doesn't have an
> +argument, this field can be left untouched and will be ignored by
> the kernel.
> +Each command should be sent by writing the struct directly to the
> character
> +device. In the event that the command you send is invalid, an error
> will be
> +returned by the character device and a more descriptive error will
> be printed
> +to the kernel log. Only one command can be sent at a time, any
> additional data
> +written to the character device after the initial command will be
> ignored.
> + To close the virtual serio port, just close /dev/userio.
> +
> +4. Commands
> +~~~~~~~~~~~
> +
> +4.1 USERIO_CMD_REGISTER
> +~~~~~~~~~~~~~~~~~~~~~~~
> + Registers the port with the serio driver and begins transmitting
> data back and
> +forth. Registration can only be performed once a port type is set
> with
> +USERIO_CMD_SET_PORT_TYPE. Has no argument.
> +
> +4.2 USERIO_CMD_SET_PORT_TYPE
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> + Sets the type of port we're emulating, where "data" is the port
> type being
> +set. Can be any of the macros from <linux/serio.h>. For example:
> SERIO_8042
> +would set the port type to be a normal PS/2 port.
> +
> +4.3 USERIO_CMD_SEND_INTERRUPT
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> + Sends an interrupt through the virtual serio port to the serio
> driver, where
> +"data" is the interrupt data being sent.
> +
> +5. Userspace tools
> +~~~~~~~~~~~~~~~~~~
> + The userio userspace tools are able to record PS/2 devices using
> some of the
> +debugging information from i8042, and play back the devices on
> /dev/userio. The
> +latest version of these tools can be found at:
> +
> + https://github.com/Lyude/ps2emu
> diff --git a/MAINTAINERS b/MAINTAINERS
> index a226416..68a0977 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -10877,6 +10877,12 @@ S: Maintained
> F: drivers/media/v4l2-core/videobuf2-*
> F: include/media/videobuf2-*
>
> +VIRTUAL PS/2 DEVICE DRIVER
> +M: Stephen Chandler Paul <thatslyude@gmail.com>
> +S: Maintained
> +F: drivers/input/serio/ps2emu.c
> +F: include/uapi/linux/ps2emu.h
> +
> VIRTIO CONSOLE DRIVER
> M: Amit Shah <amit.shah@redhat.com>
> L: virtualization@lists.linux-foundation.org
> diff --git a/drivers/input/serio/Kconfig
> b/drivers/input/serio/Kconfig
> index 200841b..22b8b58 100644
> --- a/drivers/input/serio/Kconfig
> +++ b/drivers/input/serio/Kconfig
> @@ -292,4 +292,15 @@ config SERIO_SUN4I_PS2
> To compile this driver as a module, choose M here: the
> module will be called sun4i-ps2.
>
> +config USERIO
> + tristate "Virtual serio device support"
> + help
> + Say Y here if you want to emulate serio devices using the
> userio
> + kernel module.
> +
> + To compile this driver as a module, choose M here: the
> module will be
> + called userio.
> +
> + If you are unsure, say N.
> +
> endif
> diff --git a/drivers/input/serio/Makefile
> b/drivers/input/serio/Makefile
> index c600089..2374ef9 100644
> --- a/drivers/input/serio/Makefile
> +++ b/drivers/input/serio/Makefile
> @@ -30,3 +30,4 @@ obj-$(CONFIG_SERIO_APBPS2) += apbps2.o
> obj-$(CONFIG_SERIO_OLPC_APSP) += olpc_apsp.o
> obj-$(CONFIG_HYPERV_KEYBOARD) += hyperv-keyboard.o
> obj-$(CONFIG_SERIO_SUN4I_PS2) += sun4i-ps2.o
> +obj-$(CONFIG_USERIO) += userio.o
> diff --git a/drivers/input/serio/userio.c
> b/drivers/input/serio/userio.c
> new file mode 100644
> index 0000000..578b107
> --- /dev/null
> +++ b/drivers/input/serio/userio.c
> @@ -0,0 +1,261 @@
> +/*
> + * userio kernel serio device emulation module
> + * Copyright (C) 2015 Red Hat
> + * Copyright (C) 2015 Stephen Chandler Paul <thatslyude@gmail.com>
> + *
> + * This program is free software; you can redistribute it and/or
> modify it
> + * under the terms of the GNU Lesser General Public License as
> published by the
> + * Free Software Foundation; either version 2 of the License, or (at
> your
> + * option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of
> MERCHANTABILITY or FITNESS
> + * FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public
> License for more
> + * details.
> + */
> +#include <linux/circ_buf.h>
> +#include <linux/mutex.h>
> +#include <linux/module.h>
> +#include <linux/init.h>
> +#include <linux/kernel.h>
> +#include <linux/serio.h>
> +#include <linux/slab.h>
> +#include <linux/fs.h>
> +#include <linux/miscdevice.h>
> +#include <linux/sched.h>
> +#include <linux/poll.h>
> +#include <linux/spinlock.h>
> +#include <uapi/linux/userio.h>
> +
> +#define USERIO_NAME "userio"
> +#define USERIO_BUFSIZE 16
> +
> +static const struct file_operations userio_fops;
> +static struct miscdevice userio_misc;
> +
> +struct userio_device {
> + struct serio *serio;
> +
> + bool running;
> +
> + u8 head;
> + u8 tail;
> +
> + /*
> + * Since we need to copy from userspace when modifying the
> tail and we
> + * don't want to lock up the serio driver during these
> operations, we
> + * use two different locks
> + */
> + spinlock_t head_lock;
> + struct mutex tail_lock;
> +
> + unsigned char buf[USERIO_BUFSIZE];
> +
> + wait_queue_head_t waitq;
> +};
> +
> +/**
> + * userio_device_write - Write data from serio to a userio device in
> userspace
> + * @id: The serio port for the userio device
> + * @val: The data to write to the device
> + */
> +static int userio_device_write(struct serio *id, unsigned char val)
> +{
> + struct userio_device *userio = id->port_data;
> + unsigned long flags;
> +
> + if (!userio)
> + return -1;
> +
> + spin_lock_irqsave(&userio->head_lock, flags);
> +
> + userio->buf[userio->head] = val;
> + userio->head = (userio->head + 1) % USERIO_BUFSIZE;
> +
> + spin_unlock_irqrestore(&userio->head_lock, flags);
> +
> + if (userio->head == userio->tail)
> + dev_warn(userio_misc.this_device,
> + "Buffer overflowed, userio client isn't
> keeping up");
> +
> + wake_up_interruptible(&userio->waitq);
> +
> + return 0;
> +}
> +
> +static int userio_char_open(struct inode *inode, struct file *file)
> +{
> + struct userio_device *userio;
> +
> + userio = devm_kzalloc(userio_misc.this_device,
> + sizeof(struct userio_device),
> GFP_KERNEL);
> + if (!userio)
> + return -ENOMEM;
> +
> + spin_lock_init(&userio->head_lock);
> + mutex_init(&userio->tail_lock);
> + init_waitqueue_head(&userio->waitq);
> +
> + userio->serio = kzalloc(sizeof(struct serio), GFP_KERNEL);
> + if (!userio->serio) {
> + devm_kfree(userio_misc.this_device, userio);
> + return -ENOMEM;
> + }
> +
> + userio->serio->write = userio_device_write;
> + userio->serio->port_data = userio;
> +
> + file->private_data = userio;
> +
> + return 0;
> +}
> +
> +static int userio_char_release(struct inode *inode, struct file
> *file)
> +{
> + struct userio_device *userio = file->private_data;
> +
> + if (userio->serio) {
> + userio->serio->port_data = NULL;
> +
> + if (userio->running)
> + serio_unregister_port(userio->serio);
> + else
> + kfree(userio->serio);
> + }
> +
> + devm_kfree(userio_misc.this_device, userio);
> +
> + return 0;
> +}
> +
> +static ssize_t userio_char_read(struct file *file, char __user
> *buffer,
> + size_t count, loff_t *ppos)
> +{
> + struct userio_device *userio = file->private_data;
> + int ret;
> + size_t nonwrap_len, copylen;
> +
> + if (!count)
> + return 0;
> +
> + if (file->f_flags & O_NONBLOCK && userio->head == userio
> ->tail)
> + return -EAGAIN;
> + else {
> + ret = wait_event_interruptible(userio->waitq,
> + userio->head !=
> userio->tail);
> +
> + if (ret)
> + return ret;
> + }
> +
> + mutex_lock(&userio->tail_lock);
> +
> + nonwrap_len = CIRC_CNT_TO_END(userio->head, userio->tail,
> + USERIO_BUFSIZE);
> + copylen = min(nonwrap_len, count);
> +
> + if (copy_to_user(buffer, &userio->buf[userio->tail],
> copylen)) {
> + mutex_unlock(&userio->tail_lock);
> + return -EFAULT;
> + }
> +
> + userio->tail = (userio->tail + copylen) % USERIO_BUFSIZE;
> +
> + mutex_unlock(&userio->tail_lock);
> +
> + return copylen;
> +}
> +
> +static ssize_t userio_char_write(struct file *file, const char
> __user *buffer,
> + size_t count, loff_t *ppos)
> +{
> + struct userio_device *userio = file->private_data;
> + struct userio_cmd cmd;
> +
> + if (count < sizeof(cmd))
> + return -EINVAL;
> +
> + if (copy_from_user(&cmd, buffer, sizeof(cmd)))
> + return -EFAULT;
> +
> + switch (cmd.type) {
> + case USERIO_CMD_REGISTER:
> + if (!userio->serio->id.type) {
> + dev_warn(userio_misc.this_device,
> + "No port type given on
> /dev/userio\n");
> +
> + return -EINVAL;
> + }
> + if (userio->running) {
> + dev_warn(userio_misc.this_device,
> + "Begin command sent, but we're
> already running\n");
> +
> + return -EINVAL;
> + }
> +
> + userio->running = true;
> + serio_register_port(userio->serio);
> + break;
> +
> + case USERIO_CMD_SET_PORT_TYPE:
> + if (userio->running) {
> + dev_warn(userio_misc.this_device,
> + "Can't change port type on an
> already running userio instance\n");
> +
> + return -EINVAL;
> + }
> +
> + userio->serio->id.type = cmd.data;
> + break;
> +
> + case USERIO_CMD_SEND_INTERRUPT:
> + if (!userio->running) {
> + dev_warn(userio_misc.this_device,
> + "The device must be registered
> before sending interrupts\n");
> +
> + return -EINVAL;
> + }
> +
> + serio_interrupt(userio->serio, cmd.data, 0);
> + break;
> +
> + default:
> + return -EINVAL;
> + }
> +
> + return sizeof(cmd);
> +}
> +
> +static unsigned int userio_char_poll(struct file *file, poll_table
> *wait)
> +{
> + struct userio_device *userio = file->private_data;
> +
> + poll_wait(file, &userio->waitq, wait);
> +
> + if (userio->head != userio->tail)
> + return POLLIN | POLLRDNORM;
> +
> + return 0;
> +}
> +
> +static const struct file_operations userio_fops = {
> + .owner = THIS_MODULE,
> + .open = userio_char_open,
> + .release = userio_char_release,
> + .read = userio_char_read,
> + .write = userio_char_write,
> + .poll = userio_char_poll,
> + .llseek = no_llseek,
> +};
> +
> +static struct miscdevice userio_misc = {
> + .fops = &userio_fops,
> + .minor = MISC_DYNAMIC_MINOR,
> + .name = USERIO_NAME,
> +};
> +
> +MODULE_AUTHOR("Stephen Chandler Paul <thatslyude@gmail.com>");
> +MODULE_DESCRIPTION("userio");
> +MODULE_LICENSE("GPL");
> +
> +module_driver(userio_misc, misc_register, misc_deregister);
> diff --git a/include/uapi/linux/userio.h
> b/include/uapi/linux/userio.h
> new file mode 100644
> index 0000000..da0a3d6
> --- /dev/null
> +++ b/include/uapi/linux/userio.h
> @@ -0,0 +1,42 @@
> +/*
> + * userio.h
> + * Copyright (C) 2015 Red Hat
> + * Copyright (C) 2015 Lyude (Stephen Chandler Paul) <
> cpaul@redhat.com>
> + *
> + * This program is free software; you can redistribute it and/or
> modify it
> + * under the terms of the GNU Lesser General Public License as
> published by the
> + * Free Software Foundation; either version 2 of the License, or (at
> your
> + * option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of
> MERCHANTABILITY or FITNESS
> + * FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public
> License for more
> + * details.
> + *
> + * This is the public header used for user-space communication with
> the userio
> + * driver. __attribute__((__packed__)) is used for all structs to
> keep ABI
> + * compatibility between all architectures.
> + */
> +
> +#ifndef _USERIO_H
> +#define _USERIO_H
> +
> +#include <linux/types.h>
> +
> +#define USERIO_CMD_REGISTER 0
> +#define USERIO_CMD_SET_PORT_TYPE 1
> +#define USERIO_CMD_SEND_INTERRUPT 2
> +
> +/*
> + * userio Commands
> + * All commands sent to /dev/userio are encoded using this
> structure. The type
> + * field should contain a USERIO_CMD* value that indicates what kind
> of command
> + * is being sent to userio. The data field should contain the
> accompanying
> + * argument for the command, if there is one.
> + */
> +struct userio_cmd {
> + __u8 type;
> + __u8 data;
> +} __attribute__((__packed__));
> +
> +#endif /* !_USERIO_H */
--
Cheers,
Stephen Chandler Paul
^ permalink raw reply
* Re: [PATCH -mm v9 7/8] proc: export idle flag via kpageflags
From: Andres Lagar-Cavilla @ 2015-07-22 20:46 UTC (permalink / raw)
To: Andrew Morton
Cc: Vladimir Davydov, Minchan Kim, Raghavendra K T, Johannes Weiner,
Michal Hocko, Greg Thelen, Michel Lespinasse, David Rientjes,
Pavel Emelyanov, Cyrill Gorcunov, Jonathan Corbet, linux-api,
linux-doc, linux-mm, cgroups, linux-kernel
In-Reply-To: <20150722124421.3313e8f007d76b386e1d61ec@linux-foundation.org>
[-- Attachment #1: Type: text/plain, Size: 1432 bytes --]
In page_referenced_one:
+ if (referenced)
+ clear_page_idle(page);
Andres
On Wed, Jul 22, 2015 at 12:44 PM, Andrew Morton <akpm@linux-foundation.org>
wrote:
> On Wed, 22 Jul 2015 19:25:28 +0300 Vladimir Davydov <
> vdavydov@parallels.com> wrote:
>
> > On Tue, Jul 21, 2015 at 04:35:00PM -0700, Andrew Morton wrote:
> > > On Sun, 19 Jul 2015 15:31:16 +0300 Vladimir Davydov <
> vdavydov@parallels.com> wrote:
> > >
> > > > As noted by Minchan, a benefit of reading idle flag from
> > > > /proc/kpageflags is that one can easily filter dirty and/or
> unevictable
> > > > pages while estimating the size of unused memory.
> > > >
> > > > Note that idle flag read from /proc/kpageflags may be stale in case
> the
> > > > page was accessed via a PTE, because it would be too costly to
> iterate
> > > > over all page mappings on each /proc/kpageflags read to provide an
> > > > up-to-date value. To make sure the flag is up-to-date one has to read
> > > > /proc/kpageidle first.
> > >
> > > Is there any value in teaching the regular old page scanner to update
> > > these flags? If it's doing an rmap scan anyway...
> >
> > I don't understand what you mean by "regular old page scanner". Could
> > you please elaborate?
>
> Whenever kswapd or direct reclaim perform an rmap scan, take that as an
> opportunity to also update PageIdle().
>
>
--
Andres Lagar-Cavilla | Google Kernel Team | andreslc@google.com
[-- Attachment #2: Type: text/html, Size: 3122 bytes --]
^ permalink raw reply
* Re: [RFC v3 1/1] Input: Add userio module
From: Benjamin Tissoires @ 2015-07-22 20:53 UTC (permalink / raw)
To: Stephen Chandler Paul
Cc: Dmitry Torokhov, Andrew Morton, Mauro Carvalho Chehab, Greg KH,
Arnd Bergmann, Joe Perches, Jiri Slaby, Vishnu Patekar,
Sebastian Ott, linux-doc-u79uwXL29TY76Z2rM5mHXA,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-input,
linux-api-u79uwXL29TY76Z2rM5mHXA, Benjamin Tissoires,
Hans de Goede
In-Reply-To: <1437594773-22274-2-git-send-email-cpaul-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
On Wed, Jul 22, 2015 at 3:52 PM, Stephen Chandler Paul <cpaul-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> Debugging input devices, specifically laptop touchpads, can be tricky
> without having the physical device handy. Here we try to remedy that
> with userio. This module allows an application to connect to a character
> device provided by the kernel, and emulate any serio device. In
> combination with userspace programs that can record PS/2 devices and
> replay them through the /dev/userio device, this allows developers to
> debug driver issues on the PS/2 level with devices simply by requesting
> a recording from the user experiencing the issue without having to have
> the physical hardware in front of them.
>
> Signed-off-by: Stephen Chandler Paul <cpaul-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> ---
Thanks Chandler for the new version. I think we still need to enhance
the concurrency barriers here. See inline.
> Documentation/input/userio.txt | 70 +++++++++++
> MAINTAINERS | 6 +
> drivers/input/serio/Kconfig | 11 ++
> drivers/input/serio/Makefile | 1 +
> drivers/input/serio/userio.c | 261 +++++++++++++++++++++++++++++++++++++++++
> include/uapi/linux/userio.h | 42 +++++++
> 6 files changed, 391 insertions(+)
> create mode 100644 Documentation/input/userio.txt
> create mode 100644 drivers/input/serio/userio.c
> create mode 100644 include/uapi/linux/userio.h
>
> diff --git a/Documentation/input/userio.txt b/Documentation/input/userio.txt
> new file mode 100644
> index 0000000..0880c0f
> --- /dev/null
> +++ b/Documentation/input/userio.txt
> @@ -0,0 +1,70 @@
> + The userio Protocol
> + (c) 2015 Stephen Chandler Paul <thatslyude-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> + Sponsored by Red Hat
> +--------------------------------------------------------------------------------
> +
> +1. Introduction
> +~~~~~~~~~~~~~~~
> + This module is intended to try to make the lives of input driver developers
> +easier by allowing them to test various serio devices (mainly the various
> +touchpads found on laptops) without having to have the physical device in front
> +of them. userio accomplishes this by allowing any privileged userspace program
> +to directly interact with the kernel's serio driver and control a virtual serio
> +port from there.
> +
> +2. Usage overview
> +~~~~~~~~~~~~~~~~~
> + In order to interact with the userio kernel module, one simply opens the
> +/dev/userio character device in their applications. Commands are sent to the
> +kernel module by writing to the device, and any data received from the serio
> +driver is read as-is from the /dev/userio device. All of the structures and
> +macros you need to interact with the device are defined in <linux/userio.h> and
> +<linux/serio.h>.
> +
> +3. Command Structure
> +~~~~~~~~~~~~~~~~~~~~
> + The struct used for sending commands to /dev/userio is as follows:
> +
> + struct userio_cmd {
> + __u8 type;
> + __u8 data;
> + };
> +
> + "type" describes the type of command that is being sent. This can be any one
> +of the USERIO_CMD macros defined in <linux/userio.h>. "data" is the argument
> +that goes along with the command. In the event that the command doesn't have an
> +argument, this field can be left untouched and will be ignored by the kernel.
> +Each command should be sent by writing the struct directly to the character
> +device. In the event that the command you send is invalid, an error will be
> +returned by the character device and a more descriptive error will be printed
> +to the kernel log. Only one command can be sent at a time, any additional data
> +written to the character device after the initial command will be ignored.
> + To close the virtual serio port, just close /dev/userio.
> +
> +4. Commands
> +~~~~~~~~~~~
> +
> +4.1 USERIO_CMD_REGISTER
> +~~~~~~~~~~~~~~~~~~~~~~~
> + Registers the port with the serio driver and begins transmitting data back and
> +forth. Registration can only be performed once a port type is set with
> +USERIO_CMD_SET_PORT_TYPE. Has no argument.
> +
> +4.2 USERIO_CMD_SET_PORT_TYPE
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> + Sets the type of port we're emulating, where "data" is the port type being
> +set. Can be any of the macros from <linux/serio.h>. For example: SERIO_8042
> +would set the port type to be a normal PS/2 port.
> +
> +4.3 USERIO_CMD_SEND_INTERRUPT
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> + Sends an interrupt through the virtual serio port to the serio driver, where
> +"data" is the interrupt data being sent.
> +
> +5. Userspace tools
> +~~~~~~~~~~~~~~~~~~
> + The userio userspace tools are able to record PS/2 devices using some of the
> +debugging information from i8042, and play back the devices on /dev/userio. The
> +latest version of these tools can be found at:
> +
> + https://github.com/Lyude/ps2emu
> diff --git a/MAINTAINERS b/MAINTAINERS
> index a226416..68a0977 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -10877,6 +10877,12 @@ S: Maintained
> F: drivers/media/v4l2-core/videobuf2-*
> F: include/media/videobuf2-*
>
> +VIRTUAL PS/2 DEVICE DRIVER
> +M: Stephen Chandler Paul <thatslyude-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> +S: Maintained
> +F: drivers/input/serio/ps2emu.c
> +F: include/uapi/linux/ps2emu.h
> +
> VIRTIO CONSOLE DRIVER
> M: Amit Shah <amit.shah-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> L: virtualization-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
> diff --git a/drivers/input/serio/Kconfig b/drivers/input/serio/Kconfig
> index 200841b..22b8b58 100644
> --- a/drivers/input/serio/Kconfig
> +++ b/drivers/input/serio/Kconfig
> @@ -292,4 +292,15 @@ config SERIO_SUN4I_PS2
> To compile this driver as a module, choose M here: the
> module will be called sun4i-ps2.
>
> +config USERIO
> + tristate "Virtual serio device support"
> + help
> + Say Y here if you want to emulate serio devices using the userio
> + kernel module.
> +
> + To compile this driver as a module, choose M here: the module will be
> + called userio.
> +
> + If you are unsure, say N.
> +
> endif
> diff --git a/drivers/input/serio/Makefile b/drivers/input/serio/Makefile
> index c600089..2374ef9 100644
> --- a/drivers/input/serio/Makefile
> +++ b/drivers/input/serio/Makefile
> @@ -30,3 +30,4 @@ obj-$(CONFIG_SERIO_APBPS2) += apbps2.o
> obj-$(CONFIG_SERIO_OLPC_APSP) += olpc_apsp.o
> obj-$(CONFIG_HYPERV_KEYBOARD) += hyperv-keyboard.o
> obj-$(CONFIG_SERIO_SUN4I_PS2) += sun4i-ps2.o
> +obj-$(CONFIG_USERIO) += userio.o
> diff --git a/drivers/input/serio/userio.c b/drivers/input/serio/userio.c
> new file mode 100644
> index 0000000..578b107
> --- /dev/null
> +++ b/drivers/input/serio/userio.c
> @@ -0,0 +1,261 @@
> +/*
> + * userio kernel serio device emulation module
> + * Copyright (C) 2015 Red Hat
> + * Copyright (C) 2015 Stephen Chandler Paul <thatslyude-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU Lesser General Public License as published by the
> + * Free Software Foundation; either version 2 of the License, or (at your
> + * option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
> + * FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more
> + * details.
> + */
> +#include <linux/circ_buf.h>
> +#include <linux/mutex.h>
> +#include <linux/module.h>
> +#include <linux/init.h>
> +#include <linux/kernel.h>
> +#include <linux/serio.h>
> +#include <linux/slab.h>
> +#include <linux/fs.h>
> +#include <linux/miscdevice.h>
> +#include <linux/sched.h>
> +#include <linux/poll.h>
> +#include <linux/spinlock.h>
> +#include <uapi/linux/userio.h>
> +
> +#define USERIO_NAME "userio"
> +#define USERIO_BUFSIZE 16
> +
> +static const struct file_operations userio_fops;
> +static struct miscdevice userio_misc;
> +
> +struct userio_device {
> + struct serio *serio;
> +
> + bool running;
> +
> + u8 head;
> + u8 tail;
> +
> + /*
> + * Since we need to copy from userspace when modifying the tail and we
> + * don't want to lock up the serio driver during these operations, we
> + * use two different locks
> + */
My guts tell me that this is racy. You generally want to have a single
mutex/spinlock to protect your buffer. Having 2 might introduce
interesting complex problems...
> + spinlock_t head_lock;
> + struct mutex tail_lock;
> +
> + unsigned char buf[USERIO_BUFSIZE];
> +
> + wait_queue_head_t waitq;
> +};
> +
> +/**
> + * userio_device_write - Write data from serio to a userio device in userspace
> + * @id: The serio port for the userio device
> + * @val: The data to write to the device
> + */
> +static int userio_device_write(struct serio *id, unsigned char val)
> +{
> + struct userio_device *userio = id->port_data;
> + unsigned long flags;
> +
> + if (!userio)
> + return -1;
> +
> + spin_lock_irqsave(&userio->head_lock, flags);
> +
I know Dmitry said that this could be called by several threads, but I
am not 100% sure.
This function is called by the serio driver, and I don't think we will
have more than one thread at a time in the serio driver (this would
raise interesting hardware behavior for a serial protocol).
Though protecting the head and the value here could be a good thing if
you protect also when you read them.
> + userio->buf[userio->head] = val;
> + userio->head = (userio->head + 1) % USERIO_BUFSIZE;
> +
> + spin_unlock_irqrestore(&userio->head_lock, flags);
> +
> + if (userio->head == userio->tail)
> + dev_warn(userio_misc.this_device,
> + "Buffer overflowed, userio client isn't keeping up");
> +
> + wake_up_interruptible(&userio->waitq);
> +
> + return 0;
> +}
> +
> +static int userio_char_open(struct inode *inode, struct file *file)
> +{
> + struct userio_device *userio;
> +
> + userio = devm_kzalloc(userio_misc.this_device,
> + sizeof(struct userio_device), GFP_KERNEL);
> + if (!userio)
> + return -ENOMEM;
> +
> + spin_lock_init(&userio->head_lock);
> + mutex_init(&userio->tail_lock);
> + init_waitqueue_head(&userio->waitq);
> +
> + userio->serio = kzalloc(sizeof(struct serio), GFP_KERNEL);
> + if (!userio->serio) {
> + devm_kfree(userio_misc.this_device, userio);
> + return -ENOMEM;
> + }
> +
> + userio->serio->write = userio_device_write;
> + userio->serio->port_data = userio;
> +
> + file->private_data = userio;
> +
> + return 0;
> +}
> +
> +static int userio_char_release(struct inode *inode, struct file *file)
> +{
> + struct userio_device *userio = file->private_data;
> +
> + if (userio->serio) {
> + userio->serio->port_data = NULL;
> +
> + if (userio->running)
> + serio_unregister_port(userio->serio);
> + else
> + kfree(userio->serio);
> + }
> +
> + devm_kfree(userio_misc.this_device, userio);
Calling devm_kfree directly should always raise a big red warning. The
point of having managed memory is to avoid calling kfree. If you have
to, then you probably should not call devm_kzalloc() in the first
place.
> +
> + return 0;
> +}
> +
> +static ssize_t userio_char_read(struct file *file, char __user *buffer,
> + size_t count, loff_t *ppos)
> +{
> + struct userio_device *userio = file->private_data;
> + int ret;
> + size_t nonwrap_len, copylen;
> +
> + if (!count)
> + return 0;
> +
> + if (file->f_flags & O_NONBLOCK && userio->head == userio->tail)
This section is not protected while you are accessing userio->tail.
I think it is still safe here, but I am not 100% sure.
> + return -EAGAIN;
> + else {
> + ret = wait_event_interruptible(userio->waitq,
> + userio->head != userio->tail);
I would protect this under the mutex, but OTOH, it might lead also to
some interesting lockups.
> +
> + if (ret)
> + return ret;
> + }
> +
> + mutex_lock(&userio->tail_lock);
> +
> + nonwrap_len = CIRC_CNT_TO_END(userio->head, userio->tail,
> + USERIO_BUFSIZE);
> + copylen = min(nonwrap_len, count);
> +
> + if (copy_to_user(buffer, &userio->buf[userio->tail], copylen)) {
> + mutex_unlock(&userio->tail_lock);
> + return -EFAULT;
> + }
> +
> + userio->tail = (userio->tail + copylen) % USERIO_BUFSIZE;
> +
> + mutex_unlock(&userio->tail_lock);
> +
> + return copylen;
> +}
> +
> +static ssize_t userio_char_write(struct file *file, const char __user *buffer,
> + size_t count, loff_t *ppos)
> +{
> + struct userio_device *userio = file->private_data;
> + struct userio_cmd cmd;
> +
> + if (count < sizeof(cmd))
> + return -EINVAL;
> +
> + if (copy_from_user(&cmd, buffer, sizeof(cmd)))
> + return -EFAULT;
> +
Starting from here
> + switch (cmd.type) {
> + case USERIO_CMD_REGISTER:
> + if (!userio->serio->id.type) {
> + dev_warn(userio_misc.this_device,
> + "No port type given on /dev/userio\n");
> +
> + return -EINVAL;
> + }
> + if (userio->running) {
> + dev_warn(userio_misc.this_device,
> + "Begin command sent, but we're already running\n");
> +
> + return -EINVAL;
> + }
> +
> + userio->running = true;
> + serio_register_port(userio->serio);
> + break;
> +
> + case USERIO_CMD_SET_PORT_TYPE:
> + if (userio->running) {
> + dev_warn(userio_misc.this_device,
> + "Can't change port type on an already running userio instance\n");
> +
> + return -EINVAL;
> + }
> +
> + userio->serio->id.type = cmd.data;
> + break;
> +
> + case USERIO_CMD_SEND_INTERRUPT:
> + if (!userio->running) {
> + dev_warn(userio_misc.this_device,
> + "The device must be registered before sending interrupts\n");
> +
> + return -EINVAL;
> + }
> +
> + serio_interrupt(userio->serio, cmd.data, 0);
> + break;
> +
> + default:
> + return -EINVAL;
> + }
... to here, you should protect this against several threads. You
don't want to register twice the serio port because 2 threads read
userio->running as false when issuing USERIO_CMD_REGISTER.
Likewise, there could be races between USERIO_CMD_SEND_INTERRUPT and
USERIO_CMD_REGISTER.
I don't think this user-space interface (both read and write) is time
critical, so I would simply put a lock before each interaction with
the internal state and release it when done.
> +
> + return sizeof(cmd);
> +}
> +
> +static unsigned int userio_char_poll(struct file *file, poll_table *wait)
> +{
> + struct userio_device *userio = file->private_data;
> +
> + poll_wait(file, &userio->waitq, wait);
> +
> + if (userio->head != userio->tail)
> + return POLLIN | POLLRDNORM;
> +
> + return 0;
> +}
> +
> +static const struct file_operations userio_fops = {
> + .owner = THIS_MODULE,
> + .open = userio_char_open,
> + .release = userio_char_release,
> + .read = userio_char_read,
> + .write = userio_char_write,
> + .poll = userio_char_poll,
> + .llseek = no_llseek,
> +};
> +
> +static struct miscdevice userio_misc = {
> + .fops = &userio_fops,
> + .minor = MISC_DYNAMIC_MINOR,
> + .name = USERIO_NAME,
> +};
> +
> +MODULE_AUTHOR("Stephen Chandler Paul <thatslyude-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>");
> +MODULE_DESCRIPTION("userio");
> +MODULE_LICENSE("GPL");
> +
> +module_driver(userio_misc, misc_register, misc_deregister);
> diff --git a/include/uapi/linux/userio.h b/include/uapi/linux/userio.h
> new file mode 100644
> index 0000000..da0a3d6
> --- /dev/null
> +++ b/include/uapi/linux/userio.h
> @@ -0,0 +1,42 @@
> +/*
> + * userio.h
> + * Copyright (C) 2015 Red Hat
> + * Copyright (C) 2015 Lyude (Stephen Chandler Paul) <cpaul-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU Lesser General Public License as published by the
> + * Free Software Foundation; either version 2 of the License, or (at your
> + * option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
> + * FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more
> + * details.
> + *
> + * This is the public header used for user-space communication with the userio
> + * driver. __attribute__((__packed__)) is used for all structs to keep ABI
> + * compatibility between all architectures.
> + */
> +
> +#ifndef _USERIO_H
> +#define _USERIO_H
> +
> +#include <linux/types.h>
> +
> +#define USERIO_CMD_REGISTER 0
> +#define USERIO_CMD_SET_PORT_TYPE 1
> +#define USERIO_CMD_SEND_INTERRUPT 2
> +
> +/*
> + * userio Commands
> + * All commands sent to /dev/userio are encoded using this structure. The type
> + * field should contain a USERIO_CMD* value that indicates what kind of command
> + * is being sent to userio. The data field should contain the accompanying
> + * argument for the command, if there is one.
> + */
> +struct userio_cmd {
> + __u8 type;
> + __u8 data;
> +} __attribute__((__packed__));
> +
> +#endif /* !_USERIO_H */
> --
> 2.4.3
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-input" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
Cheers,
Benjamin
^ permalink raw reply
* Re: [RFC v3 1/1] Input: Add userio module
From: Dmitry Torokhov @ 2015-07-22 21:39 UTC (permalink / raw)
To: Stephen Chandler Paul
Cc: Andrew Morton, Mauro Carvalho Chehab, Greg KH, Arnd Bergmann,
Joe Perches, Jiri Slaby, Vishnu Patekar, Sebastian Ott, linux-doc,
linux-kernel, linux-input, linux-api, Benjamin Tissoires,
Hans de Goede
In-Reply-To: <1437594773-22274-2-git-send-email-cpaul@redhat.com>
Hi Stephen,
On Wed, Jul 22, 2015 at 03:52:53PM -0400, Stephen Chandler Paul wrote:
> +static int userio_char_open(struct inode *inode, struct file *file)
> +{
> + struct userio_device *userio;
> +
> + userio = devm_kzalloc(userio_misc.this_device,
> + sizeof(struct userio_device), GFP_KERNEL);
> + if (!userio)
> + return -ENOMEM;
Never ever use devm_ API outside of device probe() calls because
devm-managed resources only freed after failed probe() or after remove()
callback on driver model device and not any other device (block, char,
etc) in the system.
> +
> + spin_lock_init(&userio->head_lock);
> + mutex_init(&userio->tail_lock);
> + init_waitqueue_head(&userio->waitq);
> +
> + userio->serio = kzalloc(sizeof(struct serio), GFP_KERNEL);
> + if (!userio->serio) {
> + devm_kfree(userio_misc.this_device, userio);
And indeed here you have ot manually release the acquired resource. So
the only benefit you got form devm* is wasted resousrces.
> + return -ENOMEM;
> + }
> +
> + userio->serio->write = userio_device_write;
> + userio->serio->port_data = userio;
> +
> + file->private_data = userio;
> +
> + return 0;
> +}
> +
> +static int userio_char_release(struct inode *inode, struct file *file)
> +{
> + struct userio_device *userio = file->private_data;
> +
> + if (userio->serio) {
> + userio->serio->port_data = NULL;
No need to reset, it is going away.
> +
> + if (userio->running)
> + serio_unregister_port(userio->serio);
> + else
> + kfree(userio->serio);
> + }
> +
> + devm_kfree(userio_misc.this_device, userio);
> +
> + return 0;
> +}
> +
> +static ssize_t userio_char_read(struct file *file, char __user *buffer,
> + size_t count, loff_t *ppos)
> +{
> + struct userio_device *userio = file->private_data;
> + int ret;
> + size_t nonwrap_len, copylen;
> +
> + if (!count)
> + return 0;
> +
> + if (file->f_flags & O_NONBLOCK && userio->head == userio->tail)
> + return -EAGAIN;
> + else {
This is racy. If there was data and other thread "stole" it here then
your O_NONBLOCK will block. See evdev_read() how you supposed to handle
this.
Thanks.
--
Dmitry
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox