* Re: [PATCH] seccomp: add ptrace commands for suspend/resume
From: Andy Lutomirski @ 2015-06-02 19:27 UTC (permalink / raw)
To: Pavel Emelyanov
Cc: Oleg Nesterov, Tycho Andersen,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Linux API,
Kees Cook, Will Drewry, Roland McGrath, Serge E. Hallyn
In-Reply-To: <556DFDB2.3050205-bzQdu9zFT3WakBO8gow8eQ@public.gmane.org>
On Tue, Jun 2, 2015 at 12:02 PM, Pavel Emelyanov <xemul-bzQdu9zFT3WakBO8gow8eQ@public.gmane.org> wrote:
>
>>> +int suspend_seccomp(struct task_struct *task)
>>> +{
>>> + int ret = -EACCES;
>>> +
>>> + spin_lock_irq(&task->sighand->siglock);
>>> +
>>> + if (!capable(CAP_SYS_ADMIN))
>>> + goto out;
>>
>> I am puzzled ;) Why do we need ->siglock? And even if we need it, why
>> we can't check CAP_SYS_ADMIN lockless?
>>
>> And I am not sure I understand why do we need the additional security
>> check, but I leave this to you and Andy.
>>
>> If you have the rights to trace this task, then you can do anything
>> the tracee could do without the filtering.
>
> I think _this_ check is required, otherwise the seccomp-ed task (in
> filtered mode) fork-s a child, then this child ptrace-attach to parent
> (allowed) then suspend its seccomd. And -- we have unpriviledged process
> de-seccomped.
The interaction between ptrace and seccomp is already highly screwy,
and seccomp users can often escape using ptrace. That being said, I'd
rather not further enshrine it.
It might be worth changing the check to verify that the task trying to
suspect seccomp isn't itself subject to seccomp. That should get most
of the safety. We are already kind of screwed if someone tries to run
criu under seccomp due to unresolved nesting issues.
--Andy
^ permalink raw reply
* Re: [PATCH] seccomp: add ptrace commands for suspend/resume
From: Jann Horn @ 2015-06-02 19:24 UTC (permalink / raw)
To: Pavel Emelyanov
Cc: Oleg Nesterov, Tycho Andersen, linux-kernel, linux-api, Kees Cook,
Andy Lutomirski, Will Drewry, Roland McGrath, Serge E. Hallyn
In-Reply-To: <556DFDB2.3050205@parallels.com>
[-- Attachment #1: Type: text/plain, Size: 1473 bytes --]
On Tue, Jun 02, 2015 at 10:02:10PM +0300, Pavel Emelyanov wrote:
>
> >> +int suspend_seccomp(struct task_struct *task)
> >> +{
> >> + int ret = -EACCES;
> >> +
> >> + spin_lock_irq(&task->sighand->siglock);
> >> +
> >> + if (!capable(CAP_SYS_ADMIN))
> >> + goto out;
> >
> > I am puzzled ;) Why do we need ->siglock? And even if we need it, why
> > we can't check CAP_SYS_ADMIN lockless?
> >
> > And I am not sure I understand why do we need the additional security
> > check, but I leave this to you and Andy.
> >
> > If you have the rights to trace this task, then you can do anything
> > the tracee could do without the filtering.
>
> I think _this_ check is required, otherwise the seccomp-ed task (in
> filtered mode) fork-s a child, then this child ptrace-attach to parent
> (allowed) then suspend its seccomd. And -- we have unpriviledged process
> de-seccomped.
If you can ptrace(), you can already escape from seccomp. See this
section in man 2 seccomp, in the SECCOMP_RET_TRACE section:
The seccomp check will not be run again after the tracer is
notified. (This means that seccomp-based sandboxes must not
allow use of ptrace(2)—even of other sandboxed processes—
without extreme care; ptracers can use this mechanism to
escape from the seccomp sandbox.)
(But I think there have been discussions about changing that behavior in
the future?)
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH 30/98] hdspm.h: use __u8, __u32 and __u64 from linux/types.h
From: Mikko Rapeli @ 2015-06-02 19:18 UTC (permalink / raw)
To: Takashi Iwai; +Cc: linux-kernel, Jaroslav Kysela, alsa-devel, linux-api
In-Reply-To: <s5ha8wl8blx.wl-tiwai@suse.de>
On Sun, May 31, 2015 at 09:11:54AM +0200, Takashi Iwai wrote:
> At Sat, 30 May 2015 17:38:22 +0200,
> Mikko Rapeli wrote:
> >
> > Fixes userspace compilation errors like:
> >
> > sound/hdspm.h:43:2: error: unknown type name ‘uint32_t’
>
> Hmm, how do you get this error at all?
> The header includes stdint.h for user-space.
Yes, sorry. This patch removes the stdint.h include and uses kernel __u32
types instead. Will fix the commit message. Thanks!
-Mikko
^ permalink raw reply
* Re: [PATCH 29/98] scsi_netlink_fc.h: include stdint.h in userspace
From: Mikko Rapeli @ 2015-06-02 19:14 UTC (permalink / raw)
To: Frans Klaver
Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <20150530201730.GC15645-LuJdRk73L+f/9pzu0YdTqQ@public.gmane.org>
On Sat, May 30, 2015 at 10:17:30PM +0200, Frans Klaver wrote:
> On Sat, May 30, 2015 at 05:38:21PM +0200, Mikko Rapeli wrote:
> > Fixes compilation error:
> >
> > scsi/scsi_netlink_fc.h:60:2: error: expected specifier-qualifier-list before ‘uint64_t’
> >
> > Signed-off-by: Mikko Rapeli <mikko.rapeli-X3B1VOXEql0@public.gmane.org>
> > ---
> > include/uapi/scsi/scsi_netlink_fc.h | 5 +++++
> > 1 file changed, 5 insertions(+)
> >
> > diff --git a/include/uapi/scsi/scsi_netlink_fc.h b/include/uapi/scsi/scsi_netlink_fc.h
> > index 2493a0f..b857e34 100644
> > --- a/include/uapi/scsi/scsi_netlink_fc.h
> > +++ b/include/uapi/scsi/scsi_netlink_fc.h
> > @@ -21,6 +21,11 @@
> > #ifndef SCSI_NETLINK_FC_H
> > #define SCSI_NETLINK_FC_H
> >
> > +#ifdef __KERNEL__
> > +#include <linux/types.h>
> > +#else
> > +#include <stdint.h>
> > +#endif
> > #include <scsi/scsi_netlink.h>
> >
> > /*
>
> Shouldn't we be using __u64 and __u16 in this header instead?
Indeed, this was a rebase/merge botch. Thanks!
-Mikko
^ permalink raw reply
* Re: [PATCH 18/98] include/uapi/sound/emu10k1.h: added EMU10K1 version of DECLARE_BITMAP etc macros
From: Mikko Rapeli @ 2015-06-02 19:08 UTC (permalink / raw)
To: Takashi Iwai
Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA, Jaroslav Kysela,
alsa-devel-K7yf7f+aM1XWsZ/bQMPhNw,
linux-api-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <s5h1thx88tk.wl-tiwai-l3A5Bk7waGM@public.gmane.org>
On Sun, May 31, 2015 at 10:12:07AM +0200, Takashi Iwai wrote:
> At Sun, 31 May 2015 09:18:57 +0200,
> Takashi Iwai wrote:
> >
> > At Sat, 30 May 2015 17:38:10 +0200,
> > Mikko Rapeli wrote:
> > >
> > > Fixes userspace compilation error:
> > >
> > > error: expected specifier-qualifier-list before ‘DECLARE_BITMAP’
> > > DECLARE_BITMAP(gpr_valid, 0x200); /* bitmask of valid initializers */
> > >
> > > DECLARE_BITMAP etc macros are not meant for userspace headers and thus
> > > added here as private copies for emu10k.h.
> > >
> > > Fix was suggested by Arnd Bergmann <arnd-r2nGTMty4D4@public.gmane.org> in message
> > > <2168807.4Yxh5gl11Q@wuerfel> on lkml.
> > >
> > > Signed-off-by: Mikko Rapeli <mikko.rapeli-X3B1VOXEql0@public.gmane.org>
> > > ---
> > > include/uapi/sound/emu10k1.h | 23 ++++++++++++++++++++---
> > > 1 file changed, 20 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/include/uapi/sound/emu10k1.h b/include/uapi/sound/emu10k1.h
> > > index ec1535b..7575f0f 100644
> > > --- a/include/uapi/sound/emu10k1.h
> > > +++ b/include/uapi/sound/emu10k1.h
> > > @@ -34,6 +34,23 @@
> > >
> > > #define EMU10K1_FX8010_PCM_COUNT 8
> > >
> > > +/*
> > > + * Following definitions are copies from kernel headers to support compiling
> > > + * this header file in userspace. The definitions are not generally available
> > > + * in uapi headers so the needed things are copied here wtih __EMU10k1 prefix.
> > > + */
> > > +
> > > +/* From include/linux/bitops.h */
> > > +#define __EMU10K1_BITS_PER_BYTE 8
> > > +/* From include/linux/kernel.h */
> > > +#define __EMU10K1_DIV_ROUND_UP(n,d) (((n) + (d) - 1) / (d))
> > > +/* From include/linux/bitops.h */
> > > +#define __EMU10K1_BITS_TO_LONGS(nr) \
> > > + __EMU10K1_DIV_ROUND_UP(nr, __EMU10K1_BITS_PER_BYTE * sizeof(long))
> > > +/* From include/linux/types.h */
> > > +#define __EMU10K1_DECLARE_BITMAP(name,bits) \
> > > + unsigned long name[__EMU10K1_BITS_TO_LONGS(bits)]
> >
> > This is way too complicated just for a few expansion of the aligned
> > size arrays, IMO. Rather simplify it like below with a comment:
> >
> > /* the array must be aligned to unsigned long (i.e. 4 or 8 bytes) */
> > #define __EMU10K1_DECLARE_BITMAP(name) \
> > unsigned long name[(bits) / sizeof(unsigned long)]
>
> Crap, of course, it must be multiplied with 8.
>
> /* the array must be aligned to unsigned long (i.e. 32 or 64) */
> #define __EMU10K1_DECLARE_BITMAP(name) \
> unsigned long name[(bits) / (sizeof(unsigned long) * 8)]
Thanks, I'll use this definition instead.
-Mikko
^ permalink raw reply
* Re: [PATCH] seccomp: add ptrace commands for suspend/resume
From: Pavel Emelyanov @ 2015-06-02 19:02 UTC (permalink / raw)
To: Oleg Nesterov, Tycho Andersen
Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA, Kees Cook, Andy Lutomirski,
Will Drewry, Roland McGrath, Serge E. Hallyn
In-Reply-To: <20150602182829.GA23449-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>> +int suspend_seccomp(struct task_struct *task)
>> +{
>> + int ret = -EACCES;
>> +
>> + spin_lock_irq(&task->sighand->siglock);
>> +
>> + if (!capable(CAP_SYS_ADMIN))
>> + goto out;
>
> I am puzzled ;) Why do we need ->siglock? And even if we need it, why
> we can't check CAP_SYS_ADMIN lockless?
>
> And I am not sure I understand why do we need the additional security
> check, but I leave this to you and Andy.
>
> If you have the rights to trace this task, then you can do anything
> the tracee could do without the filtering.
I think _this_ check is required, otherwise the seccomp-ed task (in
filtered mode) fork-s a child, then this child ptrace-attach to parent
(allowed) then suspend its seccomd. And -- we have unpriviledged process
de-seccomped.
-- Pavel
^ permalink raw reply
* Re: [PATCH 05/98] exynos_drm.h: use __u64 from linux/types.h
From: Mikko Rapeli @ 2015-06-02 18:59 UTC (permalink / raw)
To: Russell King - ARM Linux
Cc: linux-kernel, Krzysztof Kozlowski, linux-samsung-soc,
Joonyoung Shim, David Airlie, Seung-Woo Kim, dri-devel, Inki Dae,
Kyungmin Park, Kukjin Kim, linux-api, linux-arm-kernel
In-Reply-To: <20150530164655.GM2067@n2100.arm.linux.org.uk>
On Sat, May 30, 2015 at 05:46:56PM +0100, Russell King - ARM Linux wrote:
> Note that drm/drm.h is all that should need to be included - drm/drm.h
> takes care of including linux/types.h when building on Linux platforms.
> (note: if your compiler doesn't set __linux__ then you're probably not
> using the proper compiler...)
Thanks, I'll fix this by including only drm/drm.h in exynos_drm.h,
nouveau_drm.h, radeon_drm.h and via_drm.h patches.
I'm using standard gcc from Debian for the x86 compilation.
-Mikko
^ permalink raw reply
* Re: [PATCH] seccomp: add ptrace commands for suspend/resume
From: Oleg Nesterov @ 2015-06-02 18:48 UTC (permalink / raw)
To: Tycho Andersen
Cc: Andrey Wagin, LKML, linux-api, Kees Cook, Andy Lutomirski,
Will Drewry, Roland McGrath, Pavel Emelyanov, Serge E. Hallyn
In-Reply-To: <20150602130506.GA1823@hopstrocity>
On 06/02, Tycho Andersen wrote:
>
> > Do we need to re-enable seccomp if a tracer detaches unexpectedly.
> > CRIU can be killed and we should try to not affect the task state even
> > in this case.
>
> Yes, I think Pavel's suggestion on the CRIU list of simply
> automatically re-enabling seccomp on ptrace detach
But note that you can't enable tsc if the tracer dies, in this case
the tracee can be running.
Otherwise, if we use PTRACE_O_ instead, it goes away automatically if
the tracer dies or does PTRACE_DETACH.
Oleg.
^ permalink raw reply
* Re: [PATCH] seccomp: add ptrace commands for suspend/resume
From: Oleg Nesterov @ 2015-06-02 18:28 UTC (permalink / raw)
To: Tycho Andersen
Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA, Kees Cook, Andy Lutomirski,
Will Drewry, Roland McGrath, Pavel Emelyanov, Serge E. Hallyn
In-Reply-To: <1433186918-9626-1-git-send-email-tycho.andersen-Z7WLFzj8eWMS+FvcfC7Uqw@public.gmane.org>
On 06/01, Tycho Andersen wrote:
>
> --- a/include/linux/seccomp.h
> +++ b/include/linux/seccomp.h
> @@ -25,6 +25,9 @@ struct seccomp_filter;
> struct seccomp {
> int mode;
> struct seccomp_filter *filter;
> +#ifdef CONFIG_CHECKPOINT_RESTORE
> + bool suspended;
> +#endif
Then afaics you need to change copy_seccomp() to clear ->suspended.
At least if the child is not traced.
> @@ -691,6 +697,11 @@ u32 seccomp_phase1(struct seccomp_data *sd)
> int this_syscall = sd ? sd->nr :
> syscall_get_nr(current, task_pt_regs(current));
>
> +#ifdef CONFIG_CHECKPOINT_RESTORE
> + if (unlikely(current->seccomp.suspended))
> + return SECCOMP_PHASE1_OK;
> +#endif
> +
I am wondering if PTRACE_SUSPEND_SECCOMP can just clear/set TIF_SECCOMP.
Of course, it is not that resume_seccomp() can simply do set_tsk_thread_flag,
it should be more careful. And prctl_set_seccomp() paths will need some
changes. Probably not, this would be more complex.
So perhaps it would be better to add PTRACE_O_SUSPEND_SECCOMP? This also
solves the problem with the killed tracer. Except TIF_NOTSC...
But why do we bother to play with TIF_NOTSC, could you explain?
> +int suspend_seccomp(struct task_struct *task)
> +{
> + int ret = -EACCES;
> +
> + spin_lock_irq(&task->sighand->siglock);
> +
> + if (!capable(CAP_SYS_ADMIN))
> + goto out;
I am puzzled ;) Why do we need ->siglock? And even if we need it, why
we can't check CAP_SYS_ADMIN lockless?
And I am not sure I understand why do we need the additional security
check, but I leave this to you and Andy.
If you have the rights to trace this task, then you can do anything
the tracee could do without the filtering.
> +
> + task->seccomp.suspended = true;
> +
> +#ifdef TIF_NOTSC
> + if (task->seccomp.mode == SECCOMP_MODE_STRICT)
> + clear_tsk_thread_flag(task, TIF_NOTSC);
> +#endif
> +
> + ret = 0;
> +out:
> + spin_unlock_irq(&task->sighand->siglock);
> +
> + return ret;
> +}
> +
> +int resume_seccomp(struct task_struct *task)
> +{
> + int ret = -EACCES;
> +
> + spin_lock_irq(&task->sighand->siglock);
> +
> + if (!capable(CAP_SYS_ADMIN))
> + goto out;
> +
> + task->seccomp.suspended = false;
> +
> +#ifdef TIF_NOTSC
> + if (task->seccomp.mode == SECCOMP_MODE_STRICT)
> + set_tsk_thread_flag(task, TIF_NOTSC);
> +#endif
> +
> + ret = 0;
> +out:
> + spin_unlock_irq(&task->sighand->siglock);
> +
> + return ret;
> +}
> +#endif /* CONFIG_CHECKPOINT_RESTORE */
Well, I do not think we need 2 helpers, just one which takes a boolean
will look better, imo.
Oleg.
^ permalink raw reply
* [PATCH] Update mlockall() and mmap() man pages for LOCKONFAULT flags
From: Eric B Munson @ 2015-06-02 18:13 UTC (permalink / raw)
To: Michael Kerrisk
Cc: Eric B Munson, Andrew Morton, linux-api, linux-mm, linux-man,
linux-kernel
Document the new flags for mmap() and mlockall() and their behavior.
Inlcude a change to getrlimit(2) to cover interactions with
RLIMIT_MEMLOCK. These new flags will be introduced with the 4.2 kernel.
Signed-off-by: Eric B Munson <emunson@akamai.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: linux-api@vger.kernel.org
Cc: linux-mm@kvack.org
Cc: linux-man@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
---
man2/getrlimit.2 | 9 +++++++--
man2/mlock.2 | 28 ++++++++++++++++++++++++++--
man2/mmap.2 | 21 +++++++++++++++++++++
3 files changed, 54 insertions(+), 4 deletions(-)
diff --git a/man2/getrlimit.2 b/man2/getrlimit.2
index ec464fe..729197e 100644
--- a/man2/getrlimit.2
+++ b/man2/getrlimit.2
@@ -215,7 +215,9 @@ and
and the
.BR mmap (2)
.B MAP_LOCKED
-operation.
+or
+.B MAP_LOCKONFAULT
+operations.
Since Linux 2.6.9 it also affects the
.BR shmctl (2)
.B SHM_LOCK
@@ -232,7 +234,10 @@ locks established by
.BR mlockall (2),
and
.BR mmap (2)
-.BR MAP_LOCKED ;
+with
+.B MAP_LOCKED
+or
+.BR MAP_LOCKONFAULT ;
a process can lock bytes up to this limit in each of these
two categories.
In Linux kernels before 2.6.9, this limit controlled the amount of
diff --git a/man2/mlock.2 b/man2/mlock.2
index b8487ff..139a7be 100644
--- a/man2/mlock.2
+++ b/man2/mlock.2
@@ -96,9 +96,31 @@ process in the future.
These could be for instance new pages required
by a growing heap and stack as well as new memory-mapped files or
shared memory regions.
+.B MCL_FUTURE
+will attempt to make all pages present when the address
+space is allocated.
+.TP
+.BR MCL_ONFAULT " (since Linux 4.2)"
+Like
+.BR MCL_FUTURE ,
+but
+.B MCL_ONFAULT
+does not attempt to make all pages present when the address space is
+allocated, instead wait until each page is accessed for the first
+time before locking. Note that as with the difference between
+.B MAP_LOCKED
+and
+.B MAP_LOCKONFAULT
+for
+.BR mmap "(2),"
+the caller is charged for the entire allocated address space. See
+.BR setrlimit "(2)"
+for more details on resource limits.
.PP
If
.B MCL_FUTURE
+or
+.B MCL_ONFAULT
has been specified, then a later system call (e.g.,
.BR mmap (2),
.BR sbrk (2),
@@ -250,9 +272,11 @@ or when the process terminates.
The
.BR mlockall ()
.B MCL_FUTURE
-setting is not inherited by a child created via
+and
+.B MCL_ONFAULT
+settings are not inherited by a child created via
.BR fork (2)
-and is cleared during an
+and are cleared during an
.BR execve (2).
The memory lock on an address range is automatically removed
diff --git a/man2/mmap.2 b/man2/mmap.2
index a865612..5aa29e9 100644
--- a/man2/mmap.2
+++ b/man2/mmap.2
@@ -277,6 +277,26 @@ of the mapping.
This flag is ignored in older kernels.
.\" If set, the mapped pages will not be swapped out.
.TP
+.BR MAP_LOCKONFAULT " (since Linux 4.2)"
+Lock pages covered by this mapping after they are
+accessed for the first time. Unlike
+.BR MAP_LOCKED ,
+.B MAP_LOCKONFAULT
+does not attempt to populate the mapping immediately. Note that while
+.B MAP_LOCKONFAULT
+does not populate the mapping, the caller is charged for the full mapping
+against
+.B RLIMIT_MEMLOCK
+when the mapping is created. This allows
+.BR mmap "(2)"
+calls to fail the same way for
+.B MAP_LOCKED
+and
+.B MAP_LOCKONFAULT
+when the resource limit would be exceeded. See
+.BR setrlimit "(2)"
+for more details on resource limits.
+.TP
.BR MAP_NONBLOCK " (since Linux 2.5.46)"
Only meaningful in conjunction with
.BR MAP_POPULATE .
@@ -618,6 +638,7 @@ The relevant flags are:
.BR MAP_GROWSDOWN ,
.BR MAP_HUGETLB ,
.BR MAP_LOCKED ,
+.BR MAP_LOCKONFAULT ,
.BR MAP_NONBLOCK ,
.BR MAP_NORESERVE ,
.BR MAP_POPULATE ,
--
1.9.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH V2 3/3] Add tests for lock on fault
From: Eric B Munson @ 2015-06-02 18:13 UTC (permalink / raw)
To: Andrew Morton
Cc: Eric B Munson, Shuah Khan, Michal Hocko, linux-mm, linux-kernel,
linux-api
In-Reply-To: <1433268806-17109-1-git-send-email-emunson@akamai.com>
Test the mmap() flag, the mlockall() flag, and ensure that mlock limits
are respected. Note that the limit test needs to be run a normal user.
Signed-off-by: Eric B Munson <emunson@akamai.com>
Cc: Shuah Khan <shuahkh@osg.samsung.com>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: linux-mm@kvack.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-api@vger.kernel.org
---
tools/testing/selftests/vm/Makefile | 8 +-
tools/testing/selftests/vm/lock-on-fault.c | 145 ++++++++++++++++++++++++++++
tools/testing/selftests/vm/on-fault-limit.c | 47 +++++++++
tools/testing/selftests/vm/run_vmtests | 23 +++++
4 files changed, 222 insertions(+), 1 deletion(-)
create mode 100644 tools/testing/selftests/vm/lock-on-fault.c
create mode 100644 tools/testing/selftests/vm/on-fault-limit.c
diff --git a/tools/testing/selftests/vm/Makefile b/tools/testing/selftests/vm/Makefile
index a5ce953..32f3d20 100644
--- a/tools/testing/selftests/vm/Makefile
+++ b/tools/testing/selftests/vm/Makefile
@@ -1,7 +1,13 @@
# Makefile for vm selftests
CFLAGS = -Wall
-BINARIES = hugepage-mmap hugepage-shm map_hugetlb thuge-gen hugetlbfstest
+BINARIES = hugepage-mmap
+BINARIES += hugepage-shm
+BINARIES += hugetlbfstest
+BINARIES += lock-on-fault
+BINARIES += map_hugetlb
+BINARIES += on-fault-limit
+BINARIES += thuge-gen
BINARIES += transhuge-stress
all: $(BINARIES)
diff --git a/tools/testing/selftests/vm/lock-on-fault.c b/tools/testing/selftests/vm/lock-on-fault.c
new file mode 100644
index 0000000..4659303
--- /dev/null
+++ b/tools/testing/selftests/vm/lock-on-fault.c
@@ -0,0 +1,145 @@
+#include <sys/mman.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <sys/time.h>
+#include <sys/resource.h>
+
+#ifndef MCL_ONFAULT
+#define MCL_ONFAULT (MCL_FUTURE << 1)
+#endif
+
+#define PRESENT_BIT 0x8000000000000000
+#define PFN_MASK 0x007FFFFFFFFFFFFF
+#define UNEVICTABLE_BIT (1UL << 18)
+
+static int check_pageflags(void *map)
+{
+ FILE *file;
+ unsigned long pfn1;
+ unsigned long pfn2;
+ unsigned long offset1;
+ unsigned long offset2;
+ int ret = 1;
+
+ file = fopen("/proc/self/pagemap", "r");
+ if (!file) {
+ perror("fopen");
+ return ret;
+ }
+ offset1 = (unsigned long)map / getpagesize() * sizeof(unsigned long);
+ offset2 = ((unsigned long)map + getpagesize()) / getpagesize() * sizeof(unsigned long);
+ if (fseek(file, offset1, SEEK_SET)) {
+ perror("fseek");
+ goto out;
+ }
+
+ if (fread(&pfn1, sizeof(unsigned long), 1, file) != 1) {
+ perror("fread");
+ goto out;
+ }
+
+ if (fseek(file, offset2, SEEK_SET)) {
+ perror("fseek");
+ goto out;
+ }
+
+ if (fread(&pfn2, sizeof(unsigned long), 1, file) != 1) {
+ perror("fread");
+ goto out;
+ }
+
+ /* pfn2 should not be present */
+ if (pfn2 & PRESENT_BIT) {
+ printf("page map says 0x%lx\n", pfn2);
+ printf("present is 0x%lx\n", PRESENT_BIT);
+ goto out;
+ }
+
+ /* pfn1 should be present */
+ if ((pfn1 & PRESENT_BIT) == 0) {
+ printf("page map says 0x%lx\n", pfn1);
+ printf("present is 0x%lx\n", PRESENT_BIT);
+ goto out;
+ }
+
+ pfn1 &= PFN_MASK;
+ fclose(file);
+ file = fopen("/proc/kpageflags", "r");
+ if (!file) {
+ perror("fopen");
+ munmap(map, 2 * getpagesize());
+ return ret;
+ }
+
+ if (fseek(file, pfn1 * sizeof(unsigned long), SEEK_SET)) {
+ perror("fseek");
+ goto out;
+ }
+
+ if (fread(&pfn2, sizeof(unsigned long), 1, file) != 1) {
+ perror("fread");
+ goto out;
+ }
+
+ /* pfn2 now contains the entry from kpageflags for the first page, the
+ * unevictable bit should be set */
+ if ((pfn2 & UNEVICTABLE_BIT) == 0) {
+ printf("kpageflags says 0x%lx\n", pfn2);
+ printf("unevictable is 0x%lx\n", UNEVICTABLE_BIT);
+ goto out;
+ }
+
+ ret = 0;
+
+out:
+ fclose(file);
+ return ret;
+}
+
+static int test_mmap(int flags)
+{
+ int ret = 1;
+ void *map;
+
+ map = mmap(NULL, 2 * getpagesize(), PROT_READ | PROT_WRITE, flags, 0, 0);
+ if (map == MAP_FAILED) {
+ perror("mmap()");
+ return ret;
+ }
+
+ /* Write something into the first page to ensure it is present */
+ *(char *)map = 1;
+
+ ret = check_pageflags(map);
+
+ munmap(map, 2 * getpagesize());
+ return ret;
+}
+
+static int test_mlockall(void)
+{
+ int ret = 1;
+
+ if (mlockall(MCL_ONFAULT)) {
+ perror("mlockall");
+ return ret;
+ }
+
+ ret = test_mmap(MAP_PRIVATE | MAP_ANONYMOUS);
+ munlockall();
+ return ret;
+}
+
+#ifndef MAP_LOCKONFAULT
+#define MAP_LOCKONFAULT (MAP_HUGETLB << 1)
+#endif
+
+int main(int argc, char **argv)
+{
+ int ret = 0;
+
+ ret += test_mmap(MAP_PRIVATE | MAP_ANONYMOUS | MAP_LOCKONFAULT);
+ ret += test_mlockall();
+ return ret;
+}
diff --git a/tools/testing/selftests/vm/on-fault-limit.c b/tools/testing/selftests/vm/on-fault-limit.c
new file mode 100644
index 0000000..ed2a109
--- /dev/null
+++ b/tools/testing/selftests/vm/on-fault-limit.c
@@ -0,0 +1,47 @@
+#include <sys/mman.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <sys/time.h>
+#include <sys/resource.h>
+
+#ifndef MCL_ONFAULT
+#define MCL_ONFAULT (MCL_FUTURE << 1)
+#endif
+
+static int test_limit(void)
+{
+ int ret = 1;
+ struct rlimit lims;
+ void *map;
+
+ if (getrlimit(RLIMIT_MEMLOCK, &lims)) {
+ perror("getrlimit");
+ return ret;
+ }
+
+ if (mlockall(MCL_ONFAULT)) {
+ perror("mlockall");
+ return ret;
+ }
+
+ map = mmap(NULL, 2 * lims.rlim_max, PROT_READ | PROT_WRITE,
+ MAP_PRIVATE | MAP_ANONYMOUS | MAP_POPULATE, 0, 0);
+ if (map != MAP_FAILED)
+ printf("mmap should have failed, but didn't\n");
+ else {
+ ret = 0;
+ munmap(map, 2 * lims.rlim_max);
+ }
+
+ munlockall();
+ return ret;
+}
+
+int main(int argc, char **argv)
+{
+ int ret = 0;
+
+ ret += test_limit();
+ return ret;
+}
diff --git a/tools/testing/selftests/vm/run_vmtests b/tools/testing/selftests/vm/run_vmtests
index c87b681..c1aecce 100755
--- a/tools/testing/selftests/vm/run_vmtests
+++ b/tools/testing/selftests/vm/run_vmtests
@@ -90,4 +90,27 @@ fi
umount $mnt
rm -rf $mnt
echo $nr_hugepgs > /proc/sys/vm/nr_hugepages
+
+echo "--------------------"
+echo "running lock-on-fault"
+echo "--------------------"
+./lock-on-fault
+if [ $? -ne 0 ]; then
+ echo "[FAIL]"
+ exitcode=1
+else
+ echo "[PASS]"
+fi
+
+echo "--------------------"
+echo "running on-fault-limit"
+echo "--------------------"
+sudo -u nobody ./on-fault-limit
+if [ $? -ne 0 ]; then
+ echo "[FAIL]"
+ exitcode=1
+else
+ echo "[PASS]"
+fi
+
exit $exitcode
--
1.9.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH V2 2/3] Add mlockall flag for locking pages on fault
From: Eric B Munson @ 2015-06-02 18:13 UTC (permalink / raw)
To: Andrew Morton
Cc: Eric B Munson, Michal Hocko, Michael Kerrisk, linux-alpha,
linux-kernel, linux-mips, linux-parisc, linuxppc-dev, sparclinux,
linux-xtensa, linux-arch, linux-api, linux-mm
In-Reply-To: <1433268806-17109-1-git-send-email-emunson@akamai.com>
Building on the previous patch, extend mlockall() to give a process a
way to specify that pages should be locked when they are faulted in, but
that pre-faulting is not needed.
MCL_ONFAULT is preferrable to MCL_FUTURE for the use cases enumerated
in the previous patch becuase MCL_FUTURE will behave as if each mapping
was made with MAP_LOCKED, causing the entire mapping to be faulted in
when new space is allocated or mapped. MCL_ONFAULT allows the user to
delay the fault in cost of any given page until it is actually needed,
but then guarantees that that page will always be resident.
As with the mmap(MAP_LOCKONFAULT) case, the user is charged for the
mapping against the RLIMIT_MEMLOCK when the address space is allocated,
not when the page is faulted in. This decision was made to keep the
accounting checks out of the page fault path.
Signed-off-by: Eric B Munson <emunson@akamai.com>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: Michael Kerrisk <mtk.manpages@gmail.com>
Cc: linux-alpha@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-mips@linux-mips.org
Cc: linux-parisc@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: sparclinux@vger.kernel.org
Cc: linux-xtensa@linux-xtensa.org
Cc: linux-arch@vger.kernel.org
Cc: linux-api@vger.kernel.org
Cc: linux-mm@kvack.org
---
arch/alpha/include/uapi/asm/mman.h | 1 +
arch/mips/include/uapi/asm/mman.h | 1 +
arch/parisc/include/uapi/asm/mman.h | 1 +
arch/powerpc/include/uapi/asm/mman.h | 1 +
arch/sparc/include/uapi/asm/mman.h | 1 +
arch/tile/include/uapi/asm/mman.h | 1 +
arch/xtensa/include/uapi/asm/mman.h | 1 +
include/uapi/asm-generic/mman.h | 1 +
mm/mlock.c | 13 +++++++++----
9 files changed, 17 insertions(+), 4 deletions(-)
diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
index 15e96e1..dfdaecf 100644
--- a/arch/alpha/include/uapi/asm/mman.h
+++ b/arch/alpha/include/uapi/asm/mman.h
@@ -38,6 +38,7 @@
#define MCL_CURRENT 8192 /* lock all currently mapped pages */
#define MCL_FUTURE 16384 /* lock all additions to address space */
+#define MCL_ONFAULT 32768 /* lock all pages that are faulted in */
#define MADV_NORMAL 0 /* no further special treatment */
#define MADV_RANDOM 1 /* expect random page references */
diff --git a/arch/mips/include/uapi/asm/mman.h b/arch/mips/include/uapi/asm/mman.h
index 47846a5..f0705ff 100644
--- a/arch/mips/include/uapi/asm/mman.h
+++ b/arch/mips/include/uapi/asm/mman.h
@@ -62,6 +62,7 @@
*/
#define MCL_CURRENT 1 /* lock all current mappings */
#define MCL_FUTURE 2 /* lock all future mappings */
+#define MCL_ONFAULT 4 /* lock all pages that are faulted in */
#define MADV_NORMAL 0 /* no further special treatment */
#define MADV_RANDOM 1 /* expect random page references */
diff --git a/arch/parisc/include/uapi/asm/mman.h b/arch/parisc/include/uapi/asm/mman.h
index 1514cd7..7c2eb85 100644
--- a/arch/parisc/include/uapi/asm/mman.h
+++ b/arch/parisc/include/uapi/asm/mman.h
@@ -32,6 +32,7 @@
#define MCL_CURRENT 1 /* lock all current mappings */
#define MCL_FUTURE 2 /* lock all future mappings */
+#define MCL_ONFAULT 4 /* lock all pages that are faulted in */
#define MADV_NORMAL 0 /* no further special treatment */
#define MADV_RANDOM 1 /* expect random page references */
diff --git a/arch/powerpc/include/uapi/asm/mman.h b/arch/powerpc/include/uapi/asm/mman.h
index fce74fe..0109937 100644
--- a/arch/powerpc/include/uapi/asm/mman.h
+++ b/arch/powerpc/include/uapi/asm/mman.h
@@ -22,6 +22,7 @@
#define MCL_CURRENT 0x2000 /* lock all currently mapped pages */
#define MCL_FUTURE 0x4000 /* lock all additions to address space */
+#define MCL_ONFAULT 0x80000 /* lock all pages that are faulted in */
#define MAP_POPULATE 0x8000 /* populate (prefault) pagetables */
#define MAP_NONBLOCK 0x10000 /* do not block on IO */
diff --git a/arch/sparc/include/uapi/asm/mman.h b/arch/sparc/include/uapi/asm/mman.h
index 12425d8..f2986f7 100644
--- a/arch/sparc/include/uapi/asm/mman.h
+++ b/arch/sparc/include/uapi/asm/mman.h
@@ -17,6 +17,7 @@
#define MCL_CURRENT 0x2000 /* lock all currently mapped pages */
#define MCL_FUTURE 0x4000 /* lock all additions to address space */
+#define MCL_ONFAULT 0x80000 /* lock all pages that are faulted in */
#define MAP_POPULATE 0x8000 /* populate (prefault) pagetables */
#define MAP_NONBLOCK 0x10000 /* do not block on IO */
diff --git a/arch/tile/include/uapi/asm/mman.h b/arch/tile/include/uapi/asm/mman.h
index ec04eaf..0f7ae45 100644
--- a/arch/tile/include/uapi/asm/mman.h
+++ b/arch/tile/include/uapi/asm/mman.h
@@ -37,6 +37,7 @@
*/
#define MCL_CURRENT 1 /* lock all current mappings */
#define MCL_FUTURE 2 /* lock all future mappings */
+#define MCL_ONFAULT 4 /* lock all pages that are faulted in */
#endif /* _ASM_TILE_MMAN_H */
diff --git a/arch/xtensa/include/uapi/asm/mman.h b/arch/xtensa/include/uapi/asm/mman.h
index 42d43cc..10fbbb7 100644
--- a/arch/xtensa/include/uapi/asm/mman.h
+++ b/arch/xtensa/include/uapi/asm/mman.h
@@ -75,6 +75,7 @@
*/
#define MCL_CURRENT 1 /* lock all current mappings */
#define MCL_FUTURE 2 /* lock all future mappings */
+#define MCL_ONFAULT 4 /* lock all pages that are faulted in */
#define MADV_NORMAL 0 /* no further special treatment */
#define MADV_RANDOM 1 /* expect random page references */
diff --git a/include/uapi/asm-generic/mman.h b/include/uapi/asm-generic/mman.h
index fc4e586..7fb729b 100644
--- a/include/uapi/asm-generic/mman.h
+++ b/include/uapi/asm-generic/mman.h
@@ -18,5 +18,6 @@
#define MCL_CURRENT 1 /* lock all current mappings */
#define MCL_FUTURE 2 /* lock all future mappings */
+#define MCL_ONFAULT 4 /* lock all pages that are faulted in */
#endif /* __ASM_GENERIC_MMAN_H */
diff --git a/mm/mlock.c b/mm/mlock.c
index 6fd2cf1..f15547f 100644
--- a/mm/mlock.c
+++ b/mm/mlock.c
@@ -579,7 +579,7 @@ static int do_mlock(unsigned long start, size_t len, int on)
/* Here we know that vma->vm_start <= nstart < vma->vm_end. */
- newflags = vma->vm_flags & ~VM_LOCKED;
+ newflags = vma->vm_flags & ~(VM_LOCKED | VM_LOCKONFAULT);
if (on)
newflags |= VM_LOCKED;
@@ -662,13 +662,17 @@ static int do_mlockall(int flags)
current->mm->def_flags |= VM_LOCKED;
else
current->mm->def_flags &= ~VM_LOCKED;
- if (flags == MCL_FUTURE)
+ if (flags & MCL_ONFAULT)
+ current->mm->def_flags |= VM_LOCKONFAULT;
+ else
+ current->mm->def_flags &= ~VM_LOCKONFAULT;
+ if (flags == MCL_FUTURE || flags == MCL_ONFAULT)
goto out;
for (vma = current->mm->mmap; vma ; vma = prev->vm_next) {
vm_flags_t newflags;
- newflags = vma->vm_flags & ~VM_LOCKED;
+ newflags = vma->vm_flags & ~(VM_LOCKED | VM_LOCKONFAULT);
if (flags & MCL_CURRENT)
newflags |= VM_LOCKED;
@@ -685,7 +689,8 @@ SYSCALL_DEFINE1(mlockall, int, flags)
unsigned long lock_limit;
int ret = -EINVAL;
- if (!flags || (flags & ~(MCL_CURRENT | MCL_FUTURE)))
+ if (!flags || (flags & ~(MCL_CURRENT | MCL_FUTURE | MCL_ONFAULT)) ||
+ ((flags & MCL_FUTURE) && (flags & MCL_ONFAULT)))
goto out;
ret = -EPERM;
--
1.9.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH V2 1/3] Add mmap flag to request pages are locked after page fault
From: Eric B Munson @ 2015-06-02 18:13 UTC (permalink / raw)
To: Andrew Morton
Cc: Eric B Munson, Michal Hocko, Michael Kerrisk, linux-alpha,
linux-kernel, linux-mips, linux-parisc, linuxppc-dev, sparclinux,
linux-xtensa, linux-mm, linux-arch, linux-api
In-Reply-To: <1433268806-17109-1-git-send-email-emunson@akamai.com>
The cost of faulting in all memory to be locked can be very high when
working with large mappings. If only portions of the mapping will be
used this can incur a high penalty for locking.
For the example of a large file, this is the usage pattern for a large
statical language model (probably applies to other statical or graphical
models as well). For the security example, any application transacting
in data that cannot be swapped out (credit card data, medical records,
etc).
This patch introduces the ability to request that pages are not
pre-faulted, but are placed on the unevictable LRU when they are finally
faulted in.
To keep accounting checks out of the page fault path, users are billed
for the entire mapping lock as if MAP_LOCKED was used.
Signed-off-by: Eric B Munson <emunson@akamai.com>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: Michael Kerrisk <mtk.manpages@gmail.com>
Cc: linux-alpha@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-mips@linux-mips.org
Cc: linux-parisc@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: sparclinux@vger.kernel.org
Cc: linux-xtensa@linux-xtensa.org
Cc: linux-mm@kvack.org
Cc: linux-arch@vger.kernel.org
Cc: linux-api@vger.kernel.org
---
arch/alpha/include/uapi/asm/mman.h | 1 +
arch/mips/include/uapi/asm/mman.h | 1 +
arch/parisc/include/uapi/asm/mman.h | 1 +
arch/powerpc/include/uapi/asm/mman.h | 1 +
arch/sparc/include/uapi/asm/mman.h | 1 +
arch/tile/include/uapi/asm/mman.h | 1 +
arch/xtensa/include/uapi/asm/mman.h | 1 +
include/linux/mm.h | 1 +
include/linux/mman.h | 3 ++-
include/uapi/asm-generic/mman.h | 1 +
mm/mmap.c | 4 ++--
mm/swap.c | 3 ++-
12 files changed, 15 insertions(+), 4 deletions(-)
diff --git a/arch/alpha/include/uapi/asm/mman.h b/arch/alpha/include/uapi/asm/mman.h
index 0086b47..15e96e1 100644
--- a/arch/alpha/include/uapi/asm/mman.h
+++ b/arch/alpha/include/uapi/asm/mman.h
@@ -30,6 +30,7 @@
#define MAP_NONBLOCK 0x40000 /* do not block on IO */
#define MAP_STACK 0x80000 /* give out an address that is best suited for process/thread stacks */
#define MAP_HUGETLB 0x100000 /* create a huge page mapping */
+#define MAP_LOCKONFAULT 0x200000 /* Lock pages after they are faulted in, do not prefault */
#define MS_ASYNC 1 /* sync memory asynchronously */
#define MS_SYNC 2 /* synchronous memory sync */
diff --git a/arch/mips/include/uapi/asm/mman.h b/arch/mips/include/uapi/asm/mman.h
index cfcb876..47846a5 100644
--- a/arch/mips/include/uapi/asm/mman.h
+++ b/arch/mips/include/uapi/asm/mman.h
@@ -48,6 +48,7 @@
#define MAP_NONBLOCK 0x20000 /* do not block on IO */
#define MAP_STACK 0x40000 /* give out an address that is best suited for process/thread stacks */
#define MAP_HUGETLB 0x80000 /* create a huge page mapping */
+#define MAP_LOCKONFAULT 0x100000 /* Lock pages after they are faulted in, do not prefault */
/*
* Flags for msync
diff --git a/arch/parisc/include/uapi/asm/mman.h b/arch/parisc/include/uapi/asm/mman.h
index 294d251..1514cd7 100644
--- a/arch/parisc/include/uapi/asm/mman.h
+++ b/arch/parisc/include/uapi/asm/mman.h
@@ -24,6 +24,7 @@
#define MAP_NONBLOCK 0x20000 /* do not block on IO */
#define MAP_STACK 0x40000 /* give out an address that is best suited for process/thread stacks */
#define MAP_HUGETLB 0x80000 /* create a huge page mapping */
+#define MAP_LOCKONFAULT 0x100000 /* Lock pages after they are faulted in, do not prefault */
#define MS_SYNC 1 /* synchronous memory sync */
#define MS_ASYNC 2 /* sync memory asynchronously */
diff --git a/arch/powerpc/include/uapi/asm/mman.h b/arch/powerpc/include/uapi/asm/mman.h
index 6ea26df..fce74fe 100644
--- a/arch/powerpc/include/uapi/asm/mman.h
+++ b/arch/powerpc/include/uapi/asm/mman.h
@@ -27,5 +27,6 @@
#define MAP_NONBLOCK 0x10000 /* do not block on IO */
#define MAP_STACK 0x20000 /* give out an address that is best suited for process/thread stacks */
#define MAP_HUGETLB 0x40000 /* create a huge page mapping */
+#define MAP_LOCKONFAULT 0x80000 /* Lock pages after they are faulted in, do not prefault */
#endif /* _UAPI_ASM_POWERPC_MMAN_H */
diff --git a/arch/sparc/include/uapi/asm/mman.h b/arch/sparc/include/uapi/asm/mman.h
index 0b14df3..12425d8 100644
--- a/arch/sparc/include/uapi/asm/mman.h
+++ b/arch/sparc/include/uapi/asm/mman.h
@@ -22,6 +22,7 @@
#define MAP_NONBLOCK 0x10000 /* do not block on IO */
#define MAP_STACK 0x20000 /* give out an address that is best suited for process/thread stacks */
#define MAP_HUGETLB 0x40000 /* create a huge page mapping */
+#define MAP_LOCKONFAULT 0x80000 /* Lock pages after they are faulted in, do not prefault */
#endif /* _UAPI__SPARC_MMAN_H__ */
diff --git a/arch/tile/include/uapi/asm/mman.h b/arch/tile/include/uapi/asm/mman.h
index 81b8fc3..ec04eaf 100644
--- a/arch/tile/include/uapi/asm/mman.h
+++ b/arch/tile/include/uapi/asm/mman.h
@@ -29,6 +29,7 @@
#define MAP_DENYWRITE 0x0800 /* ETXTBSY */
#define MAP_EXECUTABLE 0x1000 /* mark it as an executable */
#define MAP_HUGETLB 0x4000 /* create a huge page mapping */
+#define MAP_LOCKONFAULT 0x8000 /* Lock pages after they are faulted in, do not prefault */
/*
diff --git a/arch/xtensa/include/uapi/asm/mman.h b/arch/xtensa/include/uapi/asm/mman.h
index 201aec0..42d43cc 100644
--- a/arch/xtensa/include/uapi/asm/mman.h
+++ b/arch/xtensa/include/uapi/asm/mman.h
@@ -55,6 +55,7 @@
#define MAP_NONBLOCK 0x20000 /* do not block on IO */
#define MAP_STACK 0x40000 /* give out an address that is best suited for process/thread stacks */
#define MAP_HUGETLB 0x80000 /* create a huge page mapping */
+#define MAP_LOCKONFAULT 0x100000 /* Lock pages after they are faulted in, do not prefault */
#ifdef CONFIG_MMAP_ALLOW_UNINITIALIZED
# define MAP_UNINITIALIZED 0x4000000 /* For anonymous mmap, memory could be
* uninitialized */
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 0755b9f..3e31457 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -126,6 +126,7 @@ extern unsigned int kobjsize(const void *objp);
#define VM_PFNMAP 0x00000400 /* Page-ranges managed without "struct page", just pure PFN */
#define VM_DENYWRITE 0x00000800 /* ETXTBSY on write attempts.. */
+#define VM_LOCKONFAULT 0x00001000 /* Lock the pages covered when they are faulted in */
#define VM_LOCKED 0x00002000
#define VM_IO 0x00004000 /* Memory mapped I/O or similar */
diff --git a/include/linux/mman.h b/include/linux/mman.h
index 16373c8..437264b 100644
--- a/include/linux/mman.h
+++ b/include/linux/mman.h
@@ -86,7 +86,8 @@ calc_vm_flag_bits(unsigned long flags)
{
return _calc_vm_trans(flags, MAP_GROWSDOWN, VM_GROWSDOWN ) |
_calc_vm_trans(flags, MAP_DENYWRITE, VM_DENYWRITE ) |
- _calc_vm_trans(flags, MAP_LOCKED, VM_LOCKED );
+ _calc_vm_trans(flags, MAP_LOCKED, VM_LOCKED ) |
+ _calc_vm_trans(flags, MAP_LOCKONFAULT,VM_LOCKONFAULT);
}
unsigned long vm_commit_limit(void);
diff --git a/include/uapi/asm-generic/mman.h b/include/uapi/asm-generic/mman.h
index e9fe6fd..fc4e586 100644
--- a/include/uapi/asm-generic/mman.h
+++ b/include/uapi/asm-generic/mman.h
@@ -12,6 +12,7 @@
#define MAP_NONBLOCK 0x10000 /* do not block on IO */
#define MAP_STACK 0x20000 /* give out an address that is best suited for process/thread stacks */
#define MAP_HUGETLB 0x40000 /* create a huge page mapping */
+#define MAP_LOCKONFAULT 0x80000 /* Lock pages after they are faulted in, do not prefault */
/* Bits [26:31] are reserved, see mman-common.h for MAP_HUGETLB usage */
diff --git a/mm/mmap.c b/mm/mmap.c
index bb50cac..ba1a6bf 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -1233,7 +1233,7 @@ static inline int mlock_future_check(struct mm_struct *mm,
unsigned long locked, lock_limit;
/* mlock MCL_FUTURE? */
- if (flags & VM_LOCKED) {
+ if (flags & (VM_LOCKED | VM_LOCKONFAULT)) {
locked = len >> PAGE_SHIFT;
locked += mm->locked_vm;
lock_limit = rlimit(RLIMIT_MEMLOCK);
@@ -1301,7 +1301,7 @@ unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
vm_flags = calc_vm_prot_bits(prot) | calc_vm_flag_bits(flags) |
mm->def_flags | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC;
- if (flags & MAP_LOCKED)
+ if (flags & (MAP_LOCKED | MAP_LOCKONFAULT))
if (!can_do_mlock())
return -EPERM;
diff --git a/mm/swap.c b/mm/swap.c
index a7251a8..07c905e 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -711,7 +711,8 @@ void lru_cache_add_active_or_unevictable(struct page *page,
{
VM_BUG_ON_PAGE(PageLRU(page), page);
- if (likely((vma->vm_flags & (VM_LOCKED | VM_SPECIAL)) != VM_LOCKED)) {
+ if (likely((vma->vm_flags & (VM_LOCKED | VM_LOCKONFAULT)) == 0) ||
+ (vma->vm_flags & VM_SPECIAL)) {
SetPageActive(page);
lru_cache_add(page);
return;
--
1.9.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH V2 0/3] Allow user to request memory to be locked on page fault
From: Eric B Munson @ 2015-06-02 18:13 UTC (permalink / raw)
To: Andrew Morton
Cc: Eric B Munson, Shuah Khan, Michal Hocko, Michael Kerrisk,
linux-alpha, linux-kernel, linux-mips, linux-parisc, linuxppc-dev,
sparclinux, linux-xtensa, linux-mm, linux-arch, linux-api
mlock() allows a user to control page out of program memory, but this
comes at the cost of faulting in the entire mapping when it is
allocated. For large mappings where the entire area is not necessary
this is not ideal.
This series introduces new flags for mmap() and mlockall() that allow a
user to specify that the covered are should not be paged out, but only
after the memory has been used the first time.
There are two main use cases that this set covers. The first is the
security focussed mlock case. A buffer is needed that cannot be written
to swap. The maximum size is known, but on average the memory used is
significantly less than this maximum. With lock on fault, the buffer
is guaranteed to never be paged out without consuming the maximum size
every time such a buffer is created.
The second use case is focussed on performance. Portions of a large
file are needed and we want to keep the used portions in memory once
accessed. This is the case for large graphical models where the path
through the graph is not known until run time. The entire graph is
unlikely to be used in a given invocation, but once a node has been
used it needs to stay resident for further processing. Given these
constraints we have a number of options. We can potentially waste a
large amount of memory by mlocking the entire region (this can also
cause a significant stall at startup as the entire file is read in).
We can mlock every page as we access them without tracking if the page
is already resident but this introduces large overhead for each access.
The third option is mapping the entire region with PROT_NONE and using
a signal handler for SIGSEGV to mprotect(PROT_READ) and mlock() the
needed page. Doing this page at a time adds a significant performance
penalty. Batching can be used to mitigate this overhead, but in order
to safely avoid trying to mprotect pages outside of the mapping, the
boundaries of each mapping to be used in this way must be tracked and
available to the signal handler. This is precisely what the mm system
in the kernel should already be doing.
For mmap(MAP_LOCKONFAULT) the user is charged against RLIMIT_MEMLOCK
as if MAP_LOCKED was used, so when the VMA is created not when the pages
are faulted in. For mlockall(MCL_ON_FAULT) the user is charged as if
MCL_FUTURE was used. This decision was made to keep the accounting
checks out of the page fault path.
To illustrate the benefit of this patch I wrote a test program that
mmaps a 5 GB file filled with random data and then makes 15,000,000
accesses to random addresses in that mapping. The test program was run
20 times for each setup. Results are reported for two program portions,
setup and execution. The setup phase is calling mmap and optionally
mlock on the entire region. For most experiments this is trivial, but
it highlights the cost of faulting in the entire region. Results are
averages across the 20 runs in milliseconds.
mmap with MAP_LOCKED:
Setup avg: 11821.193
Processing avg: 3404.286
mmap with mlock() before each access:
Setup avg: 0.054
Processing avg: 34263.201
mmap with PROT_NONE and signal handler and batch size of 1 page:
With the default value in max_map_count, this gets ENOMEM as I attempt
to change the permissions, after upping the sysctl significantly I get:
Setup avg: 0.050
Processing avg: 67690.625
mmap with PROT_NONE and signal handler and batch size of 8 pages:
Setup avg: 0.098
Processing avg: 37344.197
mmap with PROT_NONE and signal handler and batch size of 16 pages:
Setup avg: 0.0548
Processing avg: 29295.669
mmap with MAP_LOCKONFAULT:
Setup avg: 0.073
Processing avg: 18392.136
The signal handler in the batch cases faulted in memory in two steps to
avoid having to know the start and end of the faulting mapping. The
first step covers the page that caused the fault as we know that it will
be possible to lock. The second step speculatively tries to mlock and
mprotect the batch size - 1 pages that follow. There may be a clever
way to avoid this without having the program track each mapping to be
covered by this handeler in a globally accessible structure, but I could
not find it. It should be noted that with a large enough batch size
this two step fault handler can still cause the program to crash if it
reaches far beyond the end of the mapping.
These results show that if the developer knows that a majority of the
mapping will be used, it is better to try and fault it in at once,
otherwise MAP_LOCKONFAULT is significantly faster.
The performance cost of these patches are minimal on the two benchmarks
I have tested (stream and kernbench). The following are the average
values across 20 runs of each benchmark after a warmup run whose
results were discarded.
Avg throughput in MB/s from stream using 1000000 element arrays
Test 4.1-rc2 4.1-rc2+lock-on-fault
Copy: 10,979.08 10,917.34
Scale: 11,094.45 11,023.01
Add: 12,487.29 12,388.65
Triad: 12,505.77 12,418.78
Kernbench optimal load
4.1-rc2 4.1-rc2+lock-on-fault
Elapsed Time 71.046 71.324
User Time 62.117 62.352
System Time 8.926 8.969
Context Switches 14531.9 14542.5
Sleeps 14935.9 14939
Eric B Munson (3):
Add mmap flag to request pages are locked after page fault
Add mlockall flag for locking pages on fault
Add tests for lock on fault
arch/alpha/include/uapi/asm/mman.h | 2 +
arch/mips/include/uapi/asm/mman.h | 2 +
arch/parisc/include/uapi/asm/mman.h | 2 +
arch/powerpc/include/uapi/asm/mman.h | 2 +
arch/sparc/include/uapi/asm/mman.h | 2 +
arch/tile/include/uapi/asm/mman.h | 2 +
arch/xtensa/include/uapi/asm/mman.h | 2 +
include/linux/mm.h | 1 +
include/linux/mman.h | 3 +-
include/uapi/asm-generic/mman.h | 2 +
mm/mlock.c | 13 ++-
mm/mmap.c | 4 +-
mm/swap.c | 3 +-
tools/testing/selftests/vm/Makefile | 8 +-
tools/testing/selftests/vm/lock-on-fault.c | 145 ++++++++++++++++++++++++++++
tools/testing/selftests/vm/on-fault-limit.c | 47 +++++++++
tools/testing/selftests/vm/run_vmtests | 23 +++++
17 files changed, 254 insertions(+), 9 deletions(-)
create mode 100644 tools/testing/selftests/vm/lock-on-fault.c
create mode 100644 tools/testing/selftests/vm/on-fault-limit.c
Cc: Shuah Khan <shuahkh@osg.samsung.com>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: Michael Kerrisk <mtk.manpages@gmail.com>
Cc: linux-alpha@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-mips@linux-mips.org
Cc: linux-parisc@vger.kernel.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: sparclinux@vger.kernel.org
Cc: linux-xtensa@linux-xtensa.org
Cc: linux-mm@kvack.org
Cc: linux-arch@vger.kernel.org
Cc: linux-api@vger.kernel.org
--
1.9.1
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* UAPI headers including non-UAPI headers by accident?
From: Andy Lutomirski @ 2015-06-02 16:36 UTC (permalink / raw)
To: linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Linux API
include/uapi/linux/signal.h starts with:
#ifndef _UAPI_LINUX_SIGNAL_H
#define _UAPI_LINUX_SIGNAL_H
#include <asm/signal.h>
#include <asm/siginfo.h>
This causes it to include <asm/signal.h>, which is not the same thing
as <uapi/asm/signal.h>. Changing that will break userspace use of
this header, though, as the uapi/ won't get removed.
What's the correct fix? This is causing trouble with a UML build for me.
Thanks,
Andy
^ permalink raw reply
* Re: [RESEND PATCH 3/3] Add tests for lock on fault
From: Eric B Munson @ 2015-06-02 16:05 UTC (permalink / raw)
To: Shuah Khan; +Cc: Andrew Morton, linux-mm, linux-kernel, linux-api
In-Reply-To: <556DCB05.1010102@osg.samsung.com>
[-- Attachment #1: Type: text/plain, Size: 2136 bytes --]
On Tue, 02 Jun 2015, Shuah Khan wrote:
> On 05/29/2015 08:13 AM, Eric B Munson wrote:
> > Test the mmap() flag, the mlockall() flag, and ensure that mlock limits
> > are respected. Note that the limit test needs to be run a normal user.
> >
> > Signed-off-by: Eric B Munson <emunson@akamai.com>
> > Cc: Shuah Khan <shuahkh@osg.samsung.com>
> > Cc: linux-mm@kvack.org
> > Cc: linux-kernel@vger.kernel.org
> > Cc: linux-api@vger.kernel.org
> > ---
> > tools/testing/selftests/vm/Makefile | 8 +-
> > tools/testing/selftests/vm/lock-on-fault.c | 145 ++++++++++++++++++++++++++++
> > tools/testing/selftests/vm/on-fault-limit.c | 47 +++++++++
> > tools/testing/selftests/vm/run_vmtests | 23 +++++
> > 4 files changed, 222 insertions(+), 1 deletion(-)
> > create mode 100644 tools/testing/selftests/vm/lock-on-fault.c
> > create mode 100644 tools/testing/selftests/vm/on-fault-limit.c
> >
> > diff --git a/tools/testing/selftests/vm/Makefile b/tools/testing/selftests/vm/Makefile
> > index a5ce953..32f3d20 100644
> > --- a/tools/testing/selftests/vm/Makefile
> > +++ b/tools/testing/selftests/vm/Makefile
> > @@ -1,7 +1,13 @@
> > # Makefile for vm selftests
> >
> > CFLAGS = -Wall
> > -BINARIES = hugepage-mmap hugepage-shm map_hugetlb thuge-gen hugetlbfstest
> > +BINARIES = hugepage-mmap
> > +BINARIES += hugepage-shm
> > +BINARIES += hugetlbfstest
> > +BINARIES += lock-on-fault
> > +BINARIES += map_hugetlb
> > +BINARIES += on-fault-limit
> > +BINARIES += thuge-gen
> > BINARIES += transhuge-stress
> >
> > all: $(BINARIES)
> > diff --git a/tools/testing/selftests/vm/lock-on-fault.c b/tools/testing/selftests/vm/lock-on-fault.c
> > new file mode 100644
> > index 0000000..e6a9688
>
> Hi Eric,
>
> Could you please make sure make kselftest run from kernel main
> Makefile works and tools/testing/selftests/kselftest_install.sh
> works. For now you have to be in tools/testing/selftests to run
> kselftest_install.sh
>
I am working on getting V2 ready, I will ensure these work and make any
necessary adjustments before sending V2 out.
Eric
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH] seccomp: add ptrace commands for suspend/resume
From: Tycho Andersen @ 2015-06-02 15:46 UTC (permalink / raw)
To: Andy Lutomirski
Cc: linux-kernel@vger.kernel.org, Linux API, Kees Cook, Will Drewry,
Roland McGrath, Oleg Nesterov, Pavel Emelyanov, Serge E. Hallyn
In-Reply-To: <20150601201233.GC2818@hopstrocity>
Hi Andy,
On Mon, Jun 01, 2015 at 02:12:33PM -0600, Tycho Andersen wrote:
> On Mon, Jun 01, 2015 at 12:51:12PM -0700, Andy Lutomirski wrote:
> > On Mon, Jun 1, 2015 at 12:47 PM, Tycho Andersen
> > <tycho.andersen@canonical.com> wrote:
> > > On Mon, Jun 01, 2015 at 12:38:57PM -0700, Andy Lutomirski wrote:
> > >> On Mon, Jun 1, 2015 at 12:28 PM, Tycho Andersen
> > >> > +int resume_seccomp(struct task_struct *task)
> > >> > +{
> > >> > + int ret = -EACCES;
> > >> > +
> > >> > + spin_lock_irq(&task->sighand->siglock);
> > >> > +
> > >> > + if (!capable(CAP_SYS_ADMIN))
> > >> > + goto out;
> > >> > +
> > >> > + task->seccomp.suspended = false;
> > >> > +
> > >> > +#ifdef TIF_NOTSC
> > >> > + if (task->seccomp.mode == SECCOMP_MODE_STRICT)
> > >> > + set_tsk_thread_flag(task, TIF_NOTSC);
> > >> > +#endif
> > >>
> > >> Ditto. Or can the task not be running here?
> > >
> > > It is stopped since ptrace requires it to be stopped; I don't know if
> > > that's enough to guarantee correctness, though. Is there some
> > > additional barrier that is needed?
> >
> > Dunno. Does ptrace actually guarantee that for new operations?
>
> It seems to; it kept giving me ESRCH when I didn't wait for it to
> stop. I'll poke around and see if I can confirm this via the code.
It looks to me like ptrace does guarantee this. The commands that
don't require a task to be stopped are all special cases in the ptrace
syscall definition, and anything that's not one of those is protected
by a ptrace_check_attach(), which IIUC enforces that the task is
stopped.
Tycho
^ permalink raw reply
* Re: [RESEND PATCH 3/3] Add tests for lock on fault
From: Shuah Khan @ 2015-06-02 15:25 UTC (permalink / raw)
To: Eric B Munson, Andrew Morton
Cc: linux-mm-Bw31MaZKKs3YtjvyW6yDsg,
linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-api-u79uwXL29TY76Z2rM5mHXA, Shuah Khan
In-Reply-To: <1432908808-31150-4-git-send-email-emunson-JqFfY2XvxFXQT0dZR+AlfA@public.gmane.org>
On 05/29/2015 08:13 AM, Eric B Munson wrote:
> Test the mmap() flag, the mlockall() flag, and ensure that mlock limits
> are respected. Note that the limit test needs to be run a normal user.
>
> Signed-off-by: Eric B Munson <emunson-JqFfY2XvxFXQT0dZR+AlfA@public.gmane.org>
> Cc: Shuah Khan <shuahkh-JPH+aEBZ4P+UEJcrhfAQsw@public.gmane.org>
> Cc: linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org
> Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> Cc: linux-api-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> ---
> tools/testing/selftests/vm/Makefile | 8 +-
> tools/testing/selftests/vm/lock-on-fault.c | 145 ++++++++++++++++++++++++++++
> tools/testing/selftests/vm/on-fault-limit.c | 47 +++++++++
> tools/testing/selftests/vm/run_vmtests | 23 +++++
> 4 files changed, 222 insertions(+), 1 deletion(-)
> create mode 100644 tools/testing/selftests/vm/lock-on-fault.c
> create mode 100644 tools/testing/selftests/vm/on-fault-limit.c
>
> diff --git a/tools/testing/selftests/vm/Makefile b/tools/testing/selftests/vm/Makefile
> index a5ce953..32f3d20 100644
> --- a/tools/testing/selftests/vm/Makefile
> +++ b/tools/testing/selftests/vm/Makefile
> @@ -1,7 +1,13 @@
> # Makefile for vm selftests
>
> CFLAGS = -Wall
> -BINARIES = hugepage-mmap hugepage-shm map_hugetlb thuge-gen hugetlbfstest
> +BINARIES = hugepage-mmap
> +BINARIES += hugepage-shm
> +BINARIES += hugetlbfstest
> +BINARIES += lock-on-fault
> +BINARIES += map_hugetlb
> +BINARIES += on-fault-limit
> +BINARIES += thuge-gen
> BINARIES += transhuge-stress
>
> all: $(BINARIES)
> diff --git a/tools/testing/selftests/vm/lock-on-fault.c b/tools/testing/selftests/vm/lock-on-fault.c
> new file mode 100644
> index 0000000..e6a9688
Hi Eric,
Could you please make sure make kselftest run from kernel main
Makefile works and tools/testing/selftests/kselftest_install.sh
works. For now you have to be in tools/testing/selftests to run
kselftest_install.sh
thanks,
-- Shuah
--
Shuah Khan
Sr. Linux Kernel Developer
Open Source Innovation Group
Samsung Research America (Silicon Valley)
shuahkh-JPH+aEBZ4P+UEJcrhfAQsw@public.gmane.org | (970) 217-8978
^ permalink raw reply
* Re: [RESEND PATCH 0/3] Allow user to request memory to be locked on page fault
From: Eric B Munson @ 2015-06-02 14:25 UTC (permalink / raw)
To: Andrew Morton
Cc: Shuah Khan, Michal Hocko, linux-alpha, linux-kernel, linux-mips,
linux-parisc, linuxppc-dev, sparclinux, linux-xtensa, linux-mm,
linux-arch, linux-api
In-Reply-To: <20150601152746.abbbbb9d479c0e2dbdec2aaf@linux-foundation.org>
[-- Attachment #1: Type: text/plain, Size: 2688 bytes --]
On Mon, 01 Jun 2015, Andrew Morton wrote:
> On Fri, 29 May 2015 10:13:25 -0400 Eric B Munson <emunson@akamai.com> wrote:
>
> > mlock() allows a user to control page out of program memory, but this
> > comes at the cost of faulting in the entire mapping when it is
> > allocated. For large mappings where the entire area is not necessary
> > this is not ideal.
> >
> > This series introduces new flags for mmap() and mlockall() that allow a
> > user to specify that the covered are should not be paged out, but only
> > after the memory has been used the first time.
>
> I almost applied these, but the naming issue (below) stopped me.
>
> A few things...
>
> - The 0/n changelog should reveal how MAP_LOCKONFAULT interacts with
> rlimit(RLIMIT_MEMLOCK).
>
> I see the implementation is "as if the entire mapping will be
> faulted in" (for mmap) and "as if it was MCL_FUTURE" (for mlockall)
> which seems fine. Please include changelog text explaining and
> justifying these decisions. This stuff will need to be in the
> manpage updates as well.
Change logs are updated, and this will be included in the man page
update as well.
>
> - I think I already asked "why not just use MCL_FUTURE" but I forget
> the answer ;) In general it is a good idea to update changelogs in
> response to reviewer questions, because other people will be
> wondering the same things. Or maybe I forgot to ask. Either way,
> please address this in the changelogs.
I must have missed that question. Here is the text from the updated
mlockall changelog:
MCL_ONFAULT is preferrable to MCL_FUTURE for the use cases enumerated
in the previous patch becuase MCL_FUTURE will behave as if each mapping
was made with MAP_LOCKED, causing the entire mapping to be faulted in
when new space is allocated or mapped. MCL_ONFAULT allows the user to
delay the fault in cost of any given page until it is actually needed,
but then guarantees that that page will always be resident.
>
> - I can perhaps see the point in mmap(MAP_LOCKONFAULT) (other
> mappings don't get lock-in-memory treatment), but what's the benefit
> in mlockall(MCL_ON_FAULT) over MCL_FUTURE? (Add to changelog also,
> please).
>
> - Is there a manpage update?
I will send one out when I post V2
>
> - Can we rename patch 1/3 from "add flag to ..." to "add mmap flag to
> ...", to distinguish from 2/3 "add mlockall flag ..."?
Done
>
> - The MAP_LOCKONFAULT versus MCL_ON_FAULT inconsistency is
> irritating! Can we get these consistent please: switch to either
> MAP_LOCK_ON_FAULT or MCL_ONFAULT.
Yes, will do for V2.
>
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply
* Re: [PATCH] seccomp: add ptrace commands for suspend/resume
From: Tycho Andersen @ 2015-06-02 13:05 UTC (permalink / raw)
To: Andrey Wagin
Cc: LKML, linux-api-u79uwXL29TY76Z2rM5mHXA, Kees Cook,
Andy Lutomirski, Will Drewry, Roland McGrath, Oleg Nesterov,
Pavel Emelyanov, Serge E. Hallyn
In-Reply-To: <CANaxB-zacYuo21jLVZyEfyf=UdDnTjYvHdgNpfL+c_DXWRz-eg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
On Tue, Jun 02, 2015 at 12:36:16PM +0300, Andrey Wagin wrote:
> 2015-06-01 22:28 GMT+03:00 Tycho Andersen <tycho.andersen-Z7WLFzj8eWMS+FvcfC7Uqw@public.gmane.org>:
> > This patch is the first step in enabling checkpoint/restore of processes
> > with seccomp enabled.
> >
> > One of the things CRIU does while dumping tasks is inject code into them
> > via ptrace to collect information that is only available to the process
> > itself. However, if we are in a seccomp mode where these processes are
> > prohibited from making these syscalls, then what CRIU does kills the task.
> >
> > This patch adds a new ptrace command, PTRACE_SUSPEND_SECCOMP that enables a
> > task from the init user namespace which has CAP_SYS_ADMIN to disable (and
> > re-enable) seccomp filters for another task so that they can be
> > successfully dumped (and restored).
>
> Do we need to re-enable seccomp if a tracer detaches unexpectedly.
> CRIU can be killed and we should try to not affect the task state even
> in this case.
Yes, I think Pavel's suggestion on the CRIU list of simply
automatically re-enabling seccomp on ptrace detach is the right way to
go here; it should cover this case. The only question is whether or
not to leave the explicit ability to re-enable seccomp before detach
or not. I don't think it's necessary for CRIU, so perhaps I'll remove
it in the next version.
Tycho
^ permalink raw reply
* Re: [PATCH] seccomp: add ptrace commands for suspend/resume
From: Andrey Wagin @ 2015-06-02 9:36 UTC (permalink / raw)
To: Tycho Andersen
Cc: LKML, linux-api-u79uwXL29TY76Z2rM5mHXA, Kees Cook,
Andy Lutomirski, Will Drewry, Roland McGrath, Oleg Nesterov,
Pavel Emelyanov, Serge E. Hallyn
In-Reply-To: <1433186918-9626-1-git-send-email-tycho.andersen-Z7WLFzj8eWMS+FvcfC7Uqw@public.gmane.org>
2015-06-01 22:28 GMT+03:00 Tycho Andersen <tycho.andersen-Z7WLFzj8eWMS+FvcfC7Uqw@public.gmane.org>:
> This patch is the first step in enabling checkpoint/restore of processes
> with seccomp enabled.
>
> One of the things CRIU does while dumping tasks is inject code into them
> via ptrace to collect information that is only available to the process
> itself. However, if we are in a seccomp mode where these processes are
> prohibited from making these syscalls, then what CRIU does kills the task.
>
> This patch adds a new ptrace command, PTRACE_SUSPEND_SECCOMP that enables a
> task from the init user namespace which has CAP_SYS_ADMIN to disable (and
> re-enable) seccomp filters for another task so that they can be
> successfully dumped (and restored).
Do we need to re-enable seccomp if a tracer detaches unexpectedly.
CRIU can be killed and we should try to not affect the task state even
in this case.
Thanks,
Andrew
^ permalink raw reply
* Re: [PATCH v3 4/4] Add virtio gpu driver.
From: Michael S. Tsirkin @ 2015-06-02 8:33 UTC (permalink / raw)
To: Gerd Hoffmann
Cc: open list:ABI/API, open list, dri-devel,
open list:VIRTIO CORE, NET..., airlied
In-Reply-To: <1432300312-24792-5-git-send-email-kraxel@redhat.com>
On Fri, May 22, 2015 at 03:11:52PM +0200, Gerd Hoffmann wrote:
> From: Dave Airlie <airlied@gmail.com>
>
> This patch adds a kms driver for the virtio gpu. The xorg modesetting
> driver can handle the device just fine, the framebuffer for fbcon is
> there too.
>
> Qemu patches for the host side are under review currently.
>
> The pci version of the device comes in two variants: with and without
> vga compatibility. The former has a extra memory bar for the vga
> framebuffer, the later is a pure virtio device. The only concern for
> this driver is that in the virtio-vga case we have to kick out the
> firmware framebuffer.
>
> Initial revision has only 2d support, 3d (virgl) support requires
> some more work on the qemu side and will be added later.
>
> Signed-off-by: Dave Airlie <airlied@redhat.com>
> Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
I think it's ok to merge this through the dri tree.
Acked-by: Michael S. Tsirkin <mst@redhat.com>
> ---
> drivers/gpu/drm/Kconfig | 2 +
> drivers/gpu/drm/Makefile | 1 +
> drivers/gpu/drm/virtio/Kconfig | 14 +
> drivers/gpu/drm/virtio/Makefile | 11 +
> drivers/gpu/drm/virtio/virtgpu_debugfs.c | 64 ++++
> drivers/gpu/drm/virtio/virtgpu_display.c | 485 ++++++++++++++++++++++++
> drivers/gpu/drm/virtio/virtgpu_drm_bus.c | 91 +++++
> drivers/gpu/drm/virtio/virtgpu_drv.c | 136 +++++++
> drivers/gpu/drm/virtio/virtgpu_drv.h | 350 ++++++++++++++++++
> drivers/gpu/drm/virtio/virtgpu_fb.c | 431 ++++++++++++++++++++++
> drivers/gpu/drm/virtio/virtgpu_fence.c | 119 ++++++
> drivers/gpu/drm/virtio/virtgpu_gem.c | 140 +++++++
> drivers/gpu/drm/virtio/virtgpu_kms.c | 164 +++++++++
> drivers/gpu/drm/virtio/virtgpu_object.c | 170 +++++++++
> drivers/gpu/drm/virtio/virtgpu_plane.c | 120 ++++++
> drivers/gpu/drm/virtio/virtgpu_ttm.c | 469 +++++++++++++++++++++++
> drivers/gpu/drm/virtio/virtgpu_vq.c | 614 +++++++++++++++++++++++++++++++
> include/drm/drmP.h | 1 +
> include/uapi/linux/Kbuild | 1 +
> include/uapi/linux/virtio_gpu.h | 204 ++++++++++
> include/uapi/linux/virtio_ids.h | 1 +
> 21 files changed, 3588 insertions(+)
> create mode 100644 drivers/gpu/drm/virtio/Kconfig
> create mode 100644 drivers/gpu/drm/virtio/Makefile
> create mode 100644 drivers/gpu/drm/virtio/virtgpu_debugfs.c
> create mode 100644 drivers/gpu/drm/virtio/virtgpu_display.c
> create mode 100644 drivers/gpu/drm/virtio/virtgpu_drm_bus.c
> create mode 100644 drivers/gpu/drm/virtio/virtgpu_drv.c
> create mode 100644 drivers/gpu/drm/virtio/virtgpu_drv.h
> create mode 100644 drivers/gpu/drm/virtio/virtgpu_fb.c
> create mode 100644 drivers/gpu/drm/virtio/virtgpu_fence.c
> create mode 100644 drivers/gpu/drm/virtio/virtgpu_gem.c
> create mode 100644 drivers/gpu/drm/virtio/virtgpu_kms.c
> create mode 100644 drivers/gpu/drm/virtio/virtgpu_object.c
> create mode 100644 drivers/gpu/drm/virtio/virtgpu_plane.c
> create mode 100644 drivers/gpu/drm/virtio/virtgpu_ttm.c
> create mode 100644 drivers/gpu/drm/virtio/virtgpu_vq.c
> create mode 100644 include/uapi/linux/virtio_gpu.h
>
> diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig
> index 47f2ce8..d4b6545 100644
> --- a/drivers/gpu/drm/Kconfig
> +++ b/drivers/gpu/drm/Kconfig
> @@ -206,6 +206,8 @@ source "drivers/gpu/drm/qxl/Kconfig"
>
> source "drivers/gpu/drm/bochs/Kconfig"
>
> +source "drivers/gpu/drm/virtio/Kconfig"
> +
> source "drivers/gpu/drm/msm/Kconfig"
>
> source "drivers/gpu/drm/tegra/Kconfig"
> diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile
> index 7d4944e..dbbc101 100644
> --- a/drivers/gpu/drm/Makefile
> +++ b/drivers/gpu/drm/Makefile
> @@ -63,6 +63,7 @@ obj-$(CONFIG_DRM_OMAP) += omapdrm/
> obj-$(CONFIG_DRM_TILCDC) += tilcdc/
> obj-$(CONFIG_DRM_QXL) += qxl/
> obj-$(CONFIG_DRM_BOCHS) += bochs/
> +obj-$(CONFIG_DRM_VIRTIO_GPU) += virtio/
> obj-$(CONFIG_DRM_MSM) += msm/
> obj-$(CONFIG_DRM_TEGRA) += tegra/
> obj-$(CONFIG_DRM_STI) += sti/
> diff --git a/drivers/gpu/drm/virtio/Kconfig b/drivers/gpu/drm/virtio/Kconfig
> new file mode 100644
> index 0000000..9983ead
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/Kconfig
> @@ -0,0 +1,14 @@
> +config DRM_VIRTIO_GPU
> + tristate "Virtio GPU driver"
> + depends on DRM && VIRTIO
> + select FB_SYS_FILLRECT
> + select FB_SYS_COPYAREA
> + select FB_SYS_IMAGEBLIT
> + select DRM_KMS_HELPER
> + select DRM_KMS_FB_HELPER
> + select DRM_TTM
> + help
> + This is the virtual GPU driver for virtio. It can be used with
> + QEMU based VMMs (like KVM or Xen).
> +
> + If unsure say M.
> diff --git a/drivers/gpu/drm/virtio/Makefile b/drivers/gpu/drm/virtio/Makefile
> new file mode 100644
> index 0000000..2ee1602
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/Makefile
> @@ -0,0 +1,11 @@
> +#
> +# Makefile for the drm device driver. This driver provides support for the
> +# Direct Rendering Infrastructure (DRI) in XFree86 4.1.0 and higher.
> +
> +ccflags-y := -Iinclude/drm
> +
> +virtio-gpu-y := virtgpu_drv.o virtgpu_kms.o virtgpu_drm_bus.o virtgpu_gem.o \
> + virtgpu_fb.o virtgpu_display.o virtgpu_vq.o virtgpu_ttm.o \
> + virtgpu_fence.o virtgpu_object.o virtgpu_debugfs.o virtgpu_plane.o
> +
> +obj-$(CONFIG_DRM_VIRTIO_GPU) += virtio-gpu.o
> diff --git a/drivers/gpu/drm/virtio/virtgpu_debugfs.c b/drivers/gpu/drm/virtio/virtgpu_debugfs.c
> new file mode 100644
> index 0000000..db8b491
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/virtgpu_debugfs.c
> @@ -0,0 +1,64 @@
> +/*
> + * Copyright (C) 2015 Red Hat, Inc.
> + * All Rights Reserved.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining
> + * a copy of this software and associated documentation files (the
> + * "Software"), to deal in the Software without restriction, including
> + * without limitation the rights to use, copy, modify, merge, publish,
> + * distribute, sublicense, and/or sell copies of the Software, and to
> + * permit persons to whom the Software is furnished to do so, subject to
> + * the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the
> + * next paragraph) shall be included in all copies or substantial
> + * portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
> + * IN NO EVENT SHALL THE COPYRIGHT OWNER(S) AND/OR ITS SUPPLIERS BE
> + * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
> + * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
> + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
> + */
> +
> +#include <linux/debugfs.h>
> +
> +#include "drmP.h"
> +#include "virtgpu_drv.h"
> +
> +static int
> +virtio_gpu_debugfs_irq_info(struct seq_file *m, void *data)
> +{
> + struct drm_info_node *node = (struct drm_info_node *) m->private;
> + struct virtio_gpu_device *vgdev = node->minor->dev->dev_private;
> +
> + seq_printf(m, "fence %ld %lld\n",
> + atomic64_read(&vgdev->fence_drv.last_seq),
> + vgdev->fence_drv.sync_seq);
> + return 0;
> +}
> +
> +static struct drm_info_list virtio_gpu_debugfs_list[] = {
> + { "irq_fence", virtio_gpu_debugfs_irq_info, 0, NULL },
> +};
> +
> +#define VIRTIO_GPU_DEBUGFS_ENTRIES ARRAY_SIZE(virtio_gpu_debugfs_list)
> +
> +int
> +virtio_gpu_debugfs_init(struct drm_minor *minor)
> +{
> + drm_debugfs_create_files(virtio_gpu_debugfs_list,
> + VIRTIO_GPU_DEBUGFS_ENTRIES,
> + minor->debugfs_root, minor);
> + return 0;
> +}
> +
> +void
> +virtio_gpu_debugfs_takedown(struct drm_minor *minor)
> +{
> + drm_debugfs_remove_files(virtio_gpu_debugfs_list,
> + VIRTIO_GPU_DEBUGFS_ENTRIES,
> + minor);
> +}
> diff --git a/drivers/gpu/drm/virtio/virtgpu_display.c b/drivers/gpu/drm/virtio/virtgpu_display.c
> new file mode 100644
> index 0000000..a2eca5f
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/virtgpu_display.c
> @@ -0,0 +1,485 @@
> +/*
> + * Copyright (C) 2015 Red Hat, Inc.
> + * All Rights Reserved.
> + *
> + * Authors:
> + * Dave Airlie
> + * Alon Levy
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
> + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
> + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> + * OTHER DEALINGS IN THE SOFTWARE.
> + */
> +
> +#include "virtgpu_drv.h"
> +#include <drm/drm_crtc_helper.h>
> +#include <drm/drm_atomic_helper.h>
> +
> +#define XRES_MIN 320
> +#define YRES_MIN 200
> +
> +#define XRES_DEF 1024
> +#define YRES_DEF 768
> +
> +#define XRES_MAX 8192
> +#define YRES_MAX 8192
> +
> +static int virtio_gpu_fbdev = 1;
> +
> +MODULE_PARM_DESC(fbdev, "Disable/Enable framebuffer device & console");
> +module_param_named(fbdev, virtio_gpu_fbdev, int, 0400);
> +
> +static void virtio_gpu_crtc_gamma_set(struct drm_crtc *crtc,
> + u16 *red, u16 *green, u16 *blue,
> + uint32_t start, uint32_t size)
> +{
> + /* TODO */
> +}
> +
> +static void
> +virtio_gpu_hide_cursor(struct virtio_gpu_device *vgdev,
> + struct virtio_gpu_output *output)
> +{
> + output->cursor.hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_UPDATE_CURSOR);
> + output->cursor.resource_id = 0;
> + virtio_gpu_cursor_ping(vgdev, output);
> +}
> +
> +static int virtio_gpu_crtc_cursor_set(struct drm_crtc *crtc,
> + struct drm_file *file_priv,
> + uint32_t handle,
> + uint32_t width,
> + uint32_t height,
> + int32_t hot_x, int32_t hot_y)
> +{
> + struct virtio_gpu_device *vgdev = crtc->dev->dev_private;
> + struct virtio_gpu_output *output =
> + container_of(crtc, struct virtio_gpu_output, crtc);
> + struct drm_gem_object *gobj = NULL;
> + struct virtio_gpu_object *qobj = NULL;
> + struct virtio_gpu_fence *fence = NULL;
> + int ret = 0;
> +
> + if (handle == 0) {
> + virtio_gpu_hide_cursor(vgdev, output);
> + return 0;
> + }
> +
> + /* lookup the cursor */
> + gobj = drm_gem_object_lookup(crtc->dev, file_priv, handle);
> + if (gobj == NULL)
> + return -ENOENT;
> +
> + qobj = gem_to_virtio_gpu_obj(gobj);
> +
> + if (!qobj->hw_res_handle) {
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + virtio_gpu_cmd_transfer_to_host_2d(vgdev, qobj->hw_res_handle, 0,
> + cpu_to_le32(64),
> + cpu_to_le32(64),
> + 0, 0, &fence);
> +
> + output->cursor.hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_UPDATE_CURSOR);
> + output->cursor.resource_id = cpu_to_le32(qobj->hw_res_handle);
> + output->cursor.hot_x = cpu_to_le32(hot_x);
> + output->cursor.hot_y = cpu_to_le32(hot_y);
> + virtio_gpu_cursor_ping(vgdev, output);
> + ret = 0;
> +
> +out:
> + drm_gem_object_unreference_unlocked(gobj);
> + return ret;
> +}
> +
> +static int virtio_gpu_crtc_cursor_move(struct drm_crtc *crtc,
> + int x, int y)
> +{
> + struct virtio_gpu_device *vgdev = crtc->dev->dev_private;
> + struct virtio_gpu_output *output =
> + container_of(crtc, struct virtio_gpu_output, crtc);
> +
> + output->cursor.hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_MOVE_CURSOR);
> + output->cursor.pos.x = cpu_to_le32(x);
> + output->cursor.pos.y = cpu_to_le32(y);
> + virtio_gpu_cursor_ping(vgdev, output);
> + return 0;
> +}
> +
> +static const struct drm_crtc_funcs virtio_gpu_crtc_funcs = {
> + .cursor_set2 = virtio_gpu_crtc_cursor_set,
> + .cursor_move = virtio_gpu_crtc_cursor_move,
> + .gamma_set = virtio_gpu_crtc_gamma_set,
> + .set_config = drm_atomic_helper_set_config,
> + .destroy = drm_crtc_cleanup,
> +
> +#if 0 /* not (yet) working without vblank support according to docs */
> + .page_flip = drm_atomic_helper_page_flip,
> +#endif
> + .reset = drm_atomic_helper_crtc_reset,
> + .atomic_duplicate_state = drm_atomic_helper_crtc_duplicate_state,
> + .atomic_destroy_state = drm_atomic_helper_crtc_destroy_state,
> +};
> +
> +static void virtio_gpu_user_framebuffer_destroy(struct drm_framebuffer *fb)
> +{
> + struct virtio_gpu_framebuffer *virtio_gpu_fb
> + = to_virtio_gpu_framebuffer(fb);
> +
> + if (virtio_gpu_fb->obj)
> + drm_gem_object_unreference_unlocked(virtio_gpu_fb->obj);
> + drm_framebuffer_cleanup(fb);
> + kfree(virtio_gpu_fb);
> +}
> +
> +static int
> +virtio_gpu_framebuffer_surface_dirty(struct drm_framebuffer *fb,
> + struct drm_file *file_priv,
> + unsigned flags, unsigned color,
> + struct drm_clip_rect *clips,
> + unsigned num_clips)
> +{
> + struct virtio_gpu_framebuffer *virtio_gpu_fb
> + = to_virtio_gpu_framebuffer(fb);
> +
> + return virtio_gpu_surface_dirty(virtio_gpu_fb, clips, num_clips);
> +}
> +
> +static const struct drm_framebuffer_funcs virtio_gpu_fb_funcs = {
> + .destroy = virtio_gpu_user_framebuffer_destroy,
> + .dirty = virtio_gpu_framebuffer_surface_dirty,
> +};
> +
> +int
> +virtio_gpu_framebuffer_init(struct drm_device *dev,
> + struct virtio_gpu_framebuffer *vgfb,
> + struct drm_mode_fb_cmd2 *mode_cmd,
> + struct drm_gem_object *obj)
> +{
> + int ret;
> + struct virtio_gpu_object *bo;
> + vgfb->obj = obj;
> +
> + bo = gem_to_virtio_gpu_obj(obj);
> +
> + ret = drm_framebuffer_init(dev, &vgfb->base, &virtio_gpu_fb_funcs);
> + if (ret) {
> + vgfb->obj = NULL;
> + return ret;
> + }
> + drm_helper_mode_fill_fb_struct(&vgfb->base, mode_cmd);
> +
> + spin_lock_init(&vgfb->dirty_lock);
> + vgfb->x1 = vgfb->y1 = INT_MAX;
> + vgfb->x2 = vgfb->y2 = 0;
> + return 0;
> +}
> +
> +static bool virtio_gpu_crtc_mode_fixup(struct drm_crtc *crtc,
> + const struct drm_display_mode *mode,
> + struct drm_display_mode *adjusted_mode)
> +{
> + return true;
> +}
> +
> +static void virtio_gpu_crtc_mode_set_nofb(struct drm_crtc *crtc)
> +{
> + struct drm_device *dev = crtc->dev;
> + struct virtio_gpu_device *vgdev = dev->dev_private;
> + struct virtio_gpu_output *output = drm_crtc_to_virtio_gpu_output(crtc);
> +
> + virtio_gpu_cmd_set_scanout(vgdev, output->index, 0,
> + crtc->mode.hdisplay,
> + crtc->mode.vdisplay, 0, 0);
> +}
> +
> +static void virtio_gpu_crtc_enable(struct drm_crtc *crtc)
> +{
> +}
> +
> +static void virtio_gpu_crtc_disable(struct drm_crtc *crtc)
> +{
> + struct drm_device *dev = crtc->dev;
> + struct virtio_gpu_device *vgdev = dev->dev_private;
> + struct virtio_gpu_output *output = drm_crtc_to_virtio_gpu_output(crtc);
> +
> + virtio_gpu_cmd_set_scanout(vgdev, output->index, 0, 0, 0, 0, 0);
> +}
> +
> +static int virtio_gpu_crtc_atomic_check(struct drm_crtc *crtc,
> + struct drm_crtc_state *state)
> +{
> + return 0;
> +}
> +
> +static const struct drm_crtc_helper_funcs virtio_gpu_crtc_helper_funcs = {
> + .enable = virtio_gpu_crtc_enable,
> + .disable = virtio_gpu_crtc_disable,
> + .mode_fixup = virtio_gpu_crtc_mode_fixup,
> + .mode_set_nofb = virtio_gpu_crtc_mode_set_nofb,
> + .atomic_check = virtio_gpu_crtc_atomic_check,
> +};
> +
> +static bool virtio_gpu_enc_mode_fixup(struct drm_encoder *encoder,
> + const struct drm_display_mode *mode,
> + struct drm_display_mode *adjusted_mode)
> +{
> + return true;
> +}
> +
> +static void virtio_gpu_enc_mode_set(struct drm_encoder *encoder,
> + struct drm_display_mode *mode,
> + struct drm_display_mode *adjusted_mode)
> +{
> +}
> +
> +static void virtio_gpu_enc_enable(struct drm_encoder *encoder)
> +{
> +}
> +
> +static void virtio_gpu_enc_disable(struct drm_encoder *encoder)
> +{
> +}
> +
> +static int virtio_gpu_conn_get_modes(struct drm_connector *connector)
> +{
> + struct virtio_gpu_output *output =
> + drm_connector_to_virtio_gpu_output(connector);
> + struct drm_display_mode *mode = NULL;
> + int count, width, height;
> +
> + width = le32_to_cpu(output->info.r.width);
> + height = le32_to_cpu(output->info.r.height);
> + count = drm_add_modes_noedid(connector, XRES_MAX, YRES_MAX);
> +
> + if (width == 0 || height == 0) {
> + width = XRES_DEF;
> + height = YRES_DEF;
> + drm_set_preferred_mode(connector, XRES_DEF, YRES_DEF);
> + } else {
> + DRM_DEBUG("add mode: %dx%d\n", width, height);
> + mode = drm_cvt_mode(connector->dev, width, height, 60,
> + false, false, false);
> + mode->type |= DRM_MODE_TYPE_PREFERRED;
> + drm_mode_probed_add(connector, mode);
> + count++;
> + }
> +
> + return count;
> +}
> +
> +static int virtio_gpu_conn_mode_valid(struct drm_connector *connector,
> + struct drm_display_mode *mode)
> +{
> + struct virtio_gpu_output *output =
> + drm_connector_to_virtio_gpu_output(connector);
> + int width, height;
> +
> + width = le32_to_cpu(output->info.r.width);
> + height = le32_to_cpu(output->info.r.height);
> +
> + if (!(mode->type & DRM_MODE_TYPE_PREFERRED))
> + return MODE_OK;
> + if (mode->hdisplay == XRES_DEF && mode->vdisplay == YRES_DEF)
> + return MODE_OK;
> + if (mode->hdisplay <= width && mode->hdisplay >= width - 16 &&
> + mode->vdisplay <= height && mode->vdisplay >= height - 16)
> + return MODE_OK;
> +
> + DRM_DEBUG("del mode: %dx%d\n", mode->hdisplay, mode->vdisplay);
> + return MODE_BAD;
> +}
> +
> +static struct drm_encoder*
> +virtio_gpu_best_encoder(struct drm_connector *connector)
> +{
> + struct virtio_gpu_output *virtio_gpu_output =
> + drm_connector_to_virtio_gpu_output(connector);
> +
> + return &virtio_gpu_output->enc;
> +}
> +
> +static const struct drm_encoder_helper_funcs virtio_gpu_enc_helper_funcs = {
> + .mode_fixup = virtio_gpu_enc_mode_fixup,
> + .mode_set = virtio_gpu_enc_mode_set,
> + .enable = virtio_gpu_enc_enable,
> + .disable = virtio_gpu_enc_disable,
> +};
> +
> +static const struct drm_connector_helper_funcs virtio_gpu_conn_helper_funcs = {
> + .get_modes = virtio_gpu_conn_get_modes,
> + .mode_valid = virtio_gpu_conn_mode_valid,
> + .best_encoder = virtio_gpu_best_encoder,
> +};
> +
> +static void virtio_gpu_conn_save(struct drm_connector *connector)
> +{
> + DRM_DEBUG("\n");
> +}
> +
> +static void virtio_gpu_conn_restore(struct drm_connector *connector)
> +{
> + DRM_DEBUG("\n");
> +}
> +
> +static enum drm_connector_status virtio_gpu_conn_detect(
> + struct drm_connector *connector,
> + bool force)
> +{
> + struct virtio_gpu_output *output =
> + drm_connector_to_virtio_gpu_output(connector);
> +
> + if (output->info.enabled)
> + return connector_status_connected;
> + else
> + return connector_status_disconnected;
> +}
> +
> +static void virtio_gpu_conn_destroy(struct drm_connector *connector)
> +{
> + struct virtio_gpu_output *virtio_gpu_output =
> + drm_connector_to_virtio_gpu_output(connector);
> +
> + drm_connector_unregister(connector);
> + drm_connector_cleanup(connector);
> + kfree(virtio_gpu_output);
> +}
> +
> +static const struct drm_connector_funcs virtio_gpu_connector_funcs = {
> + .dpms = drm_atomic_helper_connector_dpms,
> + .save = virtio_gpu_conn_save,
> + .restore = virtio_gpu_conn_restore,
> + .detect = virtio_gpu_conn_detect,
> + .fill_modes = drm_helper_probe_single_connector_modes,
> + .destroy = virtio_gpu_conn_destroy,
> + .reset = drm_atomic_helper_connector_reset,
> + .atomic_duplicate_state = drm_atomic_helper_connector_duplicate_state,
> + .atomic_destroy_state = drm_atomic_helper_connector_destroy_state,
> +};
> +
> +static const struct drm_encoder_funcs virtio_gpu_enc_funcs = {
> + .destroy = drm_encoder_cleanup,
> +};
> +
> +static int vgdev_output_init(struct virtio_gpu_device *vgdev, int index)
> +{
> + struct drm_device *dev = vgdev->ddev;
> + struct virtio_gpu_output *output = vgdev->outputs + index;
> + struct drm_connector *connector = &output->conn;
> + struct drm_encoder *encoder = &output->enc;
> + struct drm_crtc *crtc = &output->crtc;
> + struct drm_plane *plane;
> +
> + output->index = index;
> + if (index == 0) {
> + output->info.enabled = cpu_to_le32(true);
> + output->info.r.width = cpu_to_le32(XRES_DEF);
> + output->info.r.height = cpu_to_le32(YRES_DEF);
> + }
> +
> + plane = virtio_gpu_plane_init(vgdev, index);
> + if (IS_ERR(plane))
> + return PTR_ERR(plane);
> + drm_crtc_init_with_planes(dev, crtc, plane, NULL,
> + &virtio_gpu_crtc_funcs);
> + drm_mode_crtc_set_gamma_size(crtc, 256);
> + drm_crtc_helper_add(crtc, &virtio_gpu_crtc_helper_funcs);
> + plane->crtc = crtc;
> +
> + drm_connector_init(dev, connector, &virtio_gpu_connector_funcs,
> + DRM_MODE_CONNECTOR_VIRTUAL);
> + drm_connector_helper_add(connector, &virtio_gpu_conn_helper_funcs);
> +
> + drm_encoder_init(dev, encoder, &virtio_gpu_enc_funcs,
> + DRM_MODE_ENCODER_VIRTUAL);
> + drm_encoder_helper_add(encoder, &virtio_gpu_enc_helper_funcs);
> + encoder->possible_crtcs = 1 << index;
> +
> + drm_mode_connector_attach_encoder(connector, encoder);
> + drm_connector_register(connector);
> + return 0;
> +}
> +
> +static struct drm_framebuffer *
> +virtio_gpu_user_framebuffer_create(struct drm_device *dev,
> + struct drm_file *file_priv,
> + struct drm_mode_fb_cmd2 *mode_cmd)
> +{
> + struct drm_gem_object *obj = NULL;
> + struct virtio_gpu_framebuffer *virtio_gpu_fb;
> + int ret;
> +
> + /* lookup object associated with res handle */
> + obj = drm_gem_object_lookup(dev, file_priv, mode_cmd->handles[0]);
> + if (!obj)
> + return ERR_PTR(-EINVAL);
> +
> + virtio_gpu_fb = kzalloc(sizeof(*virtio_gpu_fb), GFP_KERNEL);
> + if (virtio_gpu_fb == NULL)
> + return ERR_PTR(-ENOMEM);
> +
> + ret = virtio_gpu_framebuffer_init(dev, virtio_gpu_fb, mode_cmd, obj);
> + if (ret) {
> + kfree(virtio_gpu_fb);
> + if (obj)
> + drm_gem_object_unreference_unlocked(obj);
> + return NULL;
> + }
> +
> + return &virtio_gpu_fb->base;
> +}
> +
> +static const struct drm_mode_config_funcs virtio_gpu_mode_funcs = {
> + .fb_create = virtio_gpu_user_framebuffer_create,
> + .atomic_check = drm_atomic_helper_check,
> + .atomic_commit = drm_atomic_helper_commit,
> +};
> +
> +int virtio_gpu_modeset_init(struct virtio_gpu_device *vgdev)
> +{
> + int i, ret;
> +
> + drm_mode_config_init(vgdev->ddev);
> + vgdev->ddev->mode_config.funcs = (void *)&virtio_gpu_mode_funcs;
> +
> + /* modes will be validated against the framebuffer size */
> + vgdev->ddev->mode_config.min_width = XRES_MIN;
> + vgdev->ddev->mode_config.min_height = YRES_MIN;
> + vgdev->ddev->mode_config.max_width = XRES_MAX;
> + vgdev->ddev->mode_config.max_height = YRES_MAX;
> +
> + for (i = 0 ; i < vgdev->num_scanouts; ++i)
> + vgdev_output_init(vgdev, i);
> +
> + drm_mode_config_reset(vgdev->ddev);
> +
> + if (virtio_gpu_fbdev) {
> + ret = virtio_gpu_fbdev_init(vgdev);
> + if (ret)
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> +void virtio_gpu_modeset_fini(struct virtio_gpu_device *vgdev)
> +{
> + virtio_gpu_fbdev_fini(vgdev);
> + drm_mode_config_cleanup(vgdev->ddev);
> +}
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drm_bus.c b/drivers/gpu/drm/virtio/virtgpu_drm_bus.c
> new file mode 100644
> index 0000000..f4ec816
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/virtgpu_drm_bus.c
> @@ -0,0 +1,91 @@
> +/*
> + * Copyright (C) 2015 Red Hat, Inc.
> + * All Rights Reserved.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining
> + * a copy of this software and associated documentation files (the
> + * "Software"), to deal in the Software without restriction, including
> + * without limitation the rights to use, copy, modify, merge, publish,
> + * distribute, sublicense, and/or sell copies of the Software, and to
> + * permit persons to whom the Software is furnished to do so, subject to
> + * the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the
> + * next paragraph) shall be included in all copies or substantial
> + * portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
> + * IN NO EVENT SHALL THE COPYRIGHT OWNER(S) AND/OR ITS SUPPLIERS BE
> + * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
> + * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
> + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
> + */
> +
> +#include <linux/pci.h>
> +
> +#include "virtgpu_drv.h"
> +
> +int drm_virtio_set_busid(struct drm_device *dev, struct drm_master *master)
> +{
> + struct pci_dev *pdev = dev->pdev;
> +
> + if (pdev) {
> + return drm_pci_set_busid(dev, master);
> + }
> + return 0;
> +}
> +
> +int drm_virtio_init(struct drm_driver *driver, struct virtio_device *vdev)
> +{
> + struct drm_device *dev;
> + int ret;
> +
> + dev = drm_dev_alloc(driver, &vdev->dev);
> + if (!dev)
> + return -ENOMEM;
> + dev->virtdev = vdev;
> + vdev->priv = dev;
> +
> + if (strcmp(vdev->dev.parent->bus->name, "pci") == 0) {
> + struct pci_dev *pdev = to_pci_dev(vdev->dev.parent);
> + bool vga = (pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA;
> +
> + if (vga) {
> + /*
> + * Need to make sure we don't have two drivers
> + * for the same hardware here. Some day we
> + * will simply kick out the firmware
> + * (vesa/efi) framebuffer.
> + *
> + * Virtual hardware specs for virtio-vga are
> + * not finalized yet, therefore we can't add
> + * code for that yet.
> + *
> + * So ignore the device for the time being,
> + * and suggest to the user use the device
> + * variant without vga compatibility mode.
> + */
> + DRM_ERROR("virtio-vga not (yet) supported\n");
> + DRM_ERROR("please use virtio-gpu-pci instead\n");
> + ret = -ENODEV;
> + goto err_free;
> + }
> + dev->pdev = pdev;
> + }
> +
> + ret = drm_dev_register(dev, 0);
> + if (ret)
> + goto err_free;
> +
> + DRM_INFO("Initialized %s %d.%d.%d %s on minor %d\n", driver->name,
> + driver->major, driver->minor, driver->patchlevel,
> + driver->date, dev->primary->index);
> +
> + return 0;
> +
> +err_free:
> + drm_dev_unref(dev);
> + return ret;
> +}
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.c b/drivers/gpu/drm/virtio/virtgpu_drv.c
> new file mode 100644
> index 0000000..7d9610a
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.c
> @@ -0,0 +1,136 @@
> +/*
> + * Copyright (C) 2015 Red Hat, Inc.
> + * All Rights Reserved.
> + *
> + * Authors:
> + * Dave Airlie <airlied@redhat.com>
> + * Gerd Hoffmann <kraxel@redhat.com>
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the next
> + * paragraph) shall be included in all copies or substantial portions of the
> + * Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * VA LINUX SYSTEMS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
> + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
> + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> + * OTHER DEALINGS IN THE SOFTWARE.
> + */
> +
> +#include <linux/module.h>
> +#include <linux/console.h>
> +#include <linux/pci.h>
> +#include "drmP.h"
> +#include "drm/drm.h"
> +
> +#include "virtgpu_drv.h"
> +static struct drm_driver driver;
> +
> +static int virtio_gpu_modeset = -1;
> +
> +MODULE_PARM_DESC(modeset, "Disable/Enable modesetting");
> +module_param_named(modeset, virtio_gpu_modeset, int, 0400);
> +
> +static int virtio_gpu_probe(struct virtio_device *vdev)
> +{
> +#ifdef CONFIG_VGA_CONSOLE
> + if (vgacon_text_force() && virtio_gpu_modeset == -1)
> + return -EINVAL;
> +#endif
> +
> + if (virtio_gpu_modeset == 0)
> + return -EINVAL;
> +
> + return drm_virtio_init(&driver, vdev);
> +}
> +
> +static void virtio_gpu_remove(struct virtio_device *vdev)
> +{
> + struct drm_device *dev = vdev->priv;
> + drm_put_dev(dev);
> +}
> +
> +static void virtio_gpu_config_changed(struct virtio_device *vdev)
> +{
> + struct drm_device *dev = vdev->priv;
> + struct virtio_gpu_device *vgdev = dev->dev_private;
> +
> + schedule_work(&vgdev->config_changed_work);
> +}
> +
> +static struct virtio_device_id id_table[] = {
> + { VIRTIO_ID_GPU, VIRTIO_DEV_ANY_ID },
> + { 0 },
> +};
> +
> +static unsigned int features[] = {
> +};
> +static struct virtio_driver virtio_gpu_driver = {
> + .feature_table = features,
> + .feature_table_size = ARRAY_SIZE(features),
> + .driver.name = KBUILD_MODNAME,
> + .driver.owner = THIS_MODULE,
> + .id_table = id_table,
> + .probe = virtio_gpu_probe,
> + .remove = virtio_gpu_remove,
> + .config_changed = virtio_gpu_config_changed
> +};
> +
> +module_virtio_driver(virtio_gpu_driver);
> +
> +MODULE_DEVICE_TABLE(virtio, id_table);
> +MODULE_DESCRIPTION("Virtio GPU driver");
> +MODULE_LICENSE("GPL and additional rights");
> +MODULE_AUTHOR("Dave Airlie <airlied@redhat.com>");
> +MODULE_AUTHOR("Gerd Hoffmann <kraxel@redhat.com>");
> +MODULE_AUTHOR("Alon Levy");
> +
> +static const struct file_operations virtio_gpu_driver_fops = {
> + .owner = THIS_MODULE,
> + .open = drm_open,
> + .mmap = virtio_gpu_mmap,
> + .poll = drm_poll,
> + .read = drm_read,
> + .unlocked_ioctl = drm_ioctl,
> + .release = drm_release,
> +#ifdef CONFIG_COMPAT
> + .compat_ioctl = drm_compat_ioctl,
> +#endif
> + .llseek = noop_llseek,
> +};
> +
> +
> +static struct drm_driver driver = {
> + .driver_features = DRIVER_MODESET | DRIVER_GEM,
> + .set_busid = drm_virtio_set_busid,
> + .load = virtio_gpu_driver_load,
> + .unload = virtio_gpu_driver_unload,
> +
> + .dumb_create = virtio_gpu_mode_dumb_create,
> + .dumb_map_offset = virtio_gpu_mode_dumb_mmap,
> + .dumb_destroy = virtio_gpu_mode_dumb_destroy,
> +
> +#if defined(CONFIG_DEBUG_FS)
> + .debugfs_init = virtio_gpu_debugfs_init,
> + .debugfs_cleanup = virtio_gpu_debugfs_takedown,
> +#endif
> +
> + .gem_free_object = virtio_gpu_gem_free_object,
> + .fops = &virtio_gpu_driver_fops,
> +
> + .name = DRIVER_NAME,
> + .desc = DRIVER_DESC,
> + .date = DRIVER_DATE,
> + .major = DRIVER_MAJOR,
> + .minor = DRIVER_MINOR,
> + .patchlevel = DRIVER_PATCHLEVEL,
> +};
> diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h
> new file mode 100644
> index 0000000..e5a2c09
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h
> @@ -0,0 +1,350 @@
> +/*
> + * Copyright (C) 2015 Red Hat, Inc.
> + * All Rights Reserved.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining
> + * a copy of this software and associated documentation files (the
> + * "Software"), to deal in the Software without restriction, including
> + * without limitation the rights to use, copy, modify, merge, publish,
> + * distribute, sublicense, and/or sell copies of the Software, and to
> + * permit persons to whom the Software is furnished to do so, subject to
> + * the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the
> + * next paragraph) shall be included in all copies or substantial
> + * portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
> + * IN NO EVENT SHALL THE COPYRIGHT OWNER(S) AND/OR ITS SUPPLIERS BE
> + * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
> + * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
> + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
> + */
> +
> +#ifndef VIRTIO_DRV_H
> +#define VIRTIO_DRV_H
> +
> +#include <linux/virtio.h>
> +#include <linux/virtio_ids.h>
> +#include <linux/virtio_config.h>
> +#include <linux/virtio_gpu.h>
> +
> +#include <drm/drmP.h>
> +#include <drm/drm_gem.h>
> +#include <drm/drm_crtc_helper.h>
> +#include <ttm/ttm_bo_api.h>
> +#include <ttm/ttm_bo_driver.h>
> +#include <ttm/ttm_placement.h>
> +#include <ttm/ttm_module.h>
> +
> +#define DRIVER_NAME "virtio_gpu"
> +#define DRIVER_DESC "virtio GPU"
> +#define DRIVER_DATE "0"
> +
> +#define DRIVER_MAJOR 0
> +#define DRIVER_MINOR 0
> +#define DRIVER_PATCHLEVEL 1
> +
> +/* virtgpu_drm_bus.c */
> +int drm_virtio_set_busid(struct drm_device *dev, struct drm_master *master);
> +int drm_virtio_init(struct drm_driver *driver, struct virtio_device *vdev);
> +
> +struct virtio_gpu_object {
> + struct drm_gem_object gem_base;
> + uint32_t hw_res_handle;
> +
> + struct sg_table *pages;
> + void *vmap;
> + bool dumb;
> + struct ttm_place placement_code;
> + struct ttm_placement placement;
> + struct ttm_buffer_object tbo;
> + struct ttm_bo_kmap_obj kmap;
> +};
> +#define gem_to_virtio_gpu_obj(gobj) \
> + container_of((gobj), struct virtio_gpu_object, gem_base)
> +
> +struct virtio_gpu_vbuffer;
> +struct virtio_gpu_device;
> +
> +typedef void (*virtio_gpu_resp_cb)(struct virtio_gpu_device *vgdev,
> + struct virtio_gpu_vbuffer *vbuf);
> +
> +struct virtio_gpu_fence_driver {
> + atomic64_t last_seq;
> + uint64_t sync_seq;
> + struct list_head fences;
> + spinlock_t lock;
> +};
> +
> +struct virtio_gpu_fence {
> + struct fence f;
> + struct virtio_gpu_fence_driver *drv;
> + struct list_head node;
> + uint64_t seq;
> +};
> +#define to_virtio_fence(x) \
> + container_of(x, struct virtio_gpu_fence, f)
> +
> +struct virtio_gpu_vbuffer {
> + char *buf;
> + int size;
> +
> + void *data_buf;
> + uint32_t data_size;
> +
> + char *resp_buf;
> + int resp_size;
> +
> + virtio_gpu_resp_cb resp_cb;
> +
> + struct list_head list;
> +};
> +
> +struct virtio_gpu_output {
> + int index;
> + struct drm_crtc crtc;
> + struct drm_connector conn;
> + struct drm_encoder enc;
> + struct virtio_gpu_display_one info;
> + struct virtio_gpu_update_cursor cursor;
> + int cur_x;
> + int cur_y;
> +};
> +#define drm_crtc_to_virtio_gpu_output(x) \
> + container_of(x, struct virtio_gpu_output, crtc)
> +#define drm_connector_to_virtio_gpu_output(x) \
> + container_of(x, struct virtio_gpu_output, conn)
> +#define drm_encoder_to_virtio_gpu_output(x) \
> + container_of(x, struct virtio_gpu_output, enc)
> +
> +struct virtio_gpu_framebuffer {
> + struct drm_framebuffer base;
> + struct drm_gem_object *obj;
> + int x1, y1, x2, y2; /* dirty rect */
> + spinlock_t dirty_lock;
> + uint32_t hw_res_handle;
> +};
> +#define to_virtio_gpu_framebuffer(x) \
> + container_of(x, struct virtio_gpu_framebuffer, base)
> +
> +struct virtio_gpu_mman {
> + struct ttm_bo_global_ref bo_global_ref;
> + struct drm_global_reference mem_global_ref;
> + bool mem_global_referenced;
> + struct ttm_bo_device bdev;
> +};
> +
> +struct virtio_gpu_fbdev;
> +
> +struct virtio_gpu_queue {
> + struct virtqueue *vq;
> + spinlock_t qlock;
> + wait_queue_head_t ack_queue;
> + struct work_struct dequeue_work;
> +};
> +
> +struct virtio_gpu_device {
> + struct device *dev;
> + struct drm_device *ddev;
> +
> + struct virtio_device *vdev;
> +
> + struct virtio_gpu_mman mman;
> +
> + /* pointer to fbdev info structure */
> + struct virtio_gpu_fbdev *vgfbdev;
> + struct virtio_gpu_output outputs[VIRTIO_GPU_MAX_SCANOUTS];
> + uint32_t num_scanouts;
> +
> + struct virtio_gpu_queue ctrlq;
> + struct virtio_gpu_queue cursorq;
> + struct list_head free_vbufs;
> + void *vbufs;
> + bool vqs_ready;
> +
> + struct idr resource_idr;
> + spinlock_t resource_idr_lock;
> +
> + wait_queue_head_t resp_wq;
> + /* current display info */
> + spinlock_t display_info_lock;
> +
> + struct virtio_gpu_fence_driver fence_drv;
> +
> + struct idr ctx_id_idr;
> + spinlock_t ctx_id_idr_lock;
> +
> + struct work_struct config_changed_work;
> +};
> +
> +struct virtio_gpu_fpriv {
> + uint32_t ctx_id;
> +};
> +
> +/* virtio_ioctl.c */
> +#define DRM_VIRTIO_NUM_IOCTLS 10
> +extern struct drm_ioctl_desc virtio_gpu_ioctls[DRM_VIRTIO_NUM_IOCTLS];
> +
> +/* virtio_kms.c */
> +int virtio_gpu_driver_load(struct drm_device *dev, unsigned long flags);
> +int virtio_gpu_driver_unload(struct drm_device *dev);
> +
> +/* virtio_gem.c */
> +void virtio_gpu_gem_free_object(struct drm_gem_object *gem_obj);
> +int virtio_gpu_gem_init(struct virtio_gpu_device *vgdev);
> +void virtio_gpu_gem_fini(struct virtio_gpu_device *vgdev);
> +int virtio_gpu_gem_create(struct drm_file *file,
> + struct drm_device *dev,
> + uint64_t size,
> + struct drm_gem_object **obj_p,
> + uint32_t *handle_p);
> +struct virtio_gpu_object *virtio_gpu_alloc_object(struct drm_device *dev,
> + size_t size, bool kernel,
> + bool pinned);
> +int virtio_gpu_mode_dumb_create(struct drm_file *file_priv,
> + struct drm_device *dev,
> + struct drm_mode_create_dumb *args);
> +int virtio_gpu_mode_dumb_destroy(struct drm_file *file_priv,
> + struct drm_device *dev,
> + uint32_t handle);
> +int virtio_gpu_mode_dumb_mmap(struct drm_file *file_priv,
> + struct drm_device *dev,
> + uint32_t handle, uint64_t *offset_p);
> +
> +/* virtio_fb */
> +#define VIRTIO_GPUFB_CONN_LIMIT 1
> +int virtio_gpu_fbdev_init(struct virtio_gpu_device *vgdev);
> +void virtio_gpu_fbdev_fini(struct virtio_gpu_device *vgdev);
> +int virtio_gpu_surface_dirty(struct virtio_gpu_framebuffer *qfb,
> + struct drm_clip_rect *clips,
> + unsigned num_clips);
> +/* virtio vg */
> +int virtio_gpu_alloc_vbufs(struct virtio_gpu_device *vgdev);
> +void virtio_gpu_free_vbufs(struct virtio_gpu_device *vgdev);
> +void virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
> + uint32_t *resid);
> +void virtio_gpu_resource_id_put(struct virtio_gpu_device *vgdev, uint32_t id);
> +void virtio_gpu_cmd_create_resource(struct virtio_gpu_device *vgdev,
> + uint32_t resource_id,
> + uint32_t format,
> + uint32_t width,
> + uint32_t height);
> +void virtio_gpu_cmd_unref_resource(struct virtio_gpu_device *vgdev,
> + uint32_t resource_id);
> +void virtio_gpu_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev,
> + uint32_t resource_id, uint64_t offset,
> + __le32 width, __le32 height,
> + __le32 x, __le32 y,
> + struct virtio_gpu_fence **fence);
> +void virtio_gpu_cmd_resource_flush(struct virtio_gpu_device *vgdev,
> + uint32_t resource_id,
> + uint32_t x, uint32_t y,
> + uint32_t width, uint32_t height);
> +void virtio_gpu_cmd_set_scanout(struct virtio_gpu_device *vgdev,
> + uint32_t scanout_id, uint32_t resource_id,
> + uint32_t width, uint32_t height,
> + uint32_t x, uint32_t y);
> +int virtio_gpu_object_attach(struct virtio_gpu_device *vgdev,
> + struct virtio_gpu_object *obj,
> + uint32_t resource_id,
> + struct virtio_gpu_fence **fence);
> +int virtio_gpu_attach_status_page(struct virtio_gpu_device *vgdev);
> +int virtio_gpu_detach_status_page(struct virtio_gpu_device *vgdev);
> +void virtio_gpu_cursor_ping(struct virtio_gpu_device *vgdev,
> + struct virtio_gpu_output *output);
> +int virtio_gpu_cmd_get_display_info(struct virtio_gpu_device *vgdev);
> +void virtio_gpu_cmd_resource_inval_backing(struct virtio_gpu_device *vgdev,
> + uint32_t resource_id);
> +void virtio_gpu_ctrl_ack(struct virtqueue *vq);
> +void virtio_gpu_cursor_ack(struct virtqueue *vq);
> +void virtio_gpu_dequeue_ctrl_func(struct work_struct *work);
> +void virtio_gpu_dequeue_cursor_func(struct work_struct *work);
> +
> +/* virtio_gpu_display.c */
> +int virtio_gpu_framebuffer_init(struct drm_device *dev,
> + struct virtio_gpu_framebuffer *vgfb,
> + struct drm_mode_fb_cmd2 *mode_cmd,
> + struct drm_gem_object *obj);
> +int virtio_gpu_modeset_init(struct virtio_gpu_device *vgdev);
> +void virtio_gpu_modeset_fini(struct virtio_gpu_device *vgdev);
> +
> +/* virtio_gpu_plane.c */
> +struct drm_plane *virtio_gpu_plane_init(struct virtio_gpu_device *vgdev,
> + int index);
> +
> +/* virtio_gpu_ttm.c */
> +int virtio_gpu_ttm_init(struct virtio_gpu_device *vgdev);
> +void virtio_gpu_ttm_fini(struct virtio_gpu_device *vgdev);
> +int virtio_gpu_mmap(struct file *filp, struct vm_area_struct *vma);
> +
> +/* virtio_gpu_fence.c */
> +int virtio_gpu_fence_emit(struct virtio_gpu_device *vgdev,
> + struct virtio_gpu_ctrl_hdr *cmd_hdr,
> + struct virtio_gpu_fence **fence);
> +void virtio_gpu_fence_event_process(struct virtio_gpu_device *vdev,
> + u64 last_seq);
> +
> +/* virtio_gpu_object */
> +int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
> + unsigned long size, bool kernel, bool pinned,
> + struct virtio_gpu_object **bo_ptr);
> +int virtio_gpu_object_kmap(struct virtio_gpu_object *bo, void **ptr);
> +int virtio_gpu_object_get_sg_table(struct virtio_gpu_device *qdev,
> + struct virtio_gpu_object *bo);
> +void virtio_gpu_object_free_sg_table(struct virtio_gpu_object *bo);
> +int virtio_gpu_object_wait(struct virtio_gpu_object *bo, bool no_wait);
> +
> +static inline struct virtio_gpu_object*
> +virtio_gpu_object_ref(struct virtio_gpu_object *bo)
> +{
> + ttm_bo_reference(&bo->tbo);
> + return bo;
> +}
> +
> +static inline void virtio_gpu_object_unref(struct virtio_gpu_object **bo)
> +{
> + struct ttm_buffer_object *tbo;
> +
> + if ((*bo) == NULL)
> + return;
> + tbo = &((*bo)->tbo);
> + ttm_bo_unref(&tbo);
> + if (tbo == NULL)
> + *bo = NULL;
> +}
> +
> +static inline u64 virtio_gpu_object_mmap_offset(struct virtio_gpu_object *bo)
> +{
> + return drm_vma_node_offset_addr(&bo->tbo.vma_node);
> +}
> +
> +static inline int virtio_gpu_object_reserve(struct virtio_gpu_object *bo,
> + bool no_wait)
> +{
> + int r;
> +
> + r = ttm_bo_reserve(&bo->tbo, true, no_wait, false, NULL);
> + if (unlikely(r != 0)) {
> + if (r != -ERESTARTSYS) {
> + struct virtio_gpu_device *qdev =
> + bo->gem_base.dev->dev_private;
> + dev_err(qdev->dev, "%p reserve failed\n", bo);
> + }
> + return r;
> + }
> + return 0;
> +}
> +
> +static inline void virtio_gpu_object_unreserve(struct virtio_gpu_object *bo)
> +{
> + ttm_bo_unreserve(&bo->tbo);
> +}
> +
> +/* virgl debufs */
> +int virtio_gpu_debugfs_init(struct drm_minor *minor);
> +void virtio_gpu_debugfs_takedown(struct drm_minor *minor);
> +
> +#endif
> diff --git a/drivers/gpu/drm/virtio/virtgpu_fb.c b/drivers/gpu/drm/virtio/virtgpu_fb.c
> new file mode 100644
> index 0000000..25bf333
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/virtgpu_fb.c
> @@ -0,0 +1,431 @@
> +/*
> + * Copyright (C) 2015 Red Hat, Inc.
> + * All Rights Reserved.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining
> + * a copy of this software and associated documentation files (the
> + * "Software"), to deal in the Software without restriction, including
> + * without limitation the rights to use, copy, modify, merge, publish,
> + * distribute, sublicense, and/or sell copies of the Software, and to
> + * permit persons to whom the Software is furnished to do so, subject to
> + * the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the
> + * next paragraph) shall be included in all copies or substantial
> + * portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
> + * IN NO EVENT SHALL THE COPYRIGHT OWNER(S) AND/OR ITS SUPPLIERS BE
> + * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
> + * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
> + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
> + */
> +
> +#include <drm/drmP.h>
> +#include <drm/drm_fb_helper.h>
> +#include "virtgpu_drv.h"
> +
> +#define VIRTIO_GPU_FBCON_POLL_PERIOD (HZ / 60)
> +
> +struct virtio_gpu_fbdev {
> + struct drm_fb_helper helper;
> + struct virtio_gpu_framebuffer vgfb;
> + struct list_head fbdev_list;
> + struct virtio_gpu_device *vgdev;
> + struct delayed_work work;
> +};
> +
> +static int virtio_gpu_dirty_update(struct virtio_gpu_framebuffer *fb,
> + bool store, int x, int y,
> + int width, int height)
> +{
> + struct drm_device *dev = fb->base.dev;
> + struct virtio_gpu_device *vgdev = dev->dev_private;
> + bool store_for_later = false;
> + int bpp = fb->base.bits_per_pixel / 8;
> + int x2, y2;
> + unsigned long flags;
> + struct virtio_gpu_object *obj = gem_to_virtio_gpu_obj(fb->obj);
> +
> + if ((width <= 0) ||
> + (x + width > fb->base.width) ||
> + (y + height > fb->base.height)) {
> + DRM_DEBUG("values out of range %dx%d+%d+%d, fb %dx%d\n",
> + width, height, x, y,
> + fb->base.width, fb->base.height);
> + return -EINVAL;
> + }
> +
> + /*
> + * Can be called with pretty much any context (console output
> + * path). If we are in atomic just store the dirty rect info
> + * to send out the update later.
> + *
> + * Can't test inside spin lock.
> + */
> + if (in_atomic() || store)
> + store_for_later = true;
> +
> + x2 = x + width - 1;
> + y2 = y + height - 1;
> +
> + spin_lock_irqsave(&fb->dirty_lock, flags);
> +
> + if (fb->y1 < y)
> + y = fb->y1;
> + if (fb->y2 > y2)
> + y2 = fb->y2;
> + if (fb->x1 < x)
> + x = fb->x1;
> + if (fb->x2 > x2)
> + x2 = fb->x2;
> +
> + if (store_for_later) {
> + fb->x1 = x;
> + fb->x2 = x2;
> + fb->y1 = y;
> + fb->y2 = y2;
> + spin_unlock_irqrestore(&fb->dirty_lock, flags);
> + return 0;
> + }
> +
> + fb->x1 = fb->y1 = INT_MAX;
> + fb->x2 = fb->y2 = 0;
> +
> + spin_unlock_irqrestore(&fb->dirty_lock, flags);
> +
> + {
> + uint32_t offset;
> + uint32_t w = x2 - x + 1;
> + uint32_t h = y2 - y + 1;
> +
> + offset = (y * fb->base.pitches[0]) + x * bpp;
> +
> + virtio_gpu_cmd_transfer_to_host_2d(vgdev, obj->hw_res_handle,
> + offset,
> + cpu_to_le32(w),
> + cpu_to_le32(h),
> + cpu_to_le32(x),
> + cpu_to_le32(y),
> + NULL);
> +
> + }
> + virtio_gpu_cmd_resource_flush(vgdev, obj->hw_res_handle,
> + x, y, x2 - x + 1, y2 - y + 1);
> + return 0;
> +}
> +
> +int virtio_gpu_surface_dirty(struct virtio_gpu_framebuffer *vgfb,
> + struct drm_clip_rect *clips,
> + unsigned num_clips)
> +{
> + struct virtio_gpu_device *vgdev = vgfb->base.dev->dev_private;
> + struct virtio_gpu_object *obj = gem_to_virtio_gpu_obj(vgfb->obj);
> + struct drm_clip_rect norect;
> + struct drm_clip_rect *clips_ptr;
> + int left, right, top, bottom;
> + int i;
> + int inc = 1;
> + if (!num_clips) {
> + num_clips = 1;
> + clips = &norect;
> + norect.x1 = norect.y1 = 0;
> + norect.x2 = vgfb->base.width;
> + norect.y2 = vgfb->base.height;
> + }
> + left = clips->x1;
> + right = clips->x2;
> + top = clips->y1;
> + bottom = clips->y2;
> +
> + /* skip the first clip rect */
> + for (i = 1, clips_ptr = clips + inc;
> + i < num_clips; i++, clips_ptr += inc) {
> + left = min_t(int, left, (int)clips_ptr->x1);
> + right = max_t(int, right, (int)clips_ptr->x2);
> + top = min_t(int, top, (int)clips_ptr->y1);
> + bottom = max_t(int, bottom, (int)clips_ptr->y2);
> + }
> +
> + if (obj->dumb)
> + return virtio_gpu_dirty_update(vgfb, false, left, top,
> + right - left, bottom - top);
> +
> + virtio_gpu_cmd_resource_flush(vgdev, obj->hw_res_handle,
> + left, top, right - left, bottom - top);
> + return 0;
> +}
> +
> +static void virtio_gpu_fb_dirty_work(struct work_struct *work)
> +{
> + struct delayed_work *delayed_work = to_delayed_work(work);
> + struct virtio_gpu_fbdev *vfbdev =
> + container_of(delayed_work, struct virtio_gpu_fbdev, work);
> + struct virtio_gpu_framebuffer *vgfb = &vfbdev->vgfb;
> +
> + virtio_gpu_dirty_update(&vfbdev->vgfb, false, vgfb->x1, vgfb->y1,
> + vgfb->x2 - vgfb->x1, vgfb->y2 - vgfb->y1);
> +}
> +
> +static void virtio_gpu_3d_fillrect(struct fb_info *info,
> + const struct fb_fillrect *rect)
> +{
> + struct virtio_gpu_fbdev *vfbdev = info->par;
> + sys_fillrect(info, rect);
> + virtio_gpu_dirty_update(&vfbdev->vgfb, true, rect->dx, rect->dy,
> + rect->width, rect->height);
> + schedule_delayed_work(&vfbdev->work, VIRTIO_GPU_FBCON_POLL_PERIOD);
> +}
> +
> +static void virtio_gpu_3d_copyarea(struct fb_info *info,
> + const struct fb_copyarea *area)
> +{
> + struct virtio_gpu_fbdev *vfbdev = info->par;
> + sys_copyarea(info, area);
> + virtio_gpu_dirty_update(&vfbdev->vgfb, true, area->dx, area->dy,
> + area->width, area->height);
> + schedule_delayed_work(&vfbdev->work, VIRTIO_GPU_FBCON_POLL_PERIOD);
> +}
> +
> +static void virtio_gpu_3d_imageblit(struct fb_info *info,
> + const struct fb_image *image)
> +{
> + struct virtio_gpu_fbdev *vfbdev = info->par;
> + sys_imageblit(info, image);
> + virtio_gpu_dirty_update(&vfbdev->vgfb, true, image->dx, image->dy,
> + image->width, image->height);
> + schedule_delayed_work(&vfbdev->work, VIRTIO_GPU_FBCON_POLL_PERIOD);
> +}
> +
> +static struct fb_ops virtio_gpufb_ops = {
> + .owner = THIS_MODULE,
> + .fb_check_var = drm_fb_helper_check_var,
> + .fb_set_par = drm_fb_helper_set_par, /* TODO: copy vmwgfx */
> + .fb_fillrect = virtio_gpu_3d_fillrect,
> + .fb_copyarea = virtio_gpu_3d_copyarea,
> + .fb_imageblit = virtio_gpu_3d_imageblit,
> + .fb_pan_display = drm_fb_helper_pan_display,
> + .fb_blank = drm_fb_helper_blank,
> + .fb_setcmap = drm_fb_helper_setcmap,
> + .fb_debug_enter = drm_fb_helper_debug_enter,
> + .fb_debug_leave = drm_fb_helper_debug_leave,
> +};
> +
> +static int virtio_gpu_vmap_fb(struct virtio_gpu_device *vgdev,
> + struct virtio_gpu_object *obj)
> +{
> + return virtio_gpu_object_kmap(obj, NULL);
> +}
> +
> +static int virtio_gpufb_create(struct drm_fb_helper *helper,
> + struct drm_fb_helper_surface_size *sizes)
> +{
> + struct virtio_gpu_fbdev *vfbdev =
> + container_of(helper, struct virtio_gpu_fbdev, helper);
> + struct drm_device *dev = helper->dev;
> + struct virtio_gpu_device *vgdev = dev->dev_private;
> + struct fb_info *info;
> + struct drm_framebuffer *fb;
> + struct drm_mode_fb_cmd2 mode_cmd = {};
> + struct virtio_gpu_object *obj;
> + struct device *device = vgdev->dev;
> + uint32_t resid, format, size;
> + int ret;
> +
> + mode_cmd.width = sizes->surface_width;
> + mode_cmd.height = sizes->surface_height;
> + mode_cmd.pitches[0] = mode_cmd.width * 4;
> + mode_cmd.pixel_format = drm_mode_legacy_fb_format(32, 24);
> +
> + switch (mode_cmd.pixel_format) {
> +#ifdef __BIG_ENDIAN
> + case DRM_FORMAT_XRGB8888:
> + format = VIRTIO_GPU_FORMAT_X8R8G8B8_UNORM;
> + break;
> + case DRM_FORMAT_ARGB8888:
> + format = VIRTIO_GPU_FORMAT_A8R8G8B8_UNORM;
> + break;
> + case DRM_FORMAT_BGRX8888:
> + format = VIRTIO_GPU_FORMAT_B8G8R8X8_UNORM;
> + break;
> + case DRM_FORMAT_BGRA8888:
> + format = VIRTIO_GPU_FORMAT_B8G8R8A8_UNORM;
> + break;
> + case DRM_FORMAT_RGBX8888:
> + format = VIRTIO_GPU_FORMAT_R8G8B8X8_UNORM;
> + break;
> + case DRM_FORMAT_RGBA8888:
> + format = VIRTIO_GPU_FORMAT_R8G8B8A8_UNORM;
> + break;
> + case DRM_FORMAT_XBGR8888:
> + format = VIRTIO_GPU_FORMAT_X8B8G8R8_UNORM;
> + break;
> + case DRM_FORMAT_ABGR8888:
> + format = VIRTIO_GPU_FORMAT_A8B8G8R8_UNORM;
> + break;
> +#else
> + case DRM_FORMAT_XRGB8888:
> + format = VIRTIO_GPU_FORMAT_B8G8R8X8_UNORM;
> + break;
> + case DRM_FORMAT_ARGB8888:
> + format = VIRTIO_GPU_FORMAT_B8G8R8A8_UNORM;
> + break;
> + case DRM_FORMAT_BGRX8888:
> + format = VIRTIO_GPU_FORMAT_X8R8G8B8_UNORM;
> + break;
> + case DRM_FORMAT_BGRA8888:
> + format = VIRTIO_GPU_FORMAT_A8R8G8B8_UNORM;
> + break;
> + case DRM_FORMAT_RGBX8888:
> + format = VIRTIO_GPU_FORMAT_X8B8G8R8_UNORM;
> + break;
> + case DRM_FORMAT_RGBA8888:
> + format = VIRTIO_GPU_FORMAT_A8B8G8R8_UNORM;
> + break;
> + case DRM_FORMAT_XBGR8888:
> + format = VIRTIO_GPU_FORMAT_R8G8B8X8_UNORM;
> + break;
> + case DRM_FORMAT_ABGR8888:
> + format = VIRTIO_GPU_FORMAT_R8G8B8A8_UNORM;
> + break;
> +#endif
> + default:
> + DRM_ERROR("failed to find virtio gpu format for %d\n",
> + mode_cmd.pixel_format);
> + return -EINVAL;
> + }
> +
> + size = mode_cmd.pitches[0] * mode_cmd.height;
> + obj = virtio_gpu_alloc_object(dev, size, false, true);
> + if (!obj)
> + return -ENOMEM;
> +
> + virtio_gpu_resource_id_get(vgdev, &resid);
> + virtio_gpu_cmd_create_resource(vgdev, resid, format,
> + mode_cmd.width, mode_cmd.height);
> +
> + ret = virtio_gpu_vmap_fb(vgdev, obj);
> + if (ret) {
> + DRM_ERROR("failed to vmap fb %d\n", ret);
> + goto err_obj_vmap;
> + }
> +
> + /* attach the object to the resource */
> + ret = virtio_gpu_object_attach(vgdev, obj, resid, NULL);
> + if (ret)
> + goto err_obj_attach;
> +
> + info = framebuffer_alloc(0, device);
> + if (!info) {
> + ret = -ENOMEM;
> + goto err_fb_alloc;
> + }
> +
> + ret = fb_alloc_cmap(&info->cmap, 256, 0);
> + if (ret) {
> + ret = -ENOMEM;
> + goto err_fb_alloc_cmap;
> + }
> +
> + info->par = helper;
> +
> + ret = virtio_gpu_framebuffer_init(dev, &vfbdev->vgfb,
> + &mode_cmd, &obj->gem_base);
> + if (ret)
> + goto err_fb_init;
> +
> + fb = &vfbdev->vgfb.base;
> +
> + vfbdev->helper.fb = fb;
> + vfbdev->helper.fbdev = info;
> +
> + strcpy(info->fix.id, "virtiodrmfb");
> + info->flags = FBINFO_DEFAULT;
> + info->fbops = &virtio_gpufb_ops;
> + info->pixmap.flags = FB_PIXMAP_SYSTEM;
> +
> + info->screen_base = obj->vmap;
> + info->screen_size = obj->gem_base.size;
> + drm_fb_helper_fill_fix(info, fb->pitches[0], fb->depth);
> + drm_fb_helper_fill_var(info, &vfbdev->helper,
> + sizes->fb_width, sizes->fb_height);
> +
> + info->fix.mmio_start = 0;
> + info->fix.mmio_len = 0;
> + return 0;
> +
> +err_fb_init:
> + fb_dealloc_cmap(&info->cmap);
> +err_fb_alloc_cmap:
> + framebuffer_release(info);
> +err_fb_alloc:
> + virtio_gpu_cmd_resource_inval_backing(vgdev, resid);
> +err_obj_attach:
> +err_obj_vmap:
> + virtio_gpu_gem_free_object(&obj->gem_base);
> + return ret;
> +}
> +
> +static int virtio_gpu_fbdev_destroy(struct drm_device *dev,
> + struct virtio_gpu_fbdev *vgfbdev)
> +{
> + struct fb_info *info;
> + struct virtio_gpu_framebuffer *vgfb = &vgfbdev->vgfb;
> +
> + if (vgfbdev->helper.fbdev) {
> + info = vgfbdev->helper.fbdev;
> +
> + unregister_framebuffer(info);
> + framebuffer_release(info);
> + }
> + if (vgfb->obj)
> + vgfb->obj = NULL;
> + drm_fb_helper_fini(&vgfbdev->helper);
> + drm_framebuffer_cleanup(&vgfb->base);
> +
> + return 0;
> +}
> +static struct drm_fb_helper_funcs virtio_gpu_fb_helper_funcs = {
> + .fb_probe = virtio_gpufb_create,
> +};
> +
> +int virtio_gpu_fbdev_init(struct virtio_gpu_device *vgdev)
> +{
> + struct virtio_gpu_fbdev *vgfbdev;
> + int bpp_sel = 32; /* TODO: parameter from somewhere? */
> + int ret;
> +
> + vgfbdev = kzalloc(sizeof(struct virtio_gpu_fbdev), GFP_KERNEL);
> + if (!vgfbdev)
> + return -ENOMEM;
> +
> + vgfbdev->vgdev = vgdev;
> + vgdev->vgfbdev = vgfbdev;
> + INIT_DELAYED_WORK(&vgfbdev->work, virtio_gpu_fb_dirty_work);
> +
> + drm_fb_helper_prepare(vgdev->ddev, &vgfbdev->helper,
> + &virtio_gpu_fb_helper_funcs);
> + ret = drm_fb_helper_init(vgdev->ddev, &vgfbdev->helper,
> + vgdev->num_scanouts,
> + VIRTIO_GPUFB_CONN_LIMIT);
> + if (ret) {
> + kfree(vgfbdev);
> + return ret;
> + }
> +
> + drm_fb_helper_single_add_all_connectors(&vgfbdev->helper);
> + drm_fb_helper_initial_config(&vgfbdev->helper, bpp_sel);
> + return 0;
> +}
> +
> +void virtio_gpu_fbdev_fini(struct virtio_gpu_device *vgdev)
> +{
> + if (!vgdev->vgfbdev)
> + return;
> +
> + virtio_gpu_fbdev_destroy(vgdev->ddev, vgdev->vgfbdev);
> + kfree(vgdev->vgfbdev);
> + vgdev->vgfbdev = NULL;
> +}
> diff --git a/drivers/gpu/drm/virtio/virtgpu_fence.c b/drivers/gpu/drm/virtio/virtgpu_fence.c
> new file mode 100644
> index 0000000..1da6326
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/virtgpu_fence.c
> @@ -0,0 +1,119 @@
> +/*
> + * Copyright (C) 2015 Red Hat, Inc.
> + * All Rights Reserved.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining
> + * a copy of this software and associated documentation files (the
> + * "Software"), to deal in the Software without restriction, including
> + * without limitation the rights to use, copy, modify, merge, publish,
> + * distribute, sublicense, and/or sell copies of the Software, and to
> + * permit persons to whom the Software is furnished to do so, subject to
> + * the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the
> + * next paragraph) shall be included in all copies or substantial
> + * portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
> + * IN NO EVENT SHALL THE COPYRIGHT OWNER(S) AND/OR ITS SUPPLIERS BE
> + * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
> + * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
> + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
> + */
> +
> +#include <drm/drmP.h>
> +#include "virtgpu_drv.h"
> +
> +static const char *virtio_get_driver_name(struct fence *f)
> +{
> + return "virtio_gpu";
> +}
> +
> +static const char *virtio_get_timeline_name(struct fence *f)
> +{
> + return "controlq";
> +}
> +
> +static bool virtio_enable_signaling(struct fence *f)
> +{
> + return true;
> +}
> +
> +static bool virtio_signaled(struct fence *f)
> +{
> + struct virtio_gpu_fence *fence = to_virtio_fence(f);
> +
> + if (atomic64_read(&fence->drv->last_seq) >= fence->seq)
> + return true;
> + return false;
> +}
> +
> +static void virtio_fence_value_str(struct fence *f, char *str, int size)
> +{
> + struct virtio_gpu_fence *fence = to_virtio_fence(f);
> +
> + snprintf(str, size, "%llu", fence->seq);
> +}
> +
> +static void virtio_timeline_value_str(struct fence *f, char *str, int size)
> +{
> + struct virtio_gpu_fence *fence = to_virtio_fence(f);
> +
> + snprintf(str, size, "%lu", atomic64_read(&fence->drv->last_seq));
> +}
> +
> +static const struct fence_ops virtio_fence_ops = {
> + .get_driver_name = virtio_get_driver_name,
> + .get_timeline_name = virtio_get_timeline_name,
> + .enable_signaling = virtio_enable_signaling,
> + .signaled = virtio_signaled,
> + .wait = fence_default_wait,
> + .fence_value_str = virtio_fence_value_str,
> + .timeline_value_str = virtio_timeline_value_str,
> +};
> +
> +int virtio_gpu_fence_emit(struct virtio_gpu_device *vgdev,
> + struct virtio_gpu_ctrl_hdr *cmd_hdr,
> + struct virtio_gpu_fence **fence)
> +{
> + struct virtio_gpu_fence_driver *drv = &vgdev->fence_drv;
> + unsigned long irq_flags;
> +
> + *fence = kmalloc(sizeof(struct virtio_gpu_fence), GFP_KERNEL);
> + if ((*fence) == NULL)
> + return -ENOMEM;
> +
> + spin_lock_irqsave(&drv->lock, irq_flags);
> + (*fence)->drv = drv;
> + (*fence)->seq = ++drv->sync_seq;
> + fence_init(&(*fence)->f, &virtio_fence_ops, &drv->lock,
> + 0, (*fence)->seq);
> + fence_get(&(*fence)->f);
> + list_add_tail(&(*fence)->node, &drv->fences);
> + spin_unlock_irqrestore(&drv->lock, irq_flags);
> +
> + cmd_hdr->flags |= cpu_to_le32(VIRTIO_GPU_FLAG_FENCE);
> + cmd_hdr->fence_id = cpu_to_le64((*fence)->seq);
> + return 0;
> +}
> +
> +void virtio_gpu_fence_event_process(struct virtio_gpu_device *vgdev,
> + u64 last_seq)
> +{
> + struct virtio_gpu_fence_driver *drv = &vgdev->fence_drv;
> + struct virtio_gpu_fence *fence, *tmp;
> + unsigned long irq_flags;
> +
> + spin_lock_irqsave(&drv->lock, irq_flags);
> + atomic64_set(&vgdev->fence_drv.last_seq, last_seq);
> + list_for_each_entry_safe(fence, tmp, &drv->fences, node) {
> + if (last_seq < fence->seq)
> + continue;
> + fence_signal_locked(&fence->f);
> + list_del(&fence->node);
> + fence_put(&fence->f);
> + }
> + spin_unlock_irqrestore(&drv->lock, irq_flags);
> +}
> diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c b/drivers/gpu/drm/virtio/virtgpu_gem.c
> new file mode 100644
> index 0000000..cfa0d27
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/virtgpu_gem.c
> @@ -0,0 +1,140 @@
> +/*
> + * Copyright (C) 2015 Red Hat, Inc.
> + * All Rights Reserved.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining
> + * a copy of this software and associated documentation files (the
> + * "Software"), to deal in the Software without restriction, including
> + * without limitation the rights to use, copy, modify, merge, publish,
> + * distribute, sublicense, and/or sell copies of the Software, and to
> + * permit persons to whom the Software is furnished to do so, subject to
> + * the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the
> + * next paragraph) shall be included in all copies or substantial
> + * portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
> + * IN NO EVENT SHALL THE COPYRIGHT OWNER(S) AND/OR ITS SUPPLIERS BE
> + * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
> + * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
> + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
> + */
> +
> +#include <drm/drmP.h>
> +#include "virtgpu_drv.h"
> +
> +void virtio_gpu_gem_free_object(struct drm_gem_object *gem_obj)
> +{
> + struct virtio_gpu_object *obj = gem_to_virtio_gpu_obj(gem_obj);
> +
> + if (obj)
> + virtio_gpu_object_unref(&obj);
> +}
> +
> +struct virtio_gpu_object *virtio_gpu_alloc_object(struct drm_device *dev,
> + size_t size, bool kernel,
> + bool pinned)
> +{
> + struct virtio_gpu_device *vgdev = dev->dev_private;
> + struct virtio_gpu_object *obj;
> + int ret;
> +
> + ret = virtio_gpu_object_create(vgdev, size, kernel, pinned, &obj);
> + if (ret)
> + return ERR_PTR(ret);
> +
> + return obj;
> +}
> +
> +int virtio_gpu_gem_create(struct drm_file *file,
> + struct drm_device *dev,
> + uint64_t size,
> + struct drm_gem_object **obj_p,
> + uint32_t *handle_p)
> +{
> + struct virtio_gpu_object *obj;
> + int ret;
> + u32 handle;
> +
> + obj = virtio_gpu_alloc_object(dev, size, false, false);
> + if (IS_ERR(obj))
> + return PTR_ERR(obj);
> +
> + ret = drm_gem_handle_create(file, &obj->gem_base, &handle);
> + if (ret) {
> + drm_gem_object_release(&obj->gem_base);
> + return ret;
> + }
> +
> + *obj_p = &obj->gem_base;
> +
> + /* drop reference from allocate - handle holds it now */
> + drm_gem_object_unreference_unlocked(&obj->gem_base);
> +
> + *handle_p = handle;
> + return 0;
> +}
> +
> +int virtio_gpu_mode_dumb_create(struct drm_file *file_priv,
> + struct drm_device *dev,
> + struct drm_mode_create_dumb *args)
> +{
> + struct virtio_gpu_device *vgdev = dev->dev_private;
> + struct drm_gem_object *gobj;
> + struct virtio_gpu_object *obj;
> + int ret;
> + uint32_t pitch;
> + uint32_t resid;
> +
> + pitch = args->width * ((args->bpp + 1) / 8);
> + args->size = pitch * args->height;
> + args->size = ALIGN(args->size, PAGE_SIZE);
> +
> + ret = virtio_gpu_gem_create(file_priv, dev, args->size, &gobj,
> + &args->handle);
> + if (ret)
> + goto fail;
> +
> + virtio_gpu_resource_id_get(vgdev, &resid);
> + virtio_gpu_cmd_create_resource(vgdev, resid,
> + 2, args->width, args->height);
> +
> + /* attach the object to the resource */
> + obj = gem_to_virtio_gpu_obj(gobj);
> + ret = virtio_gpu_object_attach(vgdev, obj, resid, NULL);
> + if (ret)
> + goto fail;
> +
> + obj->dumb = true;
> + args->pitch = pitch;
> + return ret;
> +
> +fail:
> + return ret;
> +}
> +
> +int virtio_gpu_mode_dumb_destroy(struct drm_file *file_priv,
> + struct drm_device *dev,
> + uint32_t handle)
> +{
> + return drm_gem_handle_delete(file_priv, handle);
> +}
> +
> +int virtio_gpu_mode_dumb_mmap(struct drm_file *file_priv,
> + struct drm_device *dev,
> + uint32_t handle, uint64_t *offset_p)
> +{
> + struct drm_gem_object *gobj;
> + struct virtio_gpu_object *obj;
> + BUG_ON(!offset_p);
> + gobj = drm_gem_object_lookup(dev, file_priv, handle);
> + if (gobj == NULL)
> + return -ENOENT;
> + obj = gem_to_virtio_gpu_obj(gobj);
> + *offset_p = virtio_gpu_object_mmap_offset(obj);
> + drm_gem_object_unreference_unlocked(gobj);
> + return 0;
> +}
> diff --git a/drivers/gpu/drm/virtio/virtgpu_kms.c b/drivers/gpu/drm/virtio/virtgpu_kms.c
> new file mode 100644
> index 0000000..e503ffb
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/virtgpu_kms.c
> @@ -0,0 +1,164 @@
> +/*
> + * Copyright (C) 2015 Red Hat, Inc.
> + * All Rights Reserved.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining
> + * a copy of this software and associated documentation files (the
> + * "Software"), to deal in the Software without restriction, including
> + * without limitation the rights to use, copy, modify, merge, publish,
> + * distribute, sublicense, and/or sell copies of the Software, and to
> + * permit persons to whom the Software is furnished to do so, subject to
> + * the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the
> + * next paragraph) shall be included in all copies or substantial
> + * portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
> + * IN NO EVENT SHALL THE COPYRIGHT OWNER(S) AND/OR ITS SUPPLIERS BE
> + * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
> + * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
> + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
> + */
> +
> +#include <linux/virtio.h>
> +#include <linux/virtio_config.h>
> +#include <drm/drmP.h>
> +#include "virtgpu_drv.h"
> +
> +static void virtio_gpu_config_changed_work_func(struct work_struct *work)
> +{
> + struct virtio_gpu_device *vgdev =
> + container_of(work, struct virtio_gpu_device,
> + config_changed_work);
> + u32 events_read, events_clear = 0;
> +
> + /* read the config space */
> + virtio_cread(vgdev->vdev, struct virtio_gpu_config,
> + events_read, &events_read);
> + if (events_read & VIRTIO_GPU_EVENT_DISPLAY) {
> + virtio_gpu_cmd_get_display_info(vgdev);
> + drm_helper_hpd_irq_event(vgdev->ddev);
> + events_clear |= VIRTIO_GPU_EVENT_DISPLAY;
> + }
> + virtio_cwrite(vgdev->vdev, struct virtio_gpu_config,
> + events_clear, &events_clear);
> +}
> +
> +static void virtio_gpu_init_vq(struct virtio_gpu_queue *vgvq,
> + void (*work_func)(struct work_struct *work))
> +{
> + spin_lock_init(&vgvq->qlock);
> + init_waitqueue_head(&vgvq->ack_queue);
> + INIT_WORK(&vgvq->dequeue_work, work_func);
> +}
> +
> +int virtio_gpu_driver_load(struct drm_device *dev, unsigned long flags)
> +{
> + static vq_callback_t *callbacks[] = {
> + virtio_gpu_ctrl_ack, virtio_gpu_cursor_ack
> + };
> + static const char *names[] = { "control", "cursor" };
> +
> + struct virtio_gpu_device *vgdev;
> + /* this will expand later */
> + struct virtqueue *vqs[2];
> + u32 num_scanouts;
> + int ret;
> +
> + if (!virtio_has_feature(dev->virtdev, VIRTIO_F_VERSION_1))
> + return -ENODEV;
> +
> + vgdev = kzalloc(sizeof(struct virtio_gpu_device), GFP_KERNEL);
> + if (!vgdev)
> + return -ENOMEM;
> +
> + vgdev->ddev = dev;
> + dev->dev_private = vgdev;
> + vgdev->vdev = dev->virtdev;
> + vgdev->dev = dev->dev;
> +
> + spin_lock_init(&vgdev->display_info_lock);
> + spin_lock_init(&vgdev->ctx_id_idr_lock);
> + idr_init(&vgdev->ctx_id_idr);
> + spin_lock_init(&vgdev->resource_idr_lock);
> + idr_init(&vgdev->resource_idr);
> + init_waitqueue_head(&vgdev->resp_wq);
> + virtio_gpu_init_vq(&vgdev->ctrlq, virtio_gpu_dequeue_ctrl_func);
> + virtio_gpu_init_vq(&vgdev->cursorq, virtio_gpu_dequeue_cursor_func);
> +
> + spin_lock_init(&vgdev->fence_drv.lock);
> + INIT_LIST_HEAD(&vgdev->fence_drv.fences);
> + INIT_WORK(&vgdev->config_changed_work,
> + virtio_gpu_config_changed_work_func);
> +
> + ret = vgdev->vdev->config->find_vqs(vgdev->vdev, 2, vqs,
> + callbacks, names);
> + if (ret) {
> + DRM_ERROR("failed to find virt queues\n");
> + goto err_vqs;
> + }
> + vgdev->ctrlq.vq = vqs[0];
> + vgdev->cursorq.vq = vqs[1];
> + ret = virtio_gpu_alloc_vbufs(vgdev);
> + if (ret) {
> + DRM_ERROR("failed to alloc vbufs\n");
> + goto err_vbufs;
> + }
> +
> + ret = virtio_gpu_ttm_init(vgdev);
> + if (ret) {
> + DRM_ERROR("failed to init ttm %d\n", ret);
> + goto err_ttm;
> + }
> +
> + /* get display info */
> + virtio_cread(vgdev->vdev, struct virtio_gpu_config,
> + num_scanouts, &num_scanouts);
> + vgdev->num_scanouts = min_t(uint32_t, num_scanouts,
> + VIRTIO_GPU_MAX_SCANOUTS);
> + if (!vgdev->num_scanouts) {
> + DRM_ERROR("num_scanouts is zero\n");
> + ret = -EINVAL;
> + goto err_scanouts;
> + }
> +
> + ret = virtio_gpu_modeset_init(vgdev);
> + if (ret)
> + goto err_modeset;
> +
> + virtio_device_ready(vgdev->vdev);
> + vgdev->vqs_ready = true;
> + virtio_gpu_cmd_get_display_info(vgdev);
> + return 0;
> +
> +err_modeset:
> +err_scanouts:
> + virtio_gpu_ttm_fini(vgdev);
> +err_ttm:
> + virtio_gpu_free_vbufs(vgdev);
> +err_vbufs:
> + vgdev->vdev->config->del_vqs(vgdev->vdev);
> +err_vqs:
> + kfree(vgdev);
> + return ret;
> +}
> +
> +int virtio_gpu_driver_unload(struct drm_device *dev)
> +{
> + struct virtio_gpu_device *vgdev = dev->dev_private;
> +
> + vgdev->vqs_ready = false;
> + flush_work(&vgdev->ctrlq.dequeue_work);
> + flush_work(&vgdev->cursorq.dequeue_work);
> + flush_work(&vgdev->config_changed_work);
> + vgdev->vdev->config->del_vqs(vgdev->vdev);
> +
> + virtio_gpu_modeset_fini(vgdev);
> + virtio_gpu_ttm_fini(vgdev);
> + virtio_gpu_free_vbufs(vgdev);
> + kfree(vgdev);
> + return 0;
> +}
> diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c
> new file mode 100644
> index 0000000..2c624c7
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/virtgpu_object.c
> @@ -0,0 +1,170 @@
> +/*
> + * Copyright (C) 2015 Red Hat, Inc.
> + * All Rights Reserved.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining
> + * a copy of this software and associated documentation files (the
> + * "Software"), to deal in the Software without restriction, including
> + * without limitation the rights to use, copy, modify, merge, publish,
> + * distribute, sublicense, and/or sell copies of the Software, and to
> + * permit persons to whom the Software is furnished to do so, subject to
> + * the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the
> + * next paragraph) shall be included in all copies or substantial
> + * portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
> + * IN NO EVENT SHALL THE COPYRIGHT OWNER(S) AND/OR ITS SUPPLIERS BE
> + * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
> + * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
> + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
> + */
> +
> +#include "virtgpu_drv.h"
> +
> +static void virtio_gpu_ttm_bo_destroy(struct ttm_buffer_object *tbo)
> +{
> + struct virtio_gpu_object *bo;
> + struct virtio_gpu_device *vgdev;
> +
> + bo = container_of(tbo, struct virtio_gpu_object, tbo);
> + vgdev = (struct virtio_gpu_device *)bo->gem_base.dev->dev_private;
> +
> + if (bo->hw_res_handle)
> + virtio_gpu_cmd_unref_resource(vgdev, bo->hw_res_handle);
> + if (bo->pages)
> + virtio_gpu_object_free_sg_table(bo);
> + drm_gem_object_release(&bo->gem_base);
> + kfree(bo);
> +}
> +
> +static void virtio_gpu_init_ttm_placement(struct virtio_gpu_object *vgbo,
> + bool pinned)
> +{
> + u32 c = 1;
> + u32 pflag = pinned ? TTM_PL_FLAG_NO_EVICT : 0;
> +
> + vgbo->placement.placement = &vgbo->placement_code;
> + vgbo->placement.busy_placement = &vgbo->placement_code;
> + vgbo->placement_code.fpfn = 0;
> + vgbo->placement_code.lpfn = 0;
> + vgbo->placement_code.flags =
> + TTM_PL_MASK_CACHING | TTM_PL_FLAG_TT | pflag;
> + vgbo->placement.num_placement = c;
> + vgbo->placement.num_busy_placement = c;
> +
> +}
> +
> +int virtio_gpu_object_create(struct virtio_gpu_device *vgdev,
> + unsigned long size, bool kernel, bool pinned,
> + struct virtio_gpu_object **bo_ptr)
> +{
> + struct virtio_gpu_object *bo;
> + enum ttm_bo_type type;
> + size_t acc_size;
> + int ret;
> +
> + if (kernel)
> + type = ttm_bo_type_kernel;
> + else
> + type = ttm_bo_type_device;
> + *bo_ptr = NULL;
> +
> + acc_size = ttm_bo_dma_acc_size(&vgdev->mman.bdev, size,
> + sizeof(struct virtio_gpu_object));
> +
> + bo = kzalloc(sizeof(struct virtio_gpu_object), GFP_KERNEL);
> + if (bo == NULL)
> + return -ENOMEM;
> + size = roundup(size, PAGE_SIZE);
> + ret = drm_gem_object_init(vgdev->ddev, &bo->gem_base, size);
> + if (ret != 0)
> + goto err_gem_init;
> + bo->dumb = false;
> + virtio_gpu_init_ttm_placement(bo, pinned);
> +
> + ret = ttm_bo_init(&vgdev->mman.bdev, &bo->tbo, size, type,
> + &bo->placement, 0, !kernel, NULL, acc_size,
> + NULL, NULL, &virtio_gpu_ttm_bo_destroy);
> + if (ret != 0)
> + goto err_ttm_init;
> +
> + *bo_ptr = bo;
> + return 0;
> +
> +err_ttm_init:
> + drm_gem_object_release(&bo->gem_base);
> +err_gem_init:
> + kfree(bo);
> + return ret;
> +}
> +
> +int virtio_gpu_object_kmap(struct virtio_gpu_object *bo, void **ptr)
> +{
> + bool is_iomem;
> + int r;
> +
> + if (bo->vmap) {
> + if (ptr)
> + *ptr = bo->vmap;
> + return 0;
> + }
> + r = ttm_bo_kmap(&bo->tbo, 0, bo->tbo.num_pages, &bo->kmap);
> + if (r)
> + return r;
> + bo->vmap = ttm_kmap_obj_virtual(&bo->kmap, &is_iomem);
> + if (ptr)
> + *ptr = bo->vmap;
> + return 0;
> +}
> +
> +int virtio_gpu_object_get_sg_table(struct virtio_gpu_device *qdev,
> + struct virtio_gpu_object *bo)
> +{
> + int ret;
> + struct page **pages = bo->tbo.ttm->pages;
> + int nr_pages = bo->tbo.num_pages;
> +
> + /* wtf swapping */
> + if (bo->pages)
> + return 0;
> +
> + if (bo->tbo.ttm->state == tt_unpopulated)
> + bo->tbo.ttm->bdev->driver->ttm_tt_populate(bo->tbo.ttm);
> + bo->pages = kmalloc(sizeof(struct sg_table), GFP_KERNEL);
> + if (!bo->pages)
> + goto out;
> +
> + ret = sg_alloc_table_from_pages(bo->pages, pages, nr_pages, 0,
> + nr_pages << PAGE_SHIFT, GFP_KERNEL);
> + if (ret)
> + goto out;
> + return 0;
> +out:
> + kfree(bo->pages);
> + bo->pages = NULL;
> + return -ENOMEM;
> +}
> +
> +void virtio_gpu_object_free_sg_table(struct virtio_gpu_object *bo)
> +{
> + sg_free_table(bo->pages);
> + kfree(bo->pages);
> + bo->pages = NULL;
> +}
> +
> +int virtio_gpu_object_wait(struct virtio_gpu_object *bo, bool no_wait)
> +{
> + int r;
> +
> + r = ttm_bo_reserve(&bo->tbo, true, no_wait, false, NULL);
> + if (unlikely(r != 0))
> + return r;
> + r = ttm_bo_wait(&bo->tbo, true, true, no_wait);
> + ttm_bo_unreserve(&bo->tbo);
> + return r;
> +}
> +
> diff --git a/drivers/gpu/drm/virtio/virtgpu_plane.c b/drivers/gpu/drm/virtio/virtgpu_plane.c
> new file mode 100644
> index 0000000..4a74129
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/virtgpu_plane.c
> @@ -0,0 +1,120 @@
> +/*
> + * Copyright (C) 2015 Red Hat, Inc.
> + * All Rights Reserved.
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining
> + * a copy of this software and associated documentation files (the
> + * "Software"), to deal in the Software without restriction, including
> + * without limitation the rights to use, copy, modify, merge, publish,
> + * distribute, sublicense, and/or sell copies of the Software, and to
> + * permit persons to whom the Software is furnished to do so, subject to
> + * the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the
> + * next paragraph) shall be included in all copies or substantial
> + * portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
> + * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> + * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
> + * IN NO EVENT SHALL THE COPYRIGHT OWNER(S) AND/OR ITS SUPPLIERS BE
> + * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
> + * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
> + * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
> + */
> +
> +#include "virtgpu_drv.h"
> +#include <drm/drm_plane_helper.h>
> +#include <drm/drm_atomic_helper.h>
> +
> +static const uint32_t virtio_gpu_formats[] = {
> + DRM_FORMAT_XRGB8888,
> + DRM_FORMAT_ARGB8888,
> + DRM_FORMAT_BGRX8888,
> + DRM_FORMAT_BGRA8888,
> + DRM_FORMAT_RGBX8888,
> + DRM_FORMAT_RGBA8888,
> + DRM_FORMAT_XBGR8888,
> + DRM_FORMAT_ABGR8888,
> +};
> +
> +static void virtio_gpu_plane_destroy(struct drm_plane *plane)
> +{
> + kfree(plane);
> +}
> +
> +static const struct drm_plane_funcs virtio_gpu_plane_funcs = {
> + .update_plane = drm_atomic_helper_update_plane,
> + .disable_plane = drm_atomic_helper_disable_plane,
> + .destroy = virtio_gpu_plane_destroy,
> + .reset = drm_atomic_helper_plane_reset,
> + .atomic_duplicate_state = drm_atomic_helper_plane_duplicate_state,
> + .atomic_destroy_state = drm_atomic_helper_plane_destroy_state,
> +};
> +
> +static int virtio_gpu_plane_atomic_check(struct drm_plane *plane,
> + struct drm_plane_state *state)
> +{
> + return 0;
> +}
> +
> +static void virtio_gpu_plane_atomic_update(struct drm_plane *plane,
> + struct drm_plane_state *old_state)
> +{
> + struct drm_device *dev = plane->dev;
> + struct virtio_gpu_device *vgdev = dev->dev_private;
> + struct virtio_gpu_output *output = drm_crtc_to_virtio_gpu_output(plane->crtc);
> + struct virtio_gpu_framebuffer *vgfb;
> + struct virtio_gpu_object *bo;
> + uint32_t handle;
> +
> + if (plane->fb) {
> + vgfb = to_virtio_gpu_framebuffer(plane->fb);
> + bo = gem_to_virtio_gpu_obj(vgfb->obj);
> + handle = bo->hw_res_handle;
> + } else {
> + handle = 0;
> + }
> +
> + DRM_DEBUG("handle 0x%x, crtc %dx%d+%d+%d\n", handle,
> + plane->state->crtc_w, plane->state->crtc_h,
> + plane->state->crtc_x, plane->state->crtc_y);
> + virtio_gpu_cmd_set_scanout(vgdev, output->index, handle,
> + plane->state->crtc_w,
> + plane->state->crtc_h,
> + plane->state->crtc_x,
> + plane->state->crtc_y);
> +}
> +
> +
> +static const struct drm_plane_helper_funcs virtio_gpu_plane_helper_funcs = {
> + .atomic_check = virtio_gpu_plane_atomic_check,
> + .atomic_update = virtio_gpu_plane_atomic_update,
> +};
> +
> +struct drm_plane *virtio_gpu_plane_init(struct virtio_gpu_device *vgdev,
> + int index)
> +{
> + struct drm_device *dev = vgdev->ddev;
> + struct drm_plane *plane;
> + int ret;
> +
> + plane = kzalloc(sizeof(*plane), GFP_KERNEL);
> + if (!plane)
> + return ERR_PTR(-ENOMEM);
> +
> + ret = drm_universal_plane_init(dev, plane, 1 << index,
> + &virtio_gpu_plane_funcs,
> + virtio_gpu_formats,
> + ARRAY_SIZE(virtio_gpu_formats),
> + DRM_PLANE_TYPE_PRIMARY);
> + if (ret)
> + goto err_plane_init;
> +
> + drm_plane_helper_add(plane, &virtio_gpu_plane_helper_funcs);
> + return plane;
> +
> +err_plane_init:
> + kfree(plane);
> + return ERR_PTR(ret);
> +}
> diff --git a/drivers/gpu/drm/virtio/virtgpu_ttm.c b/drivers/gpu/drm/virtio/virtgpu_ttm.c
> new file mode 100644
> index 0000000..e0e74c6
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/virtgpu_ttm.c
> @@ -0,0 +1,469 @@
> +/*
> + * Copyright (C) 2015 Red Hat, Inc.
> + * All Rights Reserved.
> + *
> + * Authors:
> + * Dave Airlie
> + * Alon Levy
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice shall be included in
> + * all copies or substantial portions of the Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
> + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
> + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> + * OTHER DEALINGS IN THE SOFTWARE.
> + */
> +
> +#include <ttm/ttm_bo_api.h>
> +#include <ttm/ttm_bo_driver.h>
> +#include <ttm/ttm_placement.h>
> +#include <ttm/ttm_page_alloc.h>
> +#include <ttm/ttm_module.h>
> +#include <drm/drmP.h>
> +#include <drm/drm.h>
> +#include "virtgpu_drv.h"
> +
> +#include <linux/delay.h>
> +
> +#define DRM_FILE_PAGE_OFFSET (0x100000000ULL >> PAGE_SHIFT)
> +
> +static struct
> +virtio_gpu_device *virtio_gpu_get_vgdev(struct ttm_bo_device *bdev)
> +{
> + struct virtio_gpu_mman *mman;
> + struct virtio_gpu_device *vgdev;
> +
> + mman = container_of(bdev, struct virtio_gpu_mman, bdev);
> + vgdev = container_of(mman, struct virtio_gpu_device, mman);
> + return vgdev;
> +}
> +
> +static int virtio_gpu_ttm_mem_global_init(struct drm_global_reference *ref)
> +{
> + return ttm_mem_global_init(ref->object);
> +}
> +
> +static void virtio_gpu_ttm_mem_global_release(struct drm_global_reference *ref)
> +{
> + ttm_mem_global_release(ref->object);
> +}
> +
> +static int virtio_gpu_ttm_global_init(struct virtio_gpu_device *vgdev)
> +{
> + struct drm_global_reference *global_ref;
> + int r;
> +
> + vgdev->mman.mem_global_referenced = false;
> + global_ref = &vgdev->mman.mem_global_ref;
> + global_ref->global_type = DRM_GLOBAL_TTM_MEM;
> + global_ref->size = sizeof(struct ttm_mem_global);
> + global_ref->init = &virtio_gpu_ttm_mem_global_init;
> + global_ref->release = &virtio_gpu_ttm_mem_global_release;
> +
> + r = drm_global_item_ref(global_ref);
> + if (r != 0) {
> + DRM_ERROR("Failed setting up TTM memory accounting "
> + "subsystem.\n");
> + return r;
> + }
> +
> + vgdev->mman.bo_global_ref.mem_glob =
> + vgdev->mman.mem_global_ref.object;
> + global_ref = &vgdev->mman.bo_global_ref.ref;
> + global_ref->global_type = DRM_GLOBAL_TTM_BO;
> + global_ref->size = sizeof(struct ttm_bo_global);
> + global_ref->init = &ttm_bo_global_init;
> + global_ref->release = &ttm_bo_global_release;
> + r = drm_global_item_ref(global_ref);
> + if (r != 0) {
> + DRM_ERROR("Failed setting up TTM BO subsystem.\n");
> + drm_global_item_unref(&vgdev->mman.mem_global_ref);
> + return r;
> + }
> +
> + vgdev->mman.mem_global_referenced = true;
> + return 0;
> +}
> +
> +static void virtio_gpu_ttm_global_fini(struct virtio_gpu_device *vgdev)
> +{
> + if (vgdev->mman.mem_global_referenced) {
> + drm_global_item_unref(&vgdev->mman.bo_global_ref.ref);
> + drm_global_item_unref(&vgdev->mman.mem_global_ref);
> + vgdev->mman.mem_global_referenced = false;
> + }
> +}
> +
> +#if 0
> +/*
> + * Hmm, seems to not do anything useful. Leftover debug hack?
> + * Something like printing pagefaults to kernel log?
> + */
> +static struct vm_operations_struct virtio_gpu_ttm_vm_ops;
> +static const struct vm_operations_struct *ttm_vm_ops;
> +
> +static int virtio_gpu_ttm_fault(struct vm_area_struct *vma,
> + struct vm_fault *vmf)
> +{
> + struct ttm_buffer_object *bo;
> + struct virtio_gpu_device *vgdev;
> + int r;
> +
> + bo = (struct ttm_buffer_object *)vma->vm_private_data;
> + if (bo == NULL)
> + return VM_FAULT_NOPAGE;
> + vgdev = virtio_gpu_get_vgdev(bo->bdev);
> + r = ttm_vm_ops->fault(vma, vmf);
> + return r;
> +}
> +#endif
> +
> +int virtio_gpu_mmap(struct file *filp, struct vm_area_struct *vma)
> +{
> + struct drm_file *file_priv;
> + struct virtio_gpu_device *vgdev;
> + int r;
> +
> + file_priv = filp->private_data;
> + vgdev = file_priv->minor->dev->dev_private;
> + if (vgdev == NULL) {
> + DRM_ERROR(
> + "filp->private_data->minor->dev->dev_private == NULL\n");
> + return -EINVAL;
> + }
> + r = ttm_bo_mmap(filp, vma, &vgdev->mman.bdev);
> +#if 0
> + if (unlikely(r != 0))
> + return r;
> + if (unlikely(ttm_vm_ops == NULL)) {
> + ttm_vm_ops = vma->vm_ops;
> + virtio_gpu_ttm_vm_ops = *ttm_vm_ops;
> + virtio_gpu_ttm_vm_ops.fault = &virtio_gpu_ttm_fault;
> + }
> + vma->vm_ops = &virtio_gpu_ttm_vm_ops;
> + return 0;
> +#else
> + return r;
> +#endif
> +}
> +
> +static int virtio_gpu_invalidate_caches(struct ttm_bo_device *bdev,
> + uint32_t flags)
> +{
> + return 0;
> +}
> +
> +static int ttm_bo_man_get_node(struct ttm_mem_type_manager *man,
> + struct ttm_buffer_object *bo,
> + const struct ttm_place *place,
> + struct ttm_mem_reg *mem)
> +{
> + mem->mm_node = (void *)1;
> + return 0;
> +}
> +
> +static void ttm_bo_man_put_node(struct ttm_mem_type_manager *man,
> + struct ttm_mem_reg *mem)
> +{
> + mem->mm_node = (void *)NULL;
> + return;
> +}
> +
> +static int ttm_bo_man_init(struct ttm_mem_type_manager *man,
> + unsigned long p_size)
> +{
> + return 0;
> +}
> +
> +static int ttm_bo_man_takedown(struct ttm_mem_type_manager *man)
> +{
> + return 0;
> +}
> +
> +static void ttm_bo_man_debug(struct ttm_mem_type_manager *man,
> + const char *prefix)
> +{
> +}
> +
> +static const struct ttm_mem_type_manager_func virtio_gpu_bo_manager_func = {
> + ttm_bo_man_init,
> + ttm_bo_man_takedown,
> + ttm_bo_man_get_node,
> + ttm_bo_man_put_node,
> + ttm_bo_man_debug
> +};
> +
> +static int virtio_gpu_init_mem_type(struct ttm_bo_device *bdev, uint32_t type,
> + struct ttm_mem_type_manager *man)
> +{
> + struct virtio_gpu_device *vgdev;
> +
> + vgdev = virtio_gpu_get_vgdev(bdev);
> +
> + switch (type) {
> + case TTM_PL_SYSTEM:
> + /* System memory */
> + man->flags = TTM_MEMTYPE_FLAG_MAPPABLE;
> + man->available_caching = TTM_PL_MASK_CACHING;
> + man->default_caching = TTM_PL_FLAG_CACHED;
> + break;
> + case TTM_PL_TT:
> + man->func = &virtio_gpu_bo_manager_func;
> + man->flags = TTM_MEMTYPE_FLAG_MAPPABLE;
> + man->available_caching = TTM_PL_MASK_CACHING;
> + man->default_caching = TTM_PL_FLAG_CACHED;
> + break;
> + default:
> + DRM_ERROR("Unsupported memory type %u\n", (unsigned)type);
> + return -EINVAL;
> + }
> + return 0;
> +}
> +
> +static void virtio_gpu_evict_flags(struct ttm_buffer_object *bo,
> + struct ttm_placement *placement)
> +{
> + static struct ttm_place placements = {
> + .fpfn = 0,
> + .lpfn = 0,
> + .flags = TTM_PL_MASK_CACHING | TTM_PL_FLAG_SYSTEM,
> + };
> +
> + placement->placement = &placements;
> + placement->busy_placement = &placements;
> + placement->num_placement = 1;
> + placement->num_busy_placement = 1;
> + return;
> +}
> +
> +static int virtio_gpu_verify_access(struct ttm_buffer_object *bo,
> + struct file *filp)
> +{
> + return 0;
> +}
> +
> +static int virtio_gpu_ttm_io_mem_reserve(struct ttm_bo_device *bdev,
> + struct ttm_mem_reg *mem)
> +{
> + struct ttm_mem_type_manager *man = &bdev->man[mem->mem_type];
> +
> + mem->bus.addr = NULL;
> + mem->bus.offset = 0;
> + mem->bus.size = mem->num_pages << PAGE_SHIFT;
> + mem->bus.base = 0;
> + mem->bus.is_iomem = false;
> + if (!(man->flags & TTM_MEMTYPE_FLAG_MAPPABLE))
> + return -EINVAL;
> + switch (mem->mem_type) {
> + case TTM_PL_SYSTEM:
> + case TTM_PL_TT:
> + /* system memory */
> + return 0;
> + default:
> + return -EINVAL;
> + }
> + return 0;
> +}
> +
> +static void virtio_gpu_ttm_io_mem_free(struct ttm_bo_device *bdev,
> + struct ttm_mem_reg *mem)
> +{
> +}
> +
> +/*
> + * TTM backend functions.
> + */
> +struct virtio_gpu_ttm_tt {
> + struct ttm_dma_tt ttm;
> + struct virtio_gpu_device *vgdev;
> + u64 offset;
> +};
> +
> +static int virtio_gpu_ttm_backend_bind(struct ttm_tt *ttm,
> + struct ttm_mem_reg *bo_mem)
> +{
> + struct virtio_gpu_ttm_tt *gtt = (void *)ttm;
> +
> + gtt->offset = (unsigned long)(bo_mem->start << PAGE_SHIFT);
> + if (!ttm->num_pages)
> + WARN(1, "nothing to bind %lu pages for mreg %p back %p!\n",
> + ttm->num_pages, bo_mem, ttm);
> +
> + /* Not implemented */
> + return 0;
> +}
> +
> +static int virtio_gpu_ttm_backend_unbind(struct ttm_tt *ttm)
> +{
> + /* Not implemented */
> + return 0;
> +}
> +
> +static void virtio_gpu_ttm_backend_destroy(struct ttm_tt *ttm)
> +{
> + struct virtio_gpu_ttm_tt *gtt = (void *)ttm;
> +
> + ttm_dma_tt_fini(>t->ttm);
> + kfree(gtt);
> +}
> +
> +static struct ttm_backend_func virtio_gpu_backend_func = {
> + .bind = &virtio_gpu_ttm_backend_bind,
> + .unbind = &virtio_gpu_ttm_backend_unbind,
> + .destroy = &virtio_gpu_ttm_backend_destroy,
> +};
> +
> +static int virtio_gpu_ttm_tt_populate(struct ttm_tt *ttm)
> +{
> + if (ttm->state != tt_unpopulated)
> + return 0;
> +
> + return ttm_pool_populate(ttm);
> +}
> +
> +static void virtio_gpu_ttm_tt_unpopulate(struct ttm_tt *ttm)
> +{
> + ttm_pool_unpopulate(ttm);
> +}
> +
> +static struct ttm_tt *virtio_gpu_ttm_tt_create(struct ttm_bo_device *bdev,
> + unsigned long size,
> + uint32_t page_flags,
> + struct page *dummy_read_page)
> +{
> + struct virtio_gpu_device *vgdev;
> + struct virtio_gpu_ttm_tt *gtt;
> +
> + vgdev = virtio_gpu_get_vgdev(bdev);
> + gtt = kzalloc(sizeof(struct virtio_gpu_ttm_tt), GFP_KERNEL);
> + if (gtt == NULL)
> + return NULL;
> + gtt->ttm.ttm.func = &virtio_gpu_backend_func;
> + gtt->vgdev = vgdev;
> + if (ttm_dma_tt_init(>t->ttm, bdev, size, page_flags,
> + dummy_read_page)) {
> + kfree(gtt);
> + return NULL;
> + }
> + return >t->ttm.ttm;
> +}
> +
> +static void virtio_gpu_move_null(struct ttm_buffer_object *bo,
> + struct ttm_mem_reg *new_mem)
> +{
> + struct ttm_mem_reg *old_mem = &bo->mem;
> +
> + BUG_ON(old_mem->mm_node != NULL);
> + *old_mem = *new_mem;
> + new_mem->mm_node = NULL;
> +}
> +
> +static int virtio_gpu_bo_move(struct ttm_buffer_object *bo,
> + bool evict, bool interruptible,
> + bool no_wait_gpu,
> + struct ttm_mem_reg *new_mem)
> +{
> + virtio_gpu_move_null(bo, new_mem);
> + return 0;
> +}
> +
> +static void virtio_gpu_bo_move_notify(struct ttm_buffer_object *tbo,
> + struct ttm_mem_reg *new_mem)
> +{
> + struct virtio_gpu_object *bo;
> + struct virtio_gpu_device *vgdev;
> +
> + bo = container_of(tbo, struct virtio_gpu_object, tbo);
> + vgdev = (struct virtio_gpu_device *)bo->gem_base.dev->dev_private;
> +
> + if (!new_mem || (new_mem->placement & TTM_PL_FLAG_SYSTEM)) {
> + if (bo->hw_res_handle)
> + virtio_gpu_cmd_resource_inval_backing(vgdev,
> + bo->hw_res_handle);
> +
> + } else if (new_mem->placement & TTM_PL_FLAG_TT) {
> + if (bo->hw_res_handle) {
> + virtio_gpu_object_attach(vgdev, bo, bo->hw_res_handle,
> + NULL);
> + }
> + }
> +}
> +
> +static void virtio_gpu_bo_swap_notify(struct ttm_buffer_object *tbo)
> +{
> + struct virtio_gpu_object *bo;
> + struct virtio_gpu_device *vgdev;
> +
> + bo = container_of(tbo, struct virtio_gpu_object, tbo);
> + vgdev = (struct virtio_gpu_device *)bo->gem_base.dev->dev_private;
> +
> + if (bo->pages)
> + virtio_gpu_object_free_sg_table(bo);
> +}
> +
> +static struct ttm_bo_driver virtio_gpu_bo_driver = {
> + .ttm_tt_create = &virtio_gpu_ttm_tt_create,
> + .ttm_tt_populate = &virtio_gpu_ttm_tt_populate,
> + .ttm_tt_unpopulate = &virtio_gpu_ttm_tt_unpopulate,
> + .invalidate_caches = &virtio_gpu_invalidate_caches,
> + .init_mem_type = &virtio_gpu_init_mem_type,
> + .evict_flags = &virtio_gpu_evict_flags,
> + .move = &virtio_gpu_bo_move,
> + .verify_access = &virtio_gpu_verify_access,
> + .io_mem_reserve = &virtio_gpu_ttm_io_mem_reserve,
> + .io_mem_free = &virtio_gpu_ttm_io_mem_free,
> + .move_notify = &virtio_gpu_bo_move_notify,
> + .swap_notify = &virtio_gpu_bo_swap_notify,
> +};
> +
> +int virtio_gpu_ttm_init(struct virtio_gpu_device *vgdev)
> +{
> + int r;
> +
> + r = virtio_gpu_ttm_global_init(vgdev);
> + if (r)
> + return r;
> + /* No others user of address space so set it to 0 */
> + r = ttm_bo_device_init(&vgdev->mman.bdev,
> + vgdev->mman.bo_global_ref.ref.object,
> + &virtio_gpu_bo_driver,
> + vgdev->ddev->anon_inode->i_mapping,
> + DRM_FILE_PAGE_OFFSET, 0);
> + if (r) {
> + DRM_ERROR("failed initializing buffer object driver(%d).\n", r);
> + goto err_dev_init;
> + return r;
> + }
> +
> + r = ttm_bo_init_mm(&vgdev->mman.bdev, TTM_PL_TT, 0);
> + if (r) {
> + DRM_ERROR("Failed initializing GTT heap.\n");
> + goto err_mm_init;
> + return r;
> + }
> + return 0;
> +
> +err_mm_init:
> + ttm_bo_device_release(&vgdev->mman.bdev);
> +err_dev_init:
> + virtio_gpu_ttm_global_fini(vgdev);
> + return r;
> +}
> +
> +void virtio_gpu_ttm_fini(struct virtio_gpu_device *vgdev)
> +{
> + ttm_bo_device_release(&vgdev->mman.bdev);
> + virtio_gpu_ttm_global_fini(vgdev);
> + DRM_INFO("virtio_gpu: ttm finalized\n");
> +}
> diff --git a/drivers/gpu/drm/virtio/virtgpu_vq.c b/drivers/gpu/drm/virtio/virtgpu_vq.c
> new file mode 100644
> index 0000000..8fa6513e
> --- /dev/null
> +++ b/drivers/gpu/drm/virtio/virtgpu_vq.c
> @@ -0,0 +1,614 @@
> +/*
> + * Copyright (C) 2015 Red Hat, Inc.
> + * All Rights Reserved.
> + *
> + * Authors:
> + * Dave Airlie <airlied@redhat.com>
> + * Gerd Hoffmann <kraxel@redhat.com>
> + *
> + * Permission is hereby granted, free of charge, to any person obtaining a
> + * copy of this software and associated documentation files (the "Software"),
> + * to deal in the Software without restriction, including without limitation
> + * the rights to use, copy, modify, merge, publish, distribute, sublicense,
> + * and/or sell copies of the Software, and to permit persons to whom the
> + * Software is furnished to do so, subject to the following conditions:
> + *
> + * The above copyright notice and this permission notice (including the next
> + * paragraph) shall be included in all copies or substantial portions of the
> + * Software.
> + *
> + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
> + * VA LINUX SYSTEMS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
> + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
> + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> + * OTHER DEALINGS IN THE SOFTWARE.
> + */
> +
> +#include <drm/drmP.h>
> +#include "virtgpu_drv.h"
> +#include <linux/virtio.h>
> +#include <linux/virtio_config.h>
> +#include <linux/virtio_ring.h>
> +
> +#define MAX_INLINE_CMD_SIZE 96
> +#define MAX_INLINE_RESP_SIZE 24
> +#define VBUFFER_SIZE (sizeof(struct virtio_gpu_vbuffer) \
> + + MAX_INLINE_CMD_SIZE \
> + + MAX_INLINE_RESP_SIZE)
> +
> +void virtio_gpu_resource_id_get(struct virtio_gpu_device *vgdev,
> + uint32_t *resid)
> +{
> + int handle;
> +
> + idr_preload(GFP_KERNEL);
> + spin_lock(&vgdev->resource_idr_lock);
> + handle = idr_alloc(&vgdev->resource_idr, NULL, 1, 0, GFP_NOWAIT);
> + spin_unlock(&vgdev->resource_idr_lock);
> + idr_preload_end();
> + *resid = handle;
> +}
> +
> +void virtio_gpu_resource_id_put(struct virtio_gpu_device *vgdev, uint32_t id)
> +{
> + spin_lock(&vgdev->resource_idr_lock);
> + idr_remove(&vgdev->resource_idr, id);
> + spin_unlock(&vgdev->resource_idr_lock);
> +}
> +
> +void virtio_gpu_ctrl_ack(struct virtqueue *vq)
> +{
> + struct drm_device *dev = vq->vdev->priv;
> + struct virtio_gpu_device *vgdev = dev->dev_private;
> + schedule_work(&vgdev->ctrlq.dequeue_work);
> +}
> +
> +void virtio_gpu_cursor_ack(struct virtqueue *vq)
> +{
> + struct drm_device *dev = vq->vdev->priv;
> + struct virtio_gpu_device *vgdev = dev->dev_private;
> + schedule_work(&vgdev->cursorq.dequeue_work);
> +}
> +
> +int virtio_gpu_alloc_vbufs(struct virtio_gpu_device *vgdev)
> +{
> + struct virtio_gpu_vbuffer *vbuf;
> + int i, size, count = 0;
> + void *ptr;
> +
> + INIT_LIST_HEAD(&vgdev->free_vbufs);
> + count += virtqueue_get_vring_size(vgdev->ctrlq.vq);
> + count += virtqueue_get_vring_size(vgdev->cursorq.vq);
> + size = count * VBUFFER_SIZE;
> + DRM_INFO("virtio vbuffers: %d bufs, %zdB each, %dkB total.\n",
> + count, VBUFFER_SIZE, size / 1024);
> +
> + vgdev->vbufs = kzalloc(size, GFP_KERNEL);
> + if (!vgdev->vbufs)
> + return -ENOMEM;
> +
> + for (i = 0, ptr = vgdev->vbufs;
> + i < count;
> + i++, ptr += VBUFFER_SIZE) {
> + vbuf = ptr;
> + list_add(&vbuf->list, &vgdev->free_vbufs);
> + }
> + return 0;
> +}
> +
> +void virtio_gpu_free_vbufs(struct virtio_gpu_device *vgdev)
> +{
> + struct virtio_gpu_vbuffer *vbuf;
> + int i, count = 0;
> +
> + count += virtqueue_get_vring_size(vgdev->ctrlq.vq);
> + count += virtqueue_get_vring_size(vgdev->cursorq.vq);
> +
> + for (i = 0; i < count; i++) {
> + if (WARN_ON(list_empty(&vgdev->free_vbufs)))
> + return;
> + vbuf = list_first_entry(&vgdev->free_vbufs,
> + struct virtio_gpu_vbuffer, list);
> + list_del(&vbuf->list);
> + }
> + kfree(vgdev->vbufs);
> +}
> +
> +static struct virtio_gpu_vbuffer*
> +virtio_gpu_get_vbuf(struct virtio_gpu_device *vgdev,
> + int size, int resp_size, void *resp_buf,
> + virtio_gpu_resp_cb resp_cb)
> +{
> + struct virtio_gpu_vbuffer *vbuf;
> +
> + BUG_ON(list_empty(&vgdev->free_vbufs));
> + vbuf = list_first_entry(&vgdev->free_vbufs,
> + struct virtio_gpu_vbuffer, list);
> + list_del(&vbuf->list);
> + memset(vbuf, 0, VBUFFER_SIZE);
> +
> + BUG_ON(size > MAX_INLINE_CMD_SIZE);
> + vbuf->buf = (void *)vbuf + sizeof(*vbuf);
> + vbuf->size = size;
> +
> + vbuf->resp_cb = resp_cb;
> + vbuf->resp_size = resp_size;
> + if (resp_size <= MAX_INLINE_RESP_SIZE)
> + vbuf->resp_buf = (void *)vbuf->buf + size;
> + else
> + vbuf->resp_buf = resp_buf;
> + BUG_ON(!vbuf->resp_buf);
> + return vbuf;
> +}
> +
> +static void *virtio_gpu_alloc_cmd(struct virtio_gpu_device *vgdev,
> + struct virtio_gpu_vbuffer **vbuffer_p,
> + int size)
> +{
> + struct virtio_gpu_vbuffer *vbuf;
> +
> + vbuf = virtio_gpu_get_vbuf(vgdev, size,
> + sizeof(struct virtio_gpu_ctrl_hdr),
> + NULL, NULL);
> + if (IS_ERR(vbuf)) {
> + *vbuffer_p = NULL;
> + return ERR_CAST(vbuf);
> + }
> + *vbuffer_p = vbuf;
> + return vbuf->buf;
> +}
> +
> +static struct virtio_gpu_update_cursor*
> +virtio_gpu_alloc_cursor(struct virtio_gpu_device *vgdev,
> + struct virtio_gpu_vbuffer **vbuffer_p)
> +{
> + struct virtio_gpu_vbuffer *vbuf;
> +
> + vbuf = virtio_gpu_get_vbuf
> + (vgdev, sizeof(struct virtio_gpu_update_cursor),
> + 0, NULL, NULL);
> + if (IS_ERR(vbuf)) {
> + *vbuffer_p = NULL;
> + return ERR_CAST(vbuf);
> + }
> + *vbuffer_p = vbuf;
> + return (struct virtio_gpu_update_cursor *)vbuf->buf;
> +}
> +
> +static void *virtio_gpu_alloc_cmd_resp(struct virtio_gpu_device *vgdev,
> + virtio_gpu_resp_cb cb,
> + struct virtio_gpu_vbuffer **vbuffer_p,
> + int cmd_size, int resp_size,
> + void *resp_buf)
> +{
> + struct virtio_gpu_vbuffer *vbuf;
> +
> + vbuf = virtio_gpu_get_vbuf(vgdev, cmd_size,
> + resp_size, resp_buf, cb);
> + if (IS_ERR(vbuf)) {
> + *vbuffer_p = NULL;
> + return ERR_CAST(vbuf);
> + }
> + *vbuffer_p = vbuf;
> + return (struct virtio_gpu_command *)vbuf->buf;
> +}
> +
> +static void free_vbuf(struct virtio_gpu_device *vgdev,
> + struct virtio_gpu_vbuffer *vbuf)
> +{
> + if (vbuf->resp_size > MAX_INLINE_RESP_SIZE)
> + kfree(vbuf->resp_buf);
> + kfree(vbuf->data_buf);
> + list_add(&vbuf->list, &vgdev->free_vbufs);
> +}
> +
> +static void reclaim_vbufs(struct virtqueue *vq, struct list_head *reclaim_list)
> +{
> + struct virtio_gpu_vbuffer *vbuf;
> + unsigned int len;
> + int freed = 0;
> +
> + while ((vbuf = virtqueue_get_buf(vq, &len))) {
> + list_add_tail(&vbuf->list, reclaim_list);
> + freed++;
> + }
> + if (freed == 0)
> + DRM_DEBUG("Huh? zero vbufs reclaimed");
> +}
> +
> +void virtio_gpu_dequeue_ctrl_func(struct work_struct *work)
> +{
> + struct virtio_gpu_device *vgdev =
> + container_of(work, struct virtio_gpu_device,
> + ctrlq.dequeue_work);
> + struct list_head reclaim_list;
> + struct virtio_gpu_vbuffer *entry, *tmp;
> + struct virtio_gpu_ctrl_hdr *resp;
> + u64 fence_id = 0;
> +
> + INIT_LIST_HEAD(&reclaim_list);
> + spin_lock(&vgdev->ctrlq.qlock);
> + do {
> + virtqueue_disable_cb(vgdev->ctrlq.vq);
> + reclaim_vbufs(vgdev->ctrlq.vq, &reclaim_list);
> +
> + } while (!virtqueue_enable_cb(vgdev->ctrlq.vq));
> + spin_unlock(&vgdev->ctrlq.qlock);
> +
> + list_for_each_entry_safe(entry, tmp, &reclaim_list, list) {
> + resp = (struct virtio_gpu_ctrl_hdr *)entry->resp_buf;
> + if (resp->type != cpu_to_le32(VIRTIO_GPU_RESP_OK_NODATA))
> + DRM_DEBUG("response 0x%x\n", le32_to_cpu(resp->type));
> + if (resp->flags & cpu_to_le32(VIRTIO_GPU_FLAG_FENCE)) {
> + u64 f = le64_to_cpu(resp->fence_id);
> +
> + if (fence_id > f) {
> + DRM_ERROR("%s: Oops: fence %llx -> %llx\n",
> + __func__, fence_id, f);
> + } else {
> + fence_id = f;
> + }
> + }
> + if (entry->resp_cb)
> + entry->resp_cb(vgdev, entry);
> +
> + list_del(&entry->list);
> + free_vbuf(vgdev, entry);
> + }
> + wake_up(&vgdev->ctrlq.ack_queue);
> +
> + if (fence_id)
> + virtio_gpu_fence_event_process(vgdev, fence_id);
> +}
> +
> +void virtio_gpu_dequeue_cursor_func(struct work_struct *work)
> +{
> + struct virtio_gpu_device *vgdev =
> + container_of(work, struct virtio_gpu_device,
> + cursorq.dequeue_work);
> + struct list_head reclaim_list;
> + struct virtio_gpu_vbuffer *entry, *tmp;
> +
> + INIT_LIST_HEAD(&reclaim_list);
> + spin_lock(&vgdev->cursorq.qlock);
> + do {
> + virtqueue_disable_cb(vgdev->cursorq.vq);
> + reclaim_vbufs(vgdev->cursorq.vq, &reclaim_list);
> + } while (!virtqueue_enable_cb(vgdev->cursorq.vq));
> + spin_unlock(&vgdev->cursorq.qlock);
> +
> + list_for_each_entry_safe(entry, tmp, &reclaim_list, list) {
> + list_del(&entry->list);
> + free_vbuf(vgdev, entry);
> + }
> + wake_up(&vgdev->cursorq.ack_queue);
> +}
> +
> +static int virtio_gpu_queue_ctrl_buffer(struct virtio_gpu_device *vgdev,
> + struct virtio_gpu_vbuffer *vbuf)
> +{
> + struct virtqueue *vq = vgdev->ctrlq.vq;
> + struct scatterlist *sgs[3], vcmd, vout, vresp;
> + int outcnt = 0, incnt = 0;
> + int ret;
> +
> + if (!vgdev->vqs_ready)
> + return -ENODEV;
> +
> + sg_init_one(&vcmd, vbuf->buf, vbuf->size);
> + sgs[outcnt+incnt] = &vcmd;
> + outcnt++;
> +
> + if (vbuf->data_size) {
> + sg_init_one(&vout, vbuf->data_buf, vbuf->data_size);
> + sgs[outcnt + incnt] = &vout;
> + outcnt++;
> + }
> +
> + if (vbuf->resp_size) {
> + sg_init_one(&vresp, vbuf->resp_buf, vbuf->resp_size);
> + sgs[outcnt + incnt] = &vresp;
> + incnt++;
> + }
> +
> + spin_lock(&vgdev->ctrlq.qlock);
> +retry:
> + ret = virtqueue_add_sgs(vq, sgs, outcnt, incnt, vbuf, GFP_ATOMIC);
> + if (ret == -ENOSPC) {
> + spin_unlock(&vgdev->ctrlq.qlock);
> + wait_event(vgdev->ctrlq.ack_queue, vq->num_free);
> + spin_lock(&vgdev->ctrlq.qlock);
> + goto retry;
> + } else {
> + virtqueue_kick(vq);
> + }
> + spin_unlock(&vgdev->ctrlq.qlock);
> +
> + if (!ret)
> + ret = vq->num_free;
> + return ret;
> +}
> +
> +static int virtio_gpu_queue_cursor(struct virtio_gpu_device *vgdev,
> + struct virtio_gpu_vbuffer *vbuf)
> +{
> + struct virtqueue *vq = vgdev->cursorq.vq;
> + struct scatterlist *sgs[1], ccmd;
> + int ret;
> + int outcnt;
> +
> + if (!vgdev->vqs_ready)
> + return -ENODEV;
> +
> + sg_init_one(&ccmd, vbuf->buf, vbuf->size);
> + sgs[0] = &ccmd;
> + outcnt = 1;
> +
> + spin_lock(&vgdev->cursorq.qlock);
> +retry:
> + ret = virtqueue_add_sgs(vq, sgs, outcnt, 0, vbuf, GFP_ATOMIC);
> + if (ret == -ENOSPC) {
> + spin_unlock(&vgdev->cursorq.qlock);
> + wait_event(vgdev->cursorq.ack_queue, vq->num_free);
> + spin_lock(&vgdev->cursorq.qlock);
> + goto retry;
> + } else {
> + virtqueue_kick(vq);
> + }
> +
> + spin_unlock(&vgdev->cursorq.qlock);
> +
> + if (!ret)
> + ret = vq->num_free;
> + return ret;
> +}
> +
> +/* just create gem objects for userspace and long lived objects,
> + just use dma_alloced pages for the queue objects? */
> +
> +/* create a basic resource */
> +void virtio_gpu_cmd_create_resource(struct virtio_gpu_device *vgdev,
> + uint32_t resource_id,
> + uint32_t format,
> + uint32_t width,
> + uint32_t height)
> +{
> + struct virtio_gpu_resource_create_2d *cmd_p;
> + struct virtio_gpu_vbuffer *vbuf;
> +
> + cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
> + memset(cmd_p, 0, sizeof(*cmd_p));
> +
> + cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_RESOURCE_CREATE_2D);
> + cmd_p->resource_id = cpu_to_le32(resource_id);
> + cmd_p->format = cpu_to_le32(format);
> + cmd_p->width = cpu_to_le32(width);
> + cmd_p->height = cpu_to_le32(height);
> +
> + virtio_gpu_queue_ctrl_buffer(vgdev, vbuf);
> +}
> +
> +void virtio_gpu_cmd_unref_resource(struct virtio_gpu_device *vgdev,
> + uint32_t resource_id)
> +{
> + struct virtio_gpu_resource_unref *cmd_p;
> + struct virtio_gpu_vbuffer *vbuf;
> +
> + cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
> + memset(cmd_p, 0, sizeof(*cmd_p));
> +
> + cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_RESOURCE_UNREF);
> + cmd_p->resource_id = cpu_to_le32(resource_id);
> +
> + virtio_gpu_queue_ctrl_buffer(vgdev, vbuf);
> +}
> +
> +void virtio_gpu_cmd_resource_inval_backing(struct virtio_gpu_device *vgdev,
> + uint32_t resource_id)
> +{
> + struct virtio_gpu_resource_detach_backing *cmd_p;
> + struct virtio_gpu_vbuffer *vbuf;
> +
> + cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
> + memset(cmd_p, 0, sizeof(*cmd_p));
> +
> + cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_RESOURCE_DETACH_BACKING);
> + cmd_p->resource_id = cpu_to_le32(resource_id);
> +
> + virtio_gpu_queue_ctrl_buffer(vgdev, vbuf);
> +}
> +
> +void virtio_gpu_cmd_set_scanout(struct virtio_gpu_device *vgdev,
> + uint32_t scanout_id, uint32_t resource_id,
> + uint32_t width, uint32_t height,
> + uint32_t x, uint32_t y)
> +{
> + struct virtio_gpu_set_scanout *cmd_p;
> + struct virtio_gpu_vbuffer *vbuf;
> +
> + cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
> + memset(cmd_p, 0, sizeof(*cmd_p));
> +
> + cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_SET_SCANOUT);
> + cmd_p->resource_id = cpu_to_le32(resource_id);
> + cmd_p->scanout_id = cpu_to_le32(scanout_id);
> + cmd_p->r.width = cpu_to_le32(width);
> + cmd_p->r.height = cpu_to_le32(height);
> + cmd_p->r.x = cpu_to_le32(x);
> + cmd_p->r.y = cpu_to_le32(y);
> +
> + virtio_gpu_queue_ctrl_buffer(vgdev, vbuf);
> +}
> +
> +void virtio_gpu_cmd_resource_flush(struct virtio_gpu_device *vgdev,
> + uint32_t resource_id,
> + uint32_t x, uint32_t y,
> + uint32_t width, uint32_t height)
> +{
> + struct virtio_gpu_resource_flush *cmd_p;
> + struct virtio_gpu_vbuffer *vbuf;
> +
> + cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
> + memset(cmd_p, 0, sizeof(*cmd_p));
> +
> + cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_RESOURCE_FLUSH);
> + cmd_p->resource_id = cpu_to_le32(resource_id);
> + cmd_p->r.width = cpu_to_le32(width);
> + cmd_p->r.height = cpu_to_le32(height);
> + cmd_p->r.x = cpu_to_le32(x);
> + cmd_p->r.y = cpu_to_le32(y);
> +
> + virtio_gpu_queue_ctrl_buffer(vgdev, vbuf);
> +}
> +
> +void virtio_gpu_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev,
> + uint32_t resource_id, uint64_t offset,
> + __le32 width, __le32 height,
> + __le32 x, __le32 y,
> + struct virtio_gpu_fence **fence)
> +{
> + struct virtio_gpu_transfer_to_host_2d *cmd_p;
> + struct virtio_gpu_vbuffer *vbuf;
> +
> + cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
> + memset(cmd_p, 0, sizeof(*cmd_p));
> +
> + cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_TRANSFER_TO_HOST_2D);
> + cmd_p->resource_id = cpu_to_le32(resource_id);
> + cmd_p->offset = cpu_to_le64(offset);
> + cmd_p->r.width = width;
> + cmd_p->r.height = height;
> + cmd_p->r.x = x;
> + cmd_p->r.y = y;
> +
> + if (fence)
> + virtio_gpu_fence_emit(vgdev, &cmd_p->hdr, fence);
> + virtio_gpu_queue_ctrl_buffer(vgdev, vbuf);
> +}
> +
> +static void
> +virtio_gpu_cmd_resource_attach_backing(struct virtio_gpu_device *vgdev,
> + uint32_t resource_id,
> + struct virtio_gpu_mem_entry *ents,
> + uint32_t nents,
> + struct virtio_gpu_fence **fence)
> +{
> + struct virtio_gpu_resource_attach_backing *cmd_p;
> + struct virtio_gpu_vbuffer *vbuf;
> +
> + cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p));
> + memset(cmd_p, 0, sizeof(*cmd_p));
> +
> + cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_RESOURCE_ATTACH_BACKING);
> + cmd_p->resource_id = cpu_to_le32(resource_id);
> + cmd_p->nr_entries = cpu_to_le32(nents);
> +
> + vbuf->data_buf = ents;
> + vbuf->data_size = sizeof(*ents) * nents;
> +
> + if (fence)
> + virtio_gpu_fence_emit(vgdev, &cmd_p->hdr, fence);
> + virtio_gpu_queue_ctrl_buffer(vgdev, vbuf);
> +}
> +
> +static void virtio_gpu_cmd_get_display_info_cb(struct virtio_gpu_device *vgdev,
> + struct virtio_gpu_vbuffer *vbuf)
> +{
> + struct virtio_gpu_resp_display_info *resp =
> + (struct virtio_gpu_resp_display_info *)vbuf->resp_buf;
> + int i;
> +
> + spin_lock(&vgdev->display_info_lock);
> + for (i = 0; i < vgdev->num_scanouts; i++) {
> + vgdev->outputs[i].info = resp->pmodes[i];
> + if (resp->pmodes[i].enabled) {
> + DRM_DEBUG("output %d: %dx%d+%d+%d", i,
> + le32_to_cpu(resp->pmodes[i].r.width),
> + le32_to_cpu(resp->pmodes[i].r.height),
> + le32_to_cpu(resp->pmodes[i].r.x),
> + le32_to_cpu(resp->pmodes[i].r.y));
> + } else {
> + DRM_DEBUG("output %d: disabled", i);
> + }
> + }
> +
> + spin_unlock(&vgdev->display_info_lock);
> + wake_up(&vgdev->resp_wq);
> +
> + if (!drm_helper_hpd_irq_event(vgdev->ddev))
> + drm_kms_helper_hotplug_event(vgdev->ddev);
> +}
> +
> +int virtio_gpu_cmd_get_display_info(struct virtio_gpu_device *vgdev)
> +{
> + struct virtio_gpu_ctrl_hdr *cmd_p;
> + struct virtio_gpu_vbuffer *vbuf;
> + void *resp_buf;
> +
> + resp_buf = kzalloc(sizeof(struct virtio_gpu_resp_display_info),
> + GFP_KERNEL);
> + if (!resp_buf)
> + return -ENOMEM;
> +
> + cmd_p = virtio_gpu_alloc_cmd_resp
> + (vgdev, &virtio_gpu_cmd_get_display_info_cb, &vbuf,
> + sizeof(*cmd_p), sizeof(struct virtio_gpu_resp_display_info),
> + resp_buf);
> + memset(cmd_p, 0, sizeof(*cmd_p));
> +
> + cmd_p->type = cpu_to_le32(VIRTIO_GPU_CMD_GET_DISPLAY_INFO);
> + virtio_gpu_queue_ctrl_buffer(vgdev, vbuf);
> + return 0;
> +}
> +
> +int virtio_gpu_object_attach(struct virtio_gpu_device *vgdev,
> + struct virtio_gpu_object *obj,
> + uint32_t resource_id,
> + struct virtio_gpu_fence **fence)
> +{
> + struct virtio_gpu_mem_entry *ents;
> + struct scatterlist *sg;
> + int si;
> +
> + if (!obj->pages) {
> + int ret;
> + ret = virtio_gpu_object_get_sg_table(vgdev, obj);
> + if (ret)
> + return ret;
> + }
> +
> + /* gets freed when the ring has consumed it */
> + ents = kmalloc_array(obj->pages->nents,
> + sizeof(struct virtio_gpu_mem_entry),
> + GFP_KERNEL);
> + if (!ents) {
> + DRM_ERROR("failed to allocate ent list\n");
> + return -ENOMEM;
> + }
> +
> + for_each_sg(obj->pages->sgl, sg, obj->pages->nents, si) {
> + ents[si].addr = cpu_to_le64(sg_phys(sg));
> + ents[si].length = cpu_to_le32(sg->length);
> + ents[si].padding = 0;
> + }
> +
> + virtio_gpu_cmd_resource_attach_backing(vgdev, resource_id,
> + ents, obj->pages->nents,
> + fence);
> + obj->hw_res_handle = resource_id;
> + return 0;
> +}
> +
> +void virtio_gpu_cursor_ping(struct virtio_gpu_device *vgdev,
> + struct virtio_gpu_output *output)
> +{
> + struct virtio_gpu_vbuffer *vbuf;
> + struct virtio_gpu_update_cursor *cur_p;
> +
> + output->cursor.pos.scanout_id = cpu_to_le32(output->index);
> + cur_p = virtio_gpu_alloc_cursor(vgdev, &vbuf);
> + memcpy(cur_p, &output->cursor, sizeof(output->cursor));
> + virtio_gpu_queue_cursor(vgdev, vbuf);
> +}
> diff --git a/include/drm/drmP.h b/include/drm/drmP.h
> index 62c40777..1544bb1 100644
> --- a/include/drm/drmP.h
> +++ b/include/drm/drmP.h
> @@ -812,6 +812,7 @@ struct drm_device {
> #endif
>
> struct platform_device *platformdev; /**< Platform device struture */
> + struct virtio_device *virtdev;
>
> struct drm_sg_mem *sg; /**< Scatter gather memory */
> unsigned int num_crtcs; /**< Number of CRTCs on this device */
> diff --git a/include/uapi/linux/Kbuild b/include/uapi/linux/Kbuild
> index 1a0006a..4460e58 100644
> --- a/include/uapi/linux/Kbuild
> +++ b/include/uapi/linux/Kbuild
> @@ -430,6 +430,7 @@ header-y += virtio_balloon.h
> header-y += virtio_blk.h
> header-y += virtio_config.h
> header-y += virtio_console.h
> +header-y += virtio_gpu.h
> header-y += virtio_ids.h
> header-y += virtio_input.h
> header-y += virtio_net.h
> diff --git a/include/uapi/linux/virtio_gpu.h b/include/uapi/linux/virtio_gpu.h
> new file mode 100644
> index 0000000..571c4cb
> --- /dev/null
> +++ b/include/uapi/linux/virtio_gpu.h
> @@ -0,0 +1,204 @@
> +/*
> + * Virtio GPU Device
> + *
> + * Copyright Red Hat, Inc. 2013-2014
> + *
> + * Authors:
> + * Dave Airlie <airlied@redhat.com>
> + * Gerd Hoffmann <kraxel@redhat.com>
> + *
> + * This header is BSD licensed so anyone can use the definitions
> + * to implement compatible drivers/servers:
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions
> + * are met:
> + * 1. Redistributions of source code must retain the above copyright
> + * notice, this list of conditions and the following disclaimer.
> + * 2. Redistributions in binary form must reproduce the above copyright
> + * notice, this list of conditions and the following disclaimer in the
> + * documentation and/or other materials provided with the distribution.
> + * 3. Neither the name of IBM nor the names of its contributors
> + * may be used to endorse or promote products derived from this software
> + * without specific prior written permission.
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + * ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> + * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL IBM OR
> + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
> + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
> + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
> + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
> + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
> + * SUCH DAMAGE.
> + */
> +
> +#ifndef VIRTIO_GPU_HW_H
> +#define VIRTIO_GPU_HW_H
> +
> +enum virtio_gpu_ctrl_type {
> + VIRTIO_GPU_UNDEFINED = 0,
> +
> + /* 2d commands */
> + VIRTIO_GPU_CMD_GET_DISPLAY_INFO = 0x0100,
> + VIRTIO_GPU_CMD_RESOURCE_CREATE_2D,
> + VIRTIO_GPU_CMD_RESOURCE_UNREF,
> + VIRTIO_GPU_CMD_SET_SCANOUT,
> + VIRTIO_GPU_CMD_RESOURCE_FLUSH,
> + VIRTIO_GPU_CMD_TRANSFER_TO_HOST_2D,
> + VIRTIO_GPU_CMD_RESOURCE_ATTACH_BACKING,
> + VIRTIO_GPU_CMD_RESOURCE_DETACH_BACKING,
> +
> + /* cursor commands */
> + VIRTIO_GPU_CMD_UPDATE_CURSOR = 0x0300,
> + VIRTIO_GPU_CMD_MOVE_CURSOR,
> +
> + /* success responses */
> + VIRTIO_GPU_RESP_OK_NODATA = 0x1100,
> + VIRTIO_GPU_RESP_OK_DISPLAY_INFO,
> +
> + /* error responses */
> + VIRTIO_GPU_RESP_ERR_UNSPEC = 0x1200,
> + VIRTIO_GPU_RESP_ERR_OUT_OF_MEMORY,
> + VIRTIO_GPU_RESP_ERR_INVALID_SCANOUT_ID,
> + VIRTIO_GPU_RESP_ERR_INVALID_RESOURCE_ID,
> + VIRTIO_GPU_RESP_ERR_INVALID_CONTEXT_ID,
> + VIRTIO_GPU_RESP_ERR_INVALID_PARAMETER,
> +};
> +
> +#define VIRTIO_GPU_FLAG_FENCE (1 << 0)
> +
> +struct virtio_gpu_ctrl_hdr {
> + __le32 type;
> + __le32 flags;
> + __le64 fence_id;
> + __le32 ctx_id;
> + __le32 padding;
> +};
> +
> +/* data passed in the cursor vq */
> +
> +struct virtio_gpu_cursor_pos {
> + __le32 scanout_id;
> + __le32 x;
> + __le32 y;
> + __le32 padding;
> +};
> +
> +/* VIRTIO_GPU_CMD_UPDATE_CURSOR, VIRTIO_GPU_CMD_MOVE_CURSOR */
> +struct virtio_gpu_update_cursor {
> + struct virtio_gpu_ctrl_hdr hdr;
> + struct virtio_gpu_cursor_pos pos; /* update & move */
> + __le32 resource_id; /* update only */
> + __le32 hot_x; /* update only */
> + __le32 hot_y; /* update only */
> + __le32 padding;
> +};
> +
> +/* data passed in the control vq, 2d related */
> +
> +struct virtio_gpu_rect {
> + __le32 x;
> + __le32 y;
> + __le32 width;
> + __le32 height;
> +};
> +
> +/* VIRTIO_GPU_CMD_RESOURCE_UNREF */
> +struct virtio_gpu_resource_unref {
> + struct virtio_gpu_ctrl_hdr hdr;
> + __le32 resource_id;
> + __le32 padding;
> +};
> +
> +/* VIRTIO_GPU_CMD_RESOURCE_CREATE_2D: create a 2d resource with a format */
> +struct virtio_gpu_resource_create_2d {
> + struct virtio_gpu_ctrl_hdr hdr;
> + __le32 resource_id;
> + __le32 format;
> + __le32 width;
> + __le32 height;
> +};
> +
> +/* VIRTIO_GPU_CMD_SET_SCANOUT */
> +struct virtio_gpu_set_scanout {
> + struct virtio_gpu_ctrl_hdr hdr;
> + struct virtio_gpu_rect r;
> + __le32 scanout_id;
> + __le32 resource_id;
> +};
> +
> +/* VIRTIO_GPU_CMD_RESOURCE_FLUSH */
> +struct virtio_gpu_resource_flush {
> + struct virtio_gpu_ctrl_hdr hdr;
> + struct virtio_gpu_rect r;
> + __le32 resource_id;
> + __le32 padding;
> +};
> +
> +/* VIRTIO_GPU_CMD_TRANSFER_TO_HOST_2D: simple transfer to_host */
> +struct virtio_gpu_transfer_to_host_2d {
> + struct virtio_gpu_ctrl_hdr hdr;
> + struct virtio_gpu_rect r;
> + __le64 offset;
> + __le32 resource_id;
> + __le32 padding;
> +};
> +
> +struct virtio_gpu_mem_entry {
> + __le64 addr;
> + __le32 length;
> + __le32 padding;
> +};
> +
> +/* VIRTIO_GPU_CMD_RESOURCE_ATTACH_BACKING */
> +struct virtio_gpu_resource_attach_backing {
> + struct virtio_gpu_ctrl_hdr hdr;
> + __le32 resource_id;
> + __le32 nr_entries;
> +};
> +
> +/* VIRTIO_GPU_CMD_RESOURCE_DETACH_BACKING */
> +struct virtio_gpu_resource_detach_backing {
> + struct virtio_gpu_ctrl_hdr hdr;
> + __le32 resource_id;
> + __le32 padding;
> +};
> +
> +/* VIRTIO_GPU_RESP_OK_DISPLAY_INFO */
> +#define VIRTIO_GPU_MAX_SCANOUTS 16
> +struct virtio_gpu_resp_display_info {
> + struct virtio_gpu_ctrl_hdr hdr;
> + struct virtio_gpu_display_one {
> + struct virtio_gpu_rect r;
> + __le32 enabled;
> + __le32 flags;
> + } pmodes[VIRTIO_GPU_MAX_SCANOUTS];
> +};
> +
> +#define VIRTIO_GPU_EVENT_DISPLAY (1 << 0)
> +
> +struct virtio_gpu_config {
> + __u32 events_read;
> + __u32 events_clear;
> + __u32 num_scanouts;
> + __u32 reserved;
> +};
> +
> +/* simple formats for fbcon/X use */
> +enum virtio_gpu_formats {
> + VIRTIO_GPU_FORMAT_B8G8R8A8_UNORM = 1,
> + VIRTIO_GPU_FORMAT_B8G8R8X8_UNORM = 2,
> + VIRTIO_GPU_FORMAT_A8R8G8B8_UNORM = 3,
> + VIRTIO_GPU_FORMAT_X8R8G8B8_UNORM = 4,
> +
> + VIRTIO_GPU_FORMAT_R8G8B8A8_UNORM = 67,
> + VIRTIO_GPU_FORMAT_X8B8G8R8_UNORM = 68,
> +
> + VIRTIO_GPU_FORMAT_A8B8G8R8_UNORM = 121,
> + VIRTIO_GPU_FORMAT_R8G8B8X8_UNORM = 134,
> +};
> +
> +#endif
> diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h
> index 5f60aa4..77925f5 100644
> --- a/include/uapi/linux/virtio_ids.h
> +++ b/include/uapi/linux/virtio_ids.h
> @@ -39,6 +39,7 @@
> #define VIRTIO_ID_9P 9 /* 9p virtio console */
> #define VIRTIO_ID_RPROC_SERIAL 11 /* virtio remoteproc serial link */
> #define VIRTIO_ID_CAIF 12 /* Virtio caif */
> +#define VIRTIO_ID_GPU 16 /* virtio GPU */
> #define VIRTIO_ID_INPUT 18 /* virtio input */
>
> #endif /* _LINUX_VIRTIO_IDS_H */
> --
> 1.8.3.1
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/dri-devel
^ permalink raw reply
* [PATCH v5 21/21] libnvdimm: Non-Volatile Devices
From: Dan Williams @ 2015-06-02 0:16 UTC (permalink / raw)
To: linux-nvdimm
Cc: axboe, sfr, rafael, neilb, gregkh, linux-kernel, mingo,
linux-acpi, jmoyer, linux-api, akpm, hch
In-Reply-To: <20150602001134.4506.45867.stgit@dwillia2-desk3.amr.corp.intel.com>
Maintainer information and documentation for drivers/nvdimm
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Boaz Harrosh <boaz@plexistor.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Jens Axboe <axboe@fb.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Neil Brown <neilb@suse.de>
Cc: Greg KH <gregkh@linuxfoundation.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
Documentation/nvdimm/nvdimm.txt | 805 +++++++++++++++++++++++++++++++++++++++
MAINTAINERS | 39 ++
2 files changed, 838 insertions(+), 6 deletions(-)
create mode 100644 Documentation/nvdimm/nvdimm.txt
diff --git a/Documentation/nvdimm/nvdimm.txt b/Documentation/nvdimm/nvdimm.txt
new file mode 100644
index 000000000000..854f0a7c96c5
--- /dev/null
+++ b/Documentation/nvdimm/nvdimm.txt
@@ -0,0 +1,805 @@
+ LIBNVDIMM: Non-Volatile Devices
+ libnvdimm - kernel / libndctl - userspace helper library
+ linux-nvdimm@lists.01.org
+ v12
+
+
+ Glossary
+ Overview
+ Supporting Documents
+ Git Trees
+ LIBNVDIMM PMEM and BLK
+ Why BLK?
+ PMEM vs BLK
+ BLK-REGIONs, PMEM-REGIONs, Atomic Sectors, and DAX
+ Example NVDIMM Platform
+ LIBNVDIMM Kernel Device Model and LIBNDCTL Userspace API
+ LIBNDCTL: Context
+ libndctl: instantiate a new library context example
+ LIBNVDIMM/LIBNDCTL: Bus
+ libnvdimm: control class device in /sys/class
+ libnvdimm: bus
+ libndctl: bus enumeration example
+ LIBNVDIMM/LIBNDCTL: DIMM (NMEM)
+ libnvdimm: DIMM (NMEM)
+ libndctl: DIMM enumeration example
+ LIBNVDIMM/LIBNDCTL: Region
+ libnvdimm: region
+ libndctl: region enumeration example
+ Why Not Encode the Region Type into the Region Name?
+ How Do I Determine the Major Type of a Region?
+ LIBNVDIMM/LIBNDCTL: Namespace
+ libnvdimm: namespace
+ libndctl: namespace enumeration example
+ libndctl: namespace creation example
+ Why the Term "namespace"?
+ LIBNVDIMM/LIBNDCTL: Block Translation Table "btt"
+ libnvdimm: btt layout
+ libndctl: btt creation example
+ Summary LIBNDCTL Diagram
+
+
+Glossary
+--------
+
+PMEM: A system-physical-address range where writes are persistent. A
+block device composed of PMEM is capable of DAX. A PMEM address range
+may span an interleave of several DIMMs.
+
+BLK: A set of one or more programmable memory mapped apertures provided
+by a DIMM to access its media. This indirection precludes the
+performance benefit of interleaving, but enables DIMM-bounded failure
+modes.
+
+DPA: DIMM Physical Address, is a DIMM-relative offset. With one DIMM in
+the system there would be a 1:1 system-physical-address:DPA association.
+Once more DIMMs are added a memory controller interleave must be
+decoded to determine the DPA associated with a given
+system-physical-address. BLK capacity always has a 1:1 relationship
+with a single-DIMM's DPA range.
+
+DAX: File system extensions to bypass the page cache and block layer to
+mmap persistent memory, from a PMEM block device, directly into a
+process address space.
+
+BTT: Block Translation Table: Persistent memory is byte addressable.
+Existing software may have an expectation that the power-fail-atomicity
+of writes is at least one sector, 512 bytes. The BTT is an indirection
+table with atomic update semantics to front a PMEM/BLK block device
+driver and present arbitrary atomic sector sizes.
+
+LABEL: Metadata stored on a DIMM device that partitions and identifies
+(persistently names) storage between PMEM and BLK. It also partitions
+BLK storage to host BTTs with different parameters per BLK-partition.
+Note that traditional partition tables, GPT/MBR, are layered on top of a
+BLK or PMEM device.
+
+
+Overview
+--------
+
+The LIBNVDIMM subsystem provides support for three types of NVDIMMs, namely,
+PMEM, BLK, and NVDIMM devices that can simultaneously support both PMEM
+and BLK mode access. These three modes of operation are described by
+the "NVDIMM Firmware Interface Table" (NFIT) in ACPI 6. While the LIBNVDIMM
+implementation is generic and supports pre-NFIT platforms, it was guided
+by the superset of capabilities need to support this ACPI 6 definition
+for NVDIMM resources. The bulk of the kernel implementation is in place
+to handle the case where DPA accessible via PMEM is aliased with DPA
+accessible via BLK. When that occurs a LABEL is needed to reserve DPA
+for exclusive access via one mode a time.
+
+Supporting Documents
+ACPI 6: http://www.uefi.org/sites/default/files/resources/ACPI_6.0.pdf
+NVDIMM Namespace: http://pmem.io/documents/NVDIMM_Namespace_Spec.pdf
+DSM Interface Example: http://pmem.io/documents/NVDIMM_DSM_Interface_Example.pdf
+Driver Writer's Guide: http://pmem.io/documents/NVDIMM_Driver_Writers_Guide.pdf
+
+Git Trees
+LIBNVDIMM: https://git.kernel.org/cgit/linux/kernel/git/djbw/nvdimm.git
+LIBNDCTL: https://github.com/pmem/ndctl.git
+PMEM: https://github.com/01org/prd
+
+
+LIBNVDIMM PMEM and BLK
+------------------
+
+Prior to the arrival of the NFIT, non-volatile memory was described to a
+system in various ad-hoc ways. Usually only the bare minimum was
+provided, namely, a single system-physical-address range where writes
+are expected to be durable after a system power loss. Now, the NFIT
+specification standardizes not only the description of PMEM, but also
+BLK and platform message-passing entry points for control and
+configuration.
+
+For each NVDIMM access method (PMEM, BLK), LIBNVDIMM provides a block
+device driver:
+
+ 1. PMEM (nd_pmem.ko): Drives a system-physical-address range. This
+ range is contiguous in system memory and may be interleaved (hardware
+ memory controller striped) across multiple DIMMs. When interleaved the
+ platform may optionally provide details of which DIMMs are participating
+ in the interleave.
+
+ Note that while LIBNVDIMM describes system-physical-address ranges that may
+ alias with BLK access as ND_NAMESPACE_PMEM ranges and those without
+ alias as ND_NAMESPACE_IO ranges, to the nd_pmem driver there is no
+ distinction. The different device-types are an implementation detail
+ that userspace can exploit to implement policies like "only interface
+ with address ranges from certain DIMMs". It is worth noting that when
+ aliasing is present and a DIMM lacks a label, then no block device can
+ be created by default as userspace needs to do at least one allocation
+ of DPA to the PMEM range. In contrast ND_NAMESPACE_IO ranges, once
+ registered, can be immediately attached to nd_pmem.
+
+ 2. BLK (nd_blk.ko): This driver performs I/O using a set of platform
+ defined apertures. A set of apertures will all access just one DIMM.
+ Multiple windows allow multiple concurrent accesses, much like
+ tagged-command-queuing, and would likely be used by different threads or
+ different CPUs.
+
+ The NFIT specification defines a standard format for a BLK-aperture, but
+ the spec also allows for vendor specific layouts, and non-NFIT BLK
+ implementations may other designs for BLK I/O. For this reason "nd_blk"
+ calls back into platform-specific code to perform the I/O. One such
+ implementation is defined in the "Driver Writer's Guide" and "DSM
+ Interface Example".
+
+
+Why BLK?
+--------
+
+While PMEM provides direct byte-addressable CPU-load/store access to
+NVDIMM storage, it does not provide the best system RAS (recovery,
+availability, and serviceability) model. An access to a corrupted
+system-physical-address address causes a cpu exception while an access
+to a corrupted address through an BLK-aperture causes that block window
+to raise an error status in a register. The latter is more aligned with
+the standard error model that host-bus-adapter attached disks present.
+Also, if an administrator ever wants to replace a memory it is easier to
+service a system at DIMM module boundaries. Compare this to PMEM where
+data could be interleaved in an opaque hardware specific manner across
+several DIMMs.
+
+PMEM vs BLK
+BLK-apertures solve this RAS problem, but their presence is also the
+major contributing factor to the complexity of the ND subsystem. They
+complicate the implementation because PMEM and BLK alias in DPA space.
+Any given DIMM's DPA-range may contribute to one or more
+system-physical-address sets of interleaved DIMMs, *and* may also be
+accessed in its entirety through its BLK-aperture. Accessing a DPA
+through a system-physical-address while simultaneously accessing the
+same DPA through a BLK-aperture has undefined results. For this reason,
+DIMMs with this dual interface configuration include a DSM function to
+store/retrieve a LABEL. The LABEL effectively partitions the DPA-space
+into exclusive system-physical-address and BLK-aperture accessible
+regions. For simplicity a DIMM is allowed a PMEM "region" per each
+interleave set in which it is a member. The remaining DPA space can be
+carved into an arbitrary number of BLK devices with discontiguous
+extents.
+
+BLK-REGIONs, PMEM-REGIONs, Atomic Sectors, and DAX
+--------------------------------------------------
+
+One of the few
+reasons to allow multiple BLK namespaces per REGION is so that each
+BLK-namespace can be configured with a BTT with unique atomic sector
+sizes. While a PMEM device can host a BTT the LABEL specification does
+not provide for a sector size to be specified for a PMEM namespace.
+This is due to the expectation that the primary usage model for PMEM is
+via DAX, and the BTT is incompatible with DAX. However, for the cases
+where an application or filesystem still needs atomic sector update
+guarantees it can register a BTT on a PMEM device or partition. See
+LIBNVDIMM/NDCTL: Block Translation Table "btt"
+
+
+Example NVDIMM Platform
+-----------------------
+
+For the remainder of this document the following diagram will be
+referenced for any example sysfs layouts.
+
+
+ (a) (b) DIMM BLK-REGION
+ +-------------------+--------+--------+--------+
++------+ | pm0.0 | blk2.0 | pm1.0 | blk2.1 | 0 region2
+| imc0 +--+- - - region0- - - +--------+ +--------+
++--+---+ | pm0.0 | blk3.0 | pm1.0 | blk3.1 | 1 region3
+ | +-------------------+--------v v--------+
++--+---+ | |
+| cpu0 | region1
++--+---+ | |
+ | +----------------------------^ ^--------+
++--+---+ | blk4.0 | pm1.0 | blk4.0 | 2 region4
+| imc1 +--+----------------------------| +--------+
++------+ | blk5.0 | pm1.0 | blk5.0 | 3 region5
+ +----------------------------+--------+--------+
+
+In this platform we have four DIMMs and two memory controllers in one
+socket. Each unique interface (BLK or PMEM) to DPA space is identified
+by a region device with a dynamically assigned id (REGION0 - REGION5).
+
+ 1. The first portion of DIMM0 and DIMM1 are interleaved as REGION0. A
+ single PMEM namespace is created in the REGION0-SPA-range that spans
+ DIMM0 and DIMM1 with a user-specified name of "pm0.0". Some of that
+ interleaved system-physical-address range is reclaimed as BLK-aperture
+ accessed space starting at DPA-offset (a) into each DIMM. In that
+ reclaimed space we create two BLK-aperture "namespaces" from REGION2 and
+ REGION3 where "blk2.0" and "blk3.0" are just human readable names that
+ could be set to any user-desired name in the LABEL.
+
+ 2. In the last portion of DIMM0 and DIMM1 we have an interleaved
+ system-physical-address range, REGION1, that spans those two DIMMs as
+ well as DIMM2 and DIMM3. Some of REGION1 allocated to a PMEM namespace
+ named "pm1.0" the rest is reclaimed in 4 BLK-aperture namespaces (for
+ each DIMM in the interleave set), "blk2.1", "blk3.1", "blk4.0", and
+ "blk5.0".
+
+ 3. The portion of DIMM2 and DIMM3 that do not participate in the REGION1
+ interleaved system-physical-address range (i.e. the DPA address below
+ offset (b) are also included in the "blk4.0" and "blk5.0" namespaces.
+ Note, that this example shows that BLK-aperture namespaces don't need to
+ be contiguous in DPA-space.
+
+ This bus is provided by the kernel under the device
+ /sys/devices/platform/nfit_test.0 when CONFIG_NFIT_TEST is enabled and
+ the nfit_test.ko module is loaded. This not only test LIBNVDIMM but the
+ acpi_nfit.ko driver as well.
+
+
+LIBNVDIMM Kernel Device Model and LIBNDCTL Userspace API
+----------------------------------------------------
+
+What follows is a description of the LIBNVDIMM sysfs layout and a
+corresponding object hierarchy diagram as viewed through the LIBNDCTL
+api. The example sysfs paths and diagrams are relative to the Example
+NVDIMM Platform which is also the LIBNVDIMM bus used in the LIBNDCTL unit
+test.
+
+LIBNDCTL: Context
+Every api call in the LIBNDCTL library requires a context that holds the
+logging parameters and other library instance state. The library is
+based on the libabc template:
+https://git.kernel.org/cgit/linux/kernel/git/kay/libabc.git/
+
+LIBNDCTL: instantiate a new library context example
+
+ struct ndctl_ctx *ctx;
+
+ if (ndctl_new(&ctx) == 0)
+ return ctx;
+ else
+ return NULL;
+
+LIBNVDIMM/LIBNDCTL: Bus
+-------------------
+
+A bus has a 1:1 relationship with an NFIT. The current expectation for
+ACPI based systems is that there is only ever one platform-global NFIT.
+That said, it is trivial to register multiple NFITs, the specification
+does not preclude it. The infrastructure supports multiple busses and
+we we use this capability to test multiple NFIT configurations in the
+unit test.
+
+LIBNVDIMM: control class device in /sys/class
+
+This character device accepts DSM messages to be passed to DIMM
+identified by its NFIT handle.
+
+ /sys/class/nd/ndctl0
+ |-- dev
+ |-- device -> ../../../ndbus0
+ |-- subsystem -> ../../../../../../../class/nd
+
+
+
+LIBNVDIMM: bus
+
+ struct nvdimm_bus *nvdimm_bus_register(struct device *parent,
+ struct nvdimm_bus_descriptor *nfit_desc);
+
+ /sys/devices/platform/nfit_test.0/ndbus0
+ |-- btt0
+ |-- btt_seed
+ |-- commands
+ |-- nd
+ |-- nfit
+ |-- nmem0
+ |-- nmem1
+ |-- nmem2
+ |-- nmem3
+ |-- power
+ |-- provider
+ |-- region0
+ |-- region1
+ |-- region2
+ |-- region3
+ |-- region4
+ |-- region5
+ |-- uevent
+ `-- wait_probe
+
+LIBNDCTL: bus enumeration example
+Find the bus handle that describes the bus from Example NVDIMM Platform
+
+ static struct ndctl_bus *get_bus_by_provider(struct ndctl_ctx *ctx,
+ const char *provider)
+ {
+ struct ndctl_bus *bus;
+
+ ndctl_bus_foreach(ctx, bus)
+ if (strcmp(provider, ndctl_bus_get_provider(bus)) == 0)
+ return bus;
+
+ return NULL;
+ }
+
+ bus = get_bus_by_provider(ctx, "nfit_test.0");
+
+
+LIBNVDIMM/LIBNDCTL: DIMM (NMEM)
+---------------------------
+
+The DIMM device provides a character device for sending commands to
+hardware, and it is a container for LABELs. If the DIMM is defined by
+NFIT then an optional 'nfit' attribute sub-directory is available to add
+NFIT-specifics.
+
+Note that the kernel device name for "DIMMs" is "nmemX". The NFIT
+describes these devices via "Memory Device to System Physical Address
+Range Mapping Structure", and there is no requirement that they actually
+be physical DIMMs, so we use a more generic name.
+
+LIBNVDIMM: DIMM (NMEM)
+
+ struct nvdimm *nvdimm_create(struct nvdimm_bus *nvdimm_bus, void *provider_data,
+ const struct attribute_group **groups, unsigned long flags,
+ unsigned long *dsm_mask);
+
+ /sys/devices/platform/nfit_test.0/ndbus0
+ |-- nmem0
+ | |-- available_slots
+ | |-- commands
+ | |-- dev
+ | |-- devtype
+ | |-- driver -> ../../../../../bus/nd/drivers/nvdimm
+ | |-- modalias
+ | |-- nfit
+ | | |-- device
+ | | |-- format
+ | | |-- handle
+ | | |-- phys_id
+ | | |-- rev_id
+ | | |-- serial
+ | | `-- vendor
+ | |-- state
+ | |-- subsystem -> ../../../../../bus/nd
+ | `-- uevent
+ |-- nmem1
+ [..]
+
+
+LIBNDCTL: DIMM enumeration example
+
+Note, in this example we are assuming NFIT-defined DIMMs which are
+identified by an "nfit_handle" a 32-bit value where:
+Bit 3:0 DIMM number within the memory channel
+Bit 7:4 memory channel number
+Bit 11:8 memory controller ID
+Bit 15:12 socket ID (within scope of a Node controller if node controller is present)
+Bit 27:16 Node Controller ID
+Bit 31:28 Reserved
+
+ static struct ndctl_dimm *get_dimm_by_handle(struct ndctl_bus *bus,
+ unsigned int handle)
+ {
+ struct ndctl_dimm *dimm;
+
+ ndctl_dimm_foreach(bus, dimm)
+ if (ndctl_dimm_get_handle(dimm) == handle)
+ return dimm;
+
+ return NULL;
+ }
+
+ #define DIMM_HANDLE(n, s, i, c, d) \
+ (((n & 0xfff) << 16) | ((s & 0xf) << 12) | ((i & 0xf) << 8) \
+ | ((c & 0xf) << 4) | (d & 0xf))
+
+ dimm = get_dimm_by_handle(bus, DIMM_HANDLE(0, 0, 0, 0, 0));
+
+LIBNVDIMM/LIBNDCTL: Region
+----------------------
+
+A generic REGION device is registered for each PMEM range orBLK-aperture
+set. Per the example there are 6 regions: 2 PMEM and 4 BLK-aperture
+sets on the "nfit_test.0" bus. The primary role of regions are to be a
+container of "mappings". A mapping is a tuple of <DIMM,
+DPA-start-offset, length>.
+
+LIBNVDIMM provides a built-in driver for these REGION devices. This driver
+is responsible for reconciling the aliased DPA mappings across all
+regions, parsing the LABEL, if present, and then emitting NAMESPACE
+devices with the resolved/exclusive DPA-boundaries for the nd_pmem or
+nd_blk device driver to consume.
+
+In addition to the generic attributes of "mapping"s, "interleave_ways"
+and "size" the REGION device also exports some convenience attributes.
+"nstype" indicates the integer type of namespace-device this region
+emits, "devtype" duplicates the DEVTYPE variable stored by udev at the
+'add' event, "modalias" duplicates the MODALIAS variable stored by udev
+at the 'add' event, and finally, the optional "spa_index" is provided in
+the case where the region is defined by a SPA.
+
+LIBNVDIMM: region
+
+ struct nd_region *nvdimm_pmem_region_create(struct nvdimm_bus *nvdimm_bus,
+ struct nd_region_desc *ndr_desc);
+ struct nd_region *nvdimm_blk_region_create(struct nvdimm_bus *nvdimm_bus,
+ struct nd_region_desc *ndr_desc);
+
+ /sys/devices/platform/nfit_test.0/ndbus0
+ |-- region0
+ | |-- available_size
+ | |-- devtype
+ | |-- driver -> ../../../../../bus/nd/drivers/nd_region
+ | |-- init_namespaces
+ | |-- mapping0
+ | |-- mapping1
+ | |-- mappings
+ | |-- modalias
+ | |-- namespace0.0
+ | |-- namespace_seed
+ | |-- nfit
+ | | `-- spa_index
+ | |-- nstype
+ | |-- set_cookie
+ | |-- size
+ | |-- subsystem -> ../../../../../bus/nd
+ | `-- uevent
+ |-- region1
+ [..]
+
+LIBNDCTL: region enumeration example
+
+Sample region retrieval routines based on NFIT-unique data like
+"spa_index" (interleave set id) for PMEM and "nfit_handle" (dimm id) for
+BLK.
+
+ static struct ndctl_region *get_pmem_region_by_spa_index(struct ndctl_bus *bus,
+ unsigned int spa_index)
+ {
+ struct ndctl_region *region;
+
+ ndctl_region_foreach(bus, region) {
+ if (ndctl_region_get_type(region) != ND_DEVICE_REGION_PMEM)
+ continue;
+ if (ndctl_region_get_spa_index(region) == spa_index)
+ return region;
+ }
+ return NULL;
+ }
+
+ static struct ndctl_region *get_blk_region_by_dimm_handle(struct ndctl_bus *bus,
+ unsigned int handle)
+ {
+ struct ndctl_region *region;
+
+ ndctl_region_foreach(bus, region) {
+ struct ndctl_mapping *map;
+
+ if (ndctl_region_get_type(region) != ND_DEVICE_REGION_BLOCK)
+ continue;
+ ndctl_mapping_foreach(region, map) {
+ struct ndctl_dimm *dimm = ndctl_mapping_get_dimm(map);
+
+ if (ndctl_dimm_get_handle(dimm) == handle)
+ return region;
+ }
+ }
+ return NULL;
+ }
+
+
+Why Not Encode the Region Type into the Region Name?
+----------------------------------------------------
+
+At first glance it seems since NFIT defines just PMEM and BLK interface
+types that we should simply name REGION devices with something derived
+from those type names. However, the ND subsystem explicitly keeps the
+REGION name generic and expects userspace to always consider the
+region-attributes for 4 reasons:
+
+ 1. There are already more than two REGION and "namespace" types. For
+ PMEM there are two subtypes. As mentioned previously we have PMEM where
+ the constituent DIMM devices are known and anonymous PMEM. For BLK
+ regions the NFIT specification already anticipates vendor specific
+ implementations. The exact distinction of what a region contains is in
+ the region-attributes not the region-name or the region-devtype.
+
+ 2. A region with zero child-namespaces is a possible configuration. For
+ example, the NFIT allows for a DCR to be published without a
+ corresponding BLK-aperture. This equates to a DIMM that can only accept
+ control/configuration messages, but no i/o through a descendant block
+ device. Again, this "type" is advertised in the attributes ('mappings'
+ == 0) and the name does not tell you much.
+
+ 3. What if a third major interface type arises in the future? Outside
+ of vendor specific implementations, it's not difficult to envision a
+ third class of interface type beyond BLK and PMEM. With a generic name
+ for the REGION level of the device-hierarchy old userspace
+ implementations can still make sense of new kernel advertised
+ region-types. Userspace can always rely on the generic region
+ attributes like "mappings", "size", etc and the expected child devices
+ named "namespace". This generic format of the device-model hierarchy
+ allows the LIBNVDIMM and LIBNDCTL implementations to be more uniform and
+ future-proof.
+
+ 4. There are more robust mechanisms for determining the major type of a
+ region than a device name. See the next section, How Do I Determine the
+ Major Type of a Region?
+
+How Do I Determine the Major Type of a Region?
+----------------------------------------------
+
+Outside of the blanket recommendation of "use libndctl", or simply
+looking at the kernel header (/usr/include/linux/ndctl.h) to decode the
+"nstype" integer attribute, here are some other options.
+
+ 1. module alias lookup:
+
+ The whole point of region/namespace device type differentiation is to
+ decide which block-device driver will attach to a given LIBNVDIMM namespace.
+ One can simply use the modalias to lookup the resulting module. It's
+ important to note that this method is robust in the presence of a
+ vendor-specific driver down the road. If a vendor-specific
+ implementation wants to supplant the standard nd_blk driver it can with
+ minimal impact to the rest of LIBNVDIMM.
+
+ In fact, a vendor may also want to have a vendor-specific region-driver
+ (outside of nd_region). For example, if a vendor defined its own LABEL
+ format it would need its own region driver to parse that LABEL and emit
+ the resulting namespaces. The output from module resolution is more
+ accurate than a region-name or region-devtype.
+
+ 2. udev:
+
+ The kernel "devtype" is registered in the udev database
+ # udevadm info --path=/devices/platform/nfit_test.0/ndbus0/region0
+ P: /devices/platform/nfit_test.0/ndbus0/region0
+ E: DEVPATH=/devices/platform/nfit_test.0/ndbus0/region0
+ E: DEVTYPE=nd_pmem
+ E: MODALIAS=nd:t2
+ E: SUBSYSTEM=nd
+
+ # udevadm info --path=/devices/platform/nfit_test.0/ndbus0/region4
+ P: /devices/platform/nfit_test.0/ndbus0/region4
+ E: DEVPATH=/devices/platform/nfit_test.0/ndbus0/region4
+ E: DEVTYPE=nd_blk
+ E: MODALIAS=nd:t3
+ E: SUBSYSTEM=nd
+
+ ...and is available as a region attribute, but keep in mind that the
+ "devtype" does not indicate sub-type variations and scripts should
+ really be understanding the other attributes.
+
+ 3. type specific attributes:
+
+ As it currently stands a BLK-aperture region will never have a
+ "nfit/spa_index" attribute, but neither will a non-NFIT PMEM region. A
+ BLK region with a "mappings" value of 0 is, as mentioned above, a DIMM
+ that does not allow I/O. A PMEM region with a "mappings" value of zero
+ is a simple system-physical-address range.
+
+
+LIBNVDIMM/LIBNDCTL: Namespace
+-------------------------
+
+A REGION, after resolving DPA aliasing and LABEL specified boundaries,
+surfaces one or more "namespace" devices. The arrival of a "namespace"
+device currently triggers either the nd_blk or nd_pmem driver to load
+and register a disk/block device.
+
+LIBNVDIMM: namespace
+Here is a sample layout from the three major types of NAMESPACE where
+namespace0.0 represents DIMM-info-backed PMEM (note that it has a 'uuid'
+attribute), namespace2.0 represents a BLK namespace (note it has a
+'sector_size' attribute) that, and namespace6.0 represents an anonymous
+PMEM namespace (note that has no 'uuid' attribute due to not support a
+LABEL).
+
+ /sys/devices/platform/nfit_test.0/ndbus0/region0/namespace0.0
+ |-- alt_name
+ |-- devtype
+ |-- dpa_extents
+ |-- modalias
+ |-- resource
+ |-- size
+ |-- subsystem -> ../../../../../../bus/nd
+ |-- type
+ |-- uevent
+ `-- uuid
+ /sys/devices/platform/nfit_test.0/ndbus0/region2/namespace2.0
+ |-- alt_name
+ |-- devtype
+ |-- dpa_extents
+ |-- modalias
+ |-- sector_size
+ |-- size
+ |-- subsystem -> ../../../../../../bus/nd
+ |-- type
+ |-- uevent
+ `-- uuid
+ /sys/devices/platform/nfit_test.1/ndbus1/region6/namespace6.0
+ |-- block
+ | `-- pmem0
+ |-- devtype
+ |-- driver -> ../../../../../../bus/nd/drivers/pmem
+ |-- modalias
+ |-- resource
+ |-- size
+ |-- subsystem -> ../../../../../../bus/nd
+ |-- type
+ `-- uevent
+
+LIBNDCTL: namespace enumeration example
+Namespaces are indexed relative to their parent region, example below.
+These indexes are mostly static from boot to boot, but subsystem makes
+no guarantees in this regard. For a static namespace identifier use its
+'uuid' attribute.
+
+static struct ndctl_namespace *get_namespace_by_id(struct ndctl_region *region,
+ unsigned int id)
+{
+ struct ndctl_namespace *ndns;
+
+ ndctl_namespace_foreach(region, ndns)
+ if (ndctl_namespace_get_id(ndns) == id)
+ return ndns;
+
+ return NULL;
+}
+
+LIBNDCTL: namespace creation example
+Idle namespaces are automatically created by the kernel if a given
+region has enough available capacity to create a new namespace.
+Namespace instantiation involves finding an idle namespace and
+configuring it. For the most part the setting of namespace attributes
+can occur in any order, the only constraint is that 'uuid' must be set
+before 'size'. This enables the kernel to track DPA allocations
+internally with a static identifier.
+
+static int configure_namespace(struct ndctl_region *region,
+ struct ndctl_namespace *ndns,
+ struct namespace_parameters *parameters)
+{
+ char devname[50];
+
+ snprintf(devname, sizeof(devname), "namespace%d.%d",
+ ndctl_region_get_id(region), paramaters->id);
+
+ ndctl_namespace_set_alt_name(ndns, devname);
+ /* 'uuid' must be set prior to setting size! */
+ ndctl_namespace_set_uuid(ndns, paramaters->uuid);
+ ndctl_namespace_set_size(ndns, paramaters->size);
+ /* unlike pmem namespaces, blk namespaces have a sector size */
+ if (parameters->lbasize)
+ ndctl_namespace_set_sector_size(ndns, parameters->lbasize);
+ ndctl_namespace_enable(ndns);
+}
+
+
+Why the Term "namespace"?
+
+ 1. Why not "volume" for instance? "volume" ran the risk of confusing ND
+ as a volume manager like device-mapper.
+
+ 2. The term originated to describe the sub-devices that can be created
+ within a NVME controller (see the nvme specification:
+ http://www.nvmexpress.org/specifications/), and NFIT namespaces are
+ meant to parallel the capabilities and configurability of
+ NVME-namespaces.
+
+
+LIBNVDIMM/LIBNDCTL: Block Translation Table "btt"
+---------------------------------------------
+
+A BTT (design document: http://pmem.io/2014/09/23/btt.html) is a stacked
+block device driver that fronts either the whole block device or a
+partition of a block device emitted by either a PMEM or BLK NAMESPACE.
+
+LIBNVDIMM: btt layout
+Every bus will start out with at least one BTT device which is the seed
+device. To activate it set the "backing_dev", "uuid", and "sector_size"
+attributes and then bind the device to the nd_btt driver.
+
+ /sys/devices/platform/nfit_test.1/ndbus0/btt0/
+ |-- backing_dev
+ |-- delete
+ |-- devtype
+ |-- modalias
+ |-- sector_size
+ |-- subsystem -> ../../../../../bus/nd
+ |-- uevent
+ `-- uuid
+
+LIBNDCTL: btt creation example
+Similar to namespaces an idle BTT device is automatically created per
+bus. Each time this "seed" btt device is configured and enabled a new
+seed is created. Creating a BTT configuration involves two steps of
+finding and idle BTT and assigning it to front a PMEM or BLK namespace.
+
+ static struct ndctl_btt *get_idle_btt(struct ndctl_bus *bus)
+ {
+ struct ndctl_btt *btt;
+
+ ndctl_btt_foreach(bus, btt)
+ if (!ndctl_btt_is_enabled(btt) && !ndctl_btt_is_configured(btt))
+ return btt;
+
+ return NULL;
+ }
+
+ static int configure_btt(struct ndctl_bus *bus, struct btt_parameters *parameters)
+ {
+ btt = get_idle_btt(bus);
+
+ sprintf(bdevpath, "/dev/%s",
+ ndctl_namespace_get_block_device(parameters->ndns));
+ ndctl_btt_set_uuid(btt, parameters->uuid);
+ ndctl_btt_set_sector_size(btt, parameters->sector_size);
+ ndctl_btt_set_backing_dev(btt, parameters->bdevpath);
+ ndctl_btt_enable(btt);
+ }
+
+Once instantiated a "nd_btt" link will be created under the
+"backing_dev" (pmem0) block device:
+
+ /sys/block/pmem0/
+ |-- alignment_offset
+ |-- bdi -> ../../../../../../../virtual/bdi/259:0
+ |-- capability
+ |-- dev
+ |-- device -> ../../../namespace0.0
+ |-- discard_alignment
+ |-- ext_range
+ |-- holders
+ |-- inflight
+ |-- nd_btt -> ../../../../btt0
+
+...and a new inactive seed device will appear on the bus.
+
+Once a "backing_dev" is disabled its associated BTT will be
+automatically deleted. This deletion is only at the device model level.
+In order to destroy a BTT the "info block" needs to be destroyed.
+
+
+Summary LIBNDCTL Diagram
+------------------------
+
+For the given example above, here is the view of the objects as seen by the LIBNDCTL api:
+ +---+
+ |CTX| +---------+ +--------------+ +---------------+
+ +-+-+ +-> REGION0 +---> NAMESPACE0.0 +--> PMEM8 "pm0.0" |
+ | | +---------+ +--------------+ +---------------+
++-------+ | | +---------+ +--------------+ +---------------+
+| DIMM0 <-+ | +-> REGION1 +---> NAMESPACE1.0 +--> PMEM6 "pm1.0" |
++-------+ | | | +---------+ +--------------+ +---------------+
+| DIMM1 <-+ +-v--+ | +---------+ +--------------+ +---------------+
++-------+ +-+BUS0+---> REGION2 +-+-> NAMESPACE2.0 +--> ND6 "blk2.0" |
+| DIMM2 <-+ +----+ | +---------+ | +--------------+ +----------------------+
++-------+ | | +-> NAMESPACE2.1 +--> ND5 "blk2.1" | BTT2 |
+| DIMM3 <-+ | +--------------+ +----------------------+
++-------+ | +---------+ +--------------+ +---------------+
+ +-> REGION3 +-+-> NAMESPACE3.0 +--> ND4 "blk3.0" |
+ | +---------+ | +--------------+ +----------------------+
+ | +-> NAMESPACE3.1 +--> ND3 "blk3.1" | BTT1 |
+ | +--------------+ +----------------------+
+ | +---------+ +--------------+ +---------------+
+ +-> REGION4 +---> NAMESPACE4.0 +--> ND2 "blk4.0" |
+ | +---------+ +--------------+ +---------------+
+ | +---------+ +--------------+ +----------------------+
+ +-> REGION5 +---> NAMESPACE5.0 +--> ND1 "blk5.0" | BTT0 |
+ +---------+ +--------------+ +---------------+------+
+
+
diff --git a/MAINTAINERS b/MAINTAINERS
index 590304b96b03..d7aa8c9c01c5 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5912,6 +5912,39 @@ M: Sasha Levin <sasha.levin@oracle.com>
S: Maintained
F: tools/lib/lockdep/
+LIBNVDIMM: NON-VOLATILE MEMORY DEVICE SUBSYSTEM
+M: Dan Williams <dan.j.williams@intel.com>
+L: linux-nvdimm@lists.01.org
+Q: https://patchwork.kernel.org/project/linux-nvdimm/list/
+S: Supported
+F: drivers/nvdimm/*
+F: include/linux/nd.h
+F: include/linux/libnvdimm.h
+F: include/uapi/linux/ndctl.h
+
+LIBNVDIMM BLK: MMIO-APERTURE DRIVER
+M: Ross Zwisler <ross.zwisler@linux.intel.com>
+L: linux-nvdimm@lists.01.org
+Q: https://patchwork.kernel.org/project/linux-nvdimm/list/
+S: Supported
+F: drivers/nvdimm/blk.c
+F: drivers/nvdimm/region_devs.c
+F: drivers/acpi/nfit*
+
+LIBNVDIMM BTT: BLOCK TRANSLATION TABLE
+M: Vishal Verma <vishal.verma@linux.intel.com>
+L: linux-nvdimm@lists.01.org
+Q: https://patchwork.kernel.org/project/linux-nvdimm/list/
+S: Supported
+F: drivers/nvdimm/btt*
+
+LIBNVDIMM PMEM: PERSISTENT MEMORY DRIVER
+M: Ross Zwisler <ross.zwisler@linux.intel.com>
+L: linux-nvdimm@lists.01.org
+Q: https://patchwork.kernel.org/project/linux-nvdimm/list/
+S: Supported
+F: drivers/nvdimm/pmem.c
+
LINUX FOR IBM pSERIES (RS/6000)
M: Paul Mackerras <paulus@au.ibm.com>
W: http://www.ibm.com/linux/ltc/projects/ppc
@@ -8136,12 +8169,6 @@ S: Maintained
F: Documentation/blockdev/ramdisk.txt
F: drivers/block/brd.c
-PERSISTENT MEMORY DRIVER
-M: Ross Zwisler <ross.zwisler@linux.intel.com>
-L: linux-nvdimm@lists.01.org
-S: Supported
-F: drivers/block/pmem.c
-
RANDOM NUMBER DRIVER
M: "Theodore Ts'o" <tytso@mit.edu>
S: Maintained
^ permalink raw reply related
* [PATCH v5 20/21] tools/testing/nvdimm: manufactured NFITs for interface development
From: Dan Williams @ 2015-06-02 0:15 UTC (permalink / raw)
To: linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw
Cc: axboe-tSWWG44O7X1aa/9Udqfwiw, sfr-3FnU+UHB4dNDw9hX6IcOSA,
rafael-DgEjT+Ai2ygdnm+yROfE0A, neilb-l3A5Bk7waGM,
gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, mingo-DgEjT+Ai2ygdnm+yROfE0A,
linux-acpi-u79uwXL29TY76Z2rM5mHXA, jmoyer-H+wXaHxf7aLQT0dZR+AlfA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b, hch-jcswGhMUV9g
In-Reply-To: <20150602001134.4506.45867.stgit-p8uTFz9XbKj2zm6wflaqv1nYeNYlB/vhral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Manually create and register NFITs to describe 2 topologies. Topology1
is an advanced plausible configuration for BLK/PMEM aliased NVDIMMs.
Topology2 is an example configuration for current platforms that only
ship with a persistent address range.
Kernel provider "nfit_test.0" produces an NFIT with the following attributes:
(a) (b) DIMM BLK-REGION
+-------------------+--------+--------+--------+
+------+ | pm0.0 | blk2.0 | pm1.0 | blk2.1 | 0 region2
| imc0 +--+- - - region0- - - +--------+ +--------+
+--+---+ | pm0.0 | blk3.0 | pm1.0 | blk3.1 | 1 region3
| +-------------------+--------v v--------+
+--+---+ | |
| cpu0 | region1
+--+---+ | |
| +----------------------------^ ^--------+
+--+---+ | blk4.0 | pm1.0 | blk4.0 | 2 region4
| imc1 +--+----------------------------| +--------+
+------+ | blk5.0 | pm1.0 | blk5.0 | 3 region5
+----------------------------+--------+--------+
*) In this layout we have four dimms and two memory controllers in one
socket. Each unique interface ("block" or "pmem") to DPA space
is identified by a region device with a dynamically assigned id.
*) The first portion of dimm0 and dimm1 are interleaved as REGION0.
A single "pmem" namespace is created in the REGION0-"spa"-range
that spans dimm0 and dimm1 with a user-specified name of "pm0.0".
Some of that interleaved "spa" range is reclaimed as "bdw"
accessed space starting at offset (a) into each dimm. In that
reclaimed space we create two "bdw" "namespaces" from REGION2 and
REGION3 where "blk2.0" and "blk3.0" are just human readable names
that could be set to any user-desired name in the label.
*) In the last portion of dimm0 and dimm1 we have an interleaved
"spa" range, REGION1, that spans those two dimms as well as dimm2
and dimm3. Some of REGION1 allocated to a "pmem" namespace named
"pm1.0" the rest is reclaimed in 4 "bdw" namespaces (for each
dimm in the interleave set), "blk2.1", "blk3.1", "blk4.0", and
"blk5.0".
*) The portion of dimm2 and dimm3 that do not participate in the
REGION1 interleaved "spa" range (i.e. the DPA address below
offset (b) are also included in the "blk4.0" and "blk5.0"
namespaces. Note, that this example shows that "bdw" namespaces
don't need to be contiguous in DPA-space.
Kernel provider "nfit_test.1" produces an NFIT with the following attributes:
region2
+---------------------+
|---------------------|
|| pm2.0 ||
|---------------------|
+---------------------+
*) Describes a simple system-physical-address range with no backing
dimm or interleave description.
Cc: <linux-acpi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Cc: Lv Zheng <lv.zheng-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Cc: Robert Moore <robert.moore-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Cc: Rafael J. Wysocki <rafael.j.wysocki-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/acpi/Makefile | 1
drivers/acpi/nfit.c | 14
drivers/acpi/nfit.h | 14
drivers/nvdimm/Makefile | 1
drivers/nvdimm/blk.c | 2
drivers/nvdimm/btt.c | 4
drivers/nvdimm/bus.c | 4
drivers/nvdimm/dimm.c | 4
drivers/nvdimm/label.c | 6
drivers/nvdimm/namespace_devs.c | 4
drivers/nvdimm/pmem.c | 2
drivers/nvdimm/region.c | 2
drivers/nvdimm/region_devs.c | 4
tools/testing/nvdimm/Kbuild | 37 +
tools/testing/nvdimm/Makefile | 7
tools/testing/nvdimm/test/Kbuild | 8
tools/testing/nvdimm/test/iomap.c | 151 +++++
tools/testing/nvdimm/test/nfit.c | 1101 +++++++++++++++++++++++++++++++++
tools/testing/nvdimm/test/nfit_test.h | 28 +
19 files changed, 1373 insertions(+), 21 deletions(-)
create mode 100644 tools/testing/nvdimm/Kbuild
create mode 100644 tools/testing/nvdimm/Makefile
create mode 100644 tools/testing/nvdimm/test/Kbuild
create mode 100644 tools/testing/nvdimm/test/iomap.c
create mode 100644 tools/testing/nvdimm/test/nfit.c
create mode 100644 tools/testing/nvdimm/test/nfit_test.h
diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile
index f7e9c92ccdcb..f0f2103f8575 100644
--- a/drivers/acpi/Makefile
+++ b/drivers/acpi/Makefile
@@ -4,6 +4,7 @@
ccflags-y := -Os
ccflags-$(CONFIG_ACPI_DEBUG) += -DACPI_DEBUG_OUTPUT
+ccflags-y += -I$(src)
#
# ACPI Boot-Time Table Parsing
diff --git a/drivers/acpi/nfit.c b/drivers/acpi/nfit.c
index 1124c47f7b06..5731e4ab18a0 100644
--- a/drivers/acpi/nfit.c
+++ b/drivers/acpi/nfit.c
@@ -19,7 +19,7 @@
#include <linux/acpi.h>
#include <linux/sort.h>
#include <linux/io.h>
-#include "nfit.h"
+#include <nfit.h>
#include <asm-generic/io-64-nonatomic-hi-lo.h>
@@ -29,10 +29,11 @@ MODULE_PARM_DESC(force_enable_dimms, "Ignore _STA (ACPI DIMM device) status");
static u8 nfit_uuid[NFIT_UUID_MAX][16];
-static const u8 *to_nfit_uuid(enum nfit_uuids id)
+const u8 *to_nfit_uuid(enum nfit_uuids id)
{
return nfit_uuid[id];
}
+EXPORT_SYMBOL(to_nfit_uuid);
static struct acpi_nfit_desc *to_acpi_nfit_desc(struct nvdimm_bus_descriptor *nd_desc)
{
@@ -529,11 +530,12 @@ static struct attribute_group acpi_nfit_attribute_group = {
.attrs = acpi_nfit_attributes,
};
-static const struct attribute_group *acpi_nfit_attribute_groups[] = {
+const struct attribute_group *acpi_nfit_attribute_groups[] = {
&nvdimm_bus_attribute_group,
&acpi_nfit_attribute_group,
NULL,
};
+EXPORT_SYMBOL_GPL(acpi_nfit_attribute_groups);
static struct acpi_nfit_memory_map *to_nfit_memdev(struct device *dev)
{
@@ -1295,7 +1297,7 @@ static int acpi_nfit_register_region(struct acpi_nfit_desc *acpi_desc,
ndr_desc->num_mappings = blk_valid;
ndbr_desc.enable = acpi_nfit_blk_region_enable;
ndbr_desc.disable = acpi_nfit_blk_region_disable;
- ndbr_desc.do_io = acpi_nfit_blk_region_do_io;
+ ndbr_desc.do_io = acpi_desc->blk_do_io;
if (!nvdimm_blk_region_create(acpi_desc->nvdimm_bus,
ndr_desc))
return -ENOMEM;
@@ -1331,7 +1333,7 @@ static int acpi_nfit_register_regions(struct acpi_nfit_desc *acpi_desc)
return 0;
}
-static int acpi_nfit_init(struct acpi_nfit_desc *acpi_desc, acpi_size sz)
+int acpi_nfit_init(struct acpi_nfit_desc *acpi_desc, acpi_size sz)
{
struct device *dev = acpi_desc->dev;
const void *end;
@@ -1370,6 +1372,7 @@ static int acpi_nfit_init(struct acpi_nfit_desc *acpi_desc, acpi_size sz)
return acpi_nfit_register_regions(acpi_desc);
}
+EXPORT_SYMBOL_GPL(acpi_nfit_init);
static int acpi_nfit_add(struct acpi_device *adev)
{
@@ -1394,6 +1397,7 @@ static int acpi_nfit_add(struct acpi_device *adev)
dev_set_drvdata(dev, acpi_desc);
acpi_desc->dev = dev;
acpi_desc->nfit = (struct acpi_table_nfit *) tbl;
+ acpi_desc->blk_do_io = acpi_nfit_blk_region_do_io;
nd_desc = &acpi_desc->nd_desc;
nd_desc->provider_name = "ACPI.NFIT";
nd_desc->ndctl = acpi_nfit_ctl;
diff --git a/drivers/acpi/nfit.h b/drivers/acpi/nfit.h
index b2e56becc8fd..6cad6f2327b5 100644
--- a/drivers/acpi/nfit.h
+++ b/drivers/acpi/nfit.h
@@ -37,6 +37,15 @@ enum nfit_uuids {
NFIT_UUID_MAX,
};
+#define NFIT_DIMM_HANDLE(node, socket, imc, chan, dimm) \
+ (((node & 0xfff) << 16) | ((socket & 0xf) << 12) \
+ | ((imc & 0xf) << 8) | ((chan & 0xf) << 4) | (dimm & 0xf))
+#define NFIT_DIMM_NODE(handle) ((handle) >> 16 & 0xfff)
+#define NFIT_DIMM_SOCKET(handle) ((handle) >> 12 & 0xf)
+#define NFIT_DIMM_CHAN(handle) ((handle) >> 8 & 0xf)
+#define NFIT_DIMM_IMC(handle) ((handle) >> 4 & 0xf)
+#define NFIT_DIMM_DIMM(handle) ((handle) & 0xf)
+
struct nfit_spa {
struct acpi_nfit_system_address *spa;
struct list_head list;
@@ -93,6 +102,8 @@ struct acpi_nfit_desc {
struct nvdimm_bus *nvdimm_bus;
struct device *dev;
unsigned long dimm_dsm_force_en;
+ int (*blk_do_io)(struct nd_blk_region *ndbr, void *iobuf,
+ u64 len, int write, resource_size_t dpa);
};
enum nd_blk_mmio_selector {
@@ -144,4 +155,7 @@ static inline struct acpi_nfit_desc *to_acpi_desc(struct nvdimm_bus_descriptor *
{
return container_of(nd_desc, struct acpi_nfit_desc, nd_desc);
}
+
+const u8 *to_nfit_uuid(enum nfit_uuids id);
+int acpi_nfit_init(struct acpi_nfit_desc *nfit, acpi_size sz);
#endif /* __NFIT_H__ */
diff --git a/drivers/nvdimm/Makefile b/drivers/nvdimm/Makefile
index b5682e70904a..20afdb362260 100644
--- a/drivers/nvdimm/Makefile
+++ b/drivers/nvdimm/Makefile
@@ -1,3 +1,4 @@
+ccflags-y := -I$(src)
obj-$(CONFIG_LIBNVDIMM) += libnvdimm.o
obj-$(CONFIG_BLK_DEV_PMEM) += nd_pmem.o
obj-$(CONFIG_ND_BTT) += nd_btt.o
diff --git a/drivers/nvdimm/blk.c b/drivers/nvdimm/blk.c
index 37b6f52fe1c2..a958636df62c 100644
--- a/drivers/nvdimm/blk.c
+++ b/drivers/nvdimm/blk.c
@@ -19,7 +19,7 @@
#include <linux/moduleparam.h>
#include <linux/nd.h>
#include <linux/sizes.h>
-#include "nd.h"
+#include <nd.h>
struct nd_blk_device {
struct request_queue *queue;
diff --git a/drivers/nvdimm/btt.c b/drivers/nvdimm/btt.c
index 932177294f75..2d7ce9e9b624 100644
--- a/drivers/nvdimm/btt.c
+++ b/drivers/nvdimm/btt.c
@@ -23,8 +23,8 @@
#include <linux/ndctl.h>
#include <linux/fs.h>
#include <linux/nd.h>
-#include "btt.h"
-#include "nd.h"
+#include <btt.h>
+#include <nd.h>
enum log_ent_request {
LOG_NEW_ENT = 0,
diff --git a/drivers/nvdimm/bus.c b/drivers/nvdimm/bus.c
index 5c977c1306db..d8a1794733e2 100644
--- a/drivers/nvdimm/bus.c
+++ b/drivers/nvdimm/bus.c
@@ -24,8 +24,8 @@
#include <linux/io.h>
#include <linux/mm.h>
#include <linux/nd.h>
-#include "nd-private.h"
-#include "nd.h"
+#include <nd-private.h>
+#include <nd.h>
int nvdimm_major;
static int nvdimm_bus_major;
diff --git a/drivers/nvdimm/dimm.c b/drivers/nvdimm/dimm.c
index 2df97c3c3b34..03b7b1bd689d 100644
--- a/drivers/nvdimm/dimm.c
+++ b/drivers/nvdimm/dimm.c
@@ -18,8 +18,8 @@
#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/nd.h>
-#include "label.h"
-#include "nd.h"
+#include <label.h>
+#include <nd.h>
static void free_data(struct nvdimm_drvdata *ndd)
{
diff --git a/drivers/nvdimm/label.c b/drivers/nvdimm/label.c
index 6f67db5a3302..ab09bddbe7b4 100644
--- a/drivers/nvdimm/label.c
+++ b/drivers/nvdimm/label.c
@@ -15,9 +15,9 @@
#include <linux/slab.h>
#include <linux/io.h>
#include <linux/nd.h>
-#include "nd-private.h"
-#include "label.h"
-#include "nd.h"
+#include <nd-private.h>
+#include <label.h>
+#include <nd.h>
#include <asm-generic/io-64-nonatomic-lo-hi.h>
diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
index ab71b454a020..e89b0192ef10 100644
--- a/drivers/nvdimm/namespace_devs.c
+++ b/drivers/nvdimm/namespace_devs.c
@@ -14,8 +14,8 @@
#include <linux/device.h>
#include <linux/slab.h>
#include <linux/nd.h>
-#include "nd-private.h"
-#include "nd.h"
+#include <nd-private.h>
+#include <nd.h>
#include <asm-generic/io-64-nonatomic-lo-hi.h>
diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c
index 0a85a9ca3ca4..0b0f81350d5e 100644
--- a/drivers/nvdimm/pmem.c
+++ b/drivers/nvdimm/pmem.c
@@ -24,7 +24,7 @@
#include <linux/moduleparam.h>
#include <linux/slab.h>
#include <linux/nd.h>
-#include "nd.h"
+#include <nd.h>
struct pmem_device {
struct request_queue *pmem_queue;
diff --git a/drivers/nvdimm/region.c b/drivers/nvdimm/region.c
index d9d82e7a90fa..373eab4f77ca 100644
--- a/drivers/nvdimm/region.c
+++ b/drivers/nvdimm/region.c
@@ -14,7 +14,7 @@
#include <linux/module.h>
#include <linux/device.h>
#include <linux/nd.h>
-#include "nd.h"
+#include <nd.h>
struct nd_percpu_lane {
int count[CONFIG_ND_MAX_REGIONS];
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
index 5790aec081f4..86adbd81afb4 100644
--- a/drivers/nvdimm/region_devs.c
+++ b/drivers/nvdimm/region_devs.c
@@ -17,8 +17,8 @@
#include <linux/sort.h>
#include <linux/io.h>
#include <linux/nd.h>
-#include "nd-private.h"
-#include "nd.h"
+#include <nd-private.h>
+#include <nd.h>
static DEFINE_IDA(region_ida);
diff --git a/tools/testing/nvdimm/Kbuild b/tools/testing/nvdimm/Kbuild
new file mode 100644
index 000000000000..1d93099f0ed6
--- /dev/null
+++ b/tools/testing/nvdimm/Kbuild
@@ -0,0 +1,37 @@
+ldflags-y += --wrap=ioremap_cache
+ldflags-y += --wrap=ioremap_nocache
+ldflags-y += --wrap=iounmap
+ldflags-y += --wrap=__request_region
+ldflags-y += --wrap=__release_region
+
+DRIVERS := ../../../drivers
+NVDIMM_SRC := $(DRIVERS)/nvdimm
+ACPI_SRC := $(DRIVERS)/acpi
+ccflags-y := -I$(src)/$(NVDIMM_SRC)
+ccflags-y += -I$(src)/$(ACPI_SRC)
+
+obj-m += libnvdimm.o
+obj-m += nd_pmem.o
+obj-m += nd_btt.o
+obj-m += nd_blk.o
+obj-m += nfit.o
+
+nfit-y := $(ACPI_SRC)/nfit.o
+
+nd_pmem-y := $(NVDIMM_SRC)/pmem.o
+
+nd_btt-y := $(NVDIMM_SRC)/btt.o
+
+nd_blk-y := $(NVDIMM_SRC)/blk.o
+
+libnvdimm-y := $(NVDIMM_SRC)/core.o
+libnvdimm-y += $(NVDIMM_SRC)/bus.o
+libnvdimm-y += $(NVDIMM_SRC)/dimm_devs.o
+libnvdimm-y += $(NVDIMM_SRC)/dimm.o
+libnvdimm-y += $(NVDIMM_SRC)/region_devs.o
+libnvdimm-y += $(NVDIMM_SRC)/region.o
+libnvdimm-y += $(NVDIMM_SRC)/namespace_devs.o
+libnvdimm-y += $(NVDIMM_SRC)/label.o
+libnvdimm-y += $(NVDIMM_SRC)/btt_devs.o
+
+obj-m += test/
diff --git a/tools/testing/nvdimm/Makefile b/tools/testing/nvdimm/Makefile
new file mode 100644
index 000000000000..3dfe024b4e7e
--- /dev/null
+++ b/tools/testing/nvdimm/Makefile
@@ -0,0 +1,7 @@
+KDIR ?= ../../../
+
+default:
+ $(MAKE) -C $(KDIR) M=$$PWD
+
+install: default
+ $(MAKE) -C $(KDIR) M=$$PWD modules_install
diff --git a/tools/testing/nvdimm/test/Kbuild b/tools/testing/nvdimm/test/Kbuild
new file mode 100644
index 000000000000..9241064970fe
--- /dev/null
+++ b/tools/testing/nvdimm/test/Kbuild
@@ -0,0 +1,8 @@
+ccflags-y := -I$(src)/../../../../drivers/nvdimm/
+ccflags-y += -I$(src)/../../../../drivers/acpi/
+
+obj-m += nfit_test.o
+obj-m += nfit_test_iomap.o
+
+nfit_test-y := nfit.o
+nfit_test_iomap-y := iomap.o
diff --git a/tools/testing/nvdimm/test/iomap.c b/tools/testing/nvdimm/test/iomap.c
new file mode 100644
index 000000000000..c85a6f6ba559
--- /dev/null
+++ b/tools/testing/nvdimm/test/iomap.c
@@ -0,0 +1,151 @@
+/*
+ * Copyright(c) 2013-2015 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+#include <linux/rculist.h>
+#include <linux/export.h>
+#include <linux/ioport.h>
+#include <linux/module.h>
+#include <linux/types.h>
+#include <linux/io.h>
+#include "nfit_test.h"
+
+static LIST_HEAD(iomap_head);
+
+static struct iomap_ops {
+ nfit_test_lookup_fn nfit_test_lookup;
+ struct list_head list;
+} iomap_ops = {
+ .list = LIST_HEAD_INIT(iomap_ops.list),
+};
+
+void nfit_test_setup(nfit_test_lookup_fn lookup)
+{
+ iomap_ops.nfit_test_lookup = lookup;
+ list_add_rcu(&iomap_ops.list, &iomap_head);
+}
+EXPORT_SYMBOL(nfit_test_setup);
+
+void nfit_test_teardown(void)
+{
+ list_del_rcu(&iomap_ops.list);
+ synchronize_rcu();
+}
+EXPORT_SYMBOL(nfit_test_teardown);
+
+static struct nfit_test_resource *get_nfit_res(resource_size_t resource)
+{
+ struct iomap_ops *ops;
+
+ ops = list_first_or_null_rcu(&iomap_head, typeof(*ops), list);
+ if (ops)
+ return ops->nfit_test_lookup(resource);
+ return NULL;
+}
+
+void __iomem *__nfit_test_ioremap(resource_size_t offset, unsigned long size,
+ void __iomem *(*fallback_fn)(resource_size_t, unsigned long))
+{
+ struct nfit_test_resource *nfit_res;
+
+ rcu_read_lock();
+ nfit_res = get_nfit_res(offset);
+ rcu_read_unlock();
+ if (nfit_res)
+ return (void __iomem *) nfit_res->buf + offset
+ - nfit_res->res->start;
+ return fallback_fn(offset, size);
+}
+
+void __iomem *__wrap_ioremap_cache(resource_size_t offset, unsigned long size)
+{
+ return __nfit_test_ioremap(offset, size, ioremap_cache);
+}
+EXPORT_SYMBOL(__wrap_ioremap_cache);
+
+void __iomem *__wrap_ioremap_nocache(resource_size_t offset, unsigned long size)
+{
+ return __nfit_test_ioremap(offset, size, ioremap_nocache);
+}
+EXPORT_SYMBOL(__wrap_ioremap_nocache);
+
+void __wrap_iounmap(volatile void __iomem *addr)
+{
+ struct nfit_test_resource *nfit_res;
+
+ rcu_read_lock();
+ nfit_res = get_nfit_res((unsigned long) addr);
+ rcu_read_unlock();
+ if (nfit_res)
+ return;
+ return iounmap(addr);
+}
+EXPORT_SYMBOL(__wrap_iounmap);
+
+struct resource *__wrap___request_region(struct resource *parent,
+ resource_size_t start, resource_size_t n, const char *name,
+ int flags)
+{
+ struct nfit_test_resource *nfit_res;
+
+ if (parent == &iomem_resource) {
+ rcu_read_lock();
+ nfit_res = get_nfit_res(start);
+ rcu_read_unlock();
+ if (nfit_res) {
+ struct resource *res = nfit_res->res + 1;
+
+ if (start + n > nfit_res->res->start
+ + resource_size(nfit_res->res)) {
+ pr_debug("%s: start: %llx n: %llx overflow: %pr\n",
+ __func__, start, n,
+ nfit_res->res);
+ return NULL;
+ }
+
+ res->start = start;
+ res->end = start + n - 1;
+ res->name = name;
+ res->flags = resource_type(parent);
+ res->flags |= IORESOURCE_BUSY | flags;
+ pr_debug("%s: %pr\n", __func__, res);
+ return res;
+ }
+ }
+ return __request_region(parent, start, n, name, flags);
+}
+EXPORT_SYMBOL(__wrap___request_region);
+
+void __wrap___release_region(struct resource *parent, resource_size_t start,
+ resource_size_t n)
+{
+ struct nfit_test_resource *nfit_res;
+
+ if (parent == &iomem_resource) {
+ rcu_read_lock();
+ nfit_res = get_nfit_res(start);
+ rcu_read_unlock();
+ if (nfit_res) {
+ struct resource *res = nfit_res->res + 1;
+
+ if (start != res->start || resource_size(res) != n)
+ pr_info("%s: start: %llx n: %llx mismatch: %pr\n",
+ __func__, start, n, res);
+ else
+ memset(res, 0, sizeof(*res));
+ return;
+ }
+ }
+ __release_region(parent, start, n);
+}
+EXPORT_SYMBOL(__wrap___release_region);
+
+MODULE_LICENSE("GPL v2");
diff --git a/tools/testing/nvdimm/test/nfit.c b/tools/testing/nvdimm/test/nfit.c
new file mode 100644
index 000000000000..a4078252138e
--- /dev/null
+++ b/tools/testing/nvdimm/test/nfit.c
@@ -0,0 +1,1101 @@
+/*
+ * Copyright(c) 2013-2015 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#include <linux/platform_device.h>
+#include <linux/dma-mapping.h>
+#include <linux/libnvdimm.h>
+#include <linux/device.h>
+#include <linux/module.h>
+#include <linux/ndctl.h>
+#include <linux/sizes.h>
+#include <linux/slab.h>
+#include <nfit.h>
+#include <nd.h>
+#include "nfit_test.h"
+
+/*
+ * Generate an NFIT table to describe the following topology:
+ *
+ * BUS0: Interleaved PMEM regions, and aliasing with BLK regions
+ *
+ * (a) (b) DIMM BLK-REGION
+ * +----------+--------------+----------+---------+
+ * +------+ | blk2.0 | pm0.0 | blk2.1 | pm1.0 | 0 region2
+ * | imc0 +--+- - - - - region0 - - - -+----------+ +
+ * +--+---+ | blk3.0 | pm0.0 | blk3.1 | pm1.0 | 1 region3
+ * | +----------+--------------v----------v v
+ * +--+---+ | |
+ * | cpu0 | region1
+ * +--+---+ | |
+ * | +-------------------------^----------^ ^
+ * +--+---+ | blk4.0 | pm1.0 | 2 region4
+ * | imc1 +--+-------------------------+----------+ +
+ * +------+ | blk5.0 | pm1.0 | 3 region5
+ * +-------------------------+----------+-+-------+
+ *
+ * *) In this layout we have four dimms and two memory controllers in one
+ * socket. Each unique interface (BLK or PMEM) to DPA space
+ * is identified by a region device with a dynamically assigned id.
+ *
+ * *) The first portion of dimm0 and dimm1 are interleaved as REGION0.
+ * A single PMEM namespace "pm0.0" is created using half of the
+ * REGION0 SPA-range. REGION0 spans dimm0 and dimm1. PMEM namespace
+ * allocate from from the bottom of a region. The unallocated
+ * portion of REGION0 aliases with REGION2 and REGION3. That
+ * unallacted capacity is reclaimed as BLK namespaces ("blk2.0" and
+ * "blk3.0") starting at the base of each DIMM to offset (a) in those
+ * DIMMs. "pm0.0", "blk2.0" and "blk3.0" are free-form readable
+ * names that can be assigned to a namespace.
+ *
+ * *) In the last portion of dimm0 and dimm1 we have an interleaved
+ * SPA range, REGION1, that spans those two dimms as well as dimm2
+ * and dimm3. Some of REGION1 allocated to a PMEM namespace named
+ * "pm1.0" the rest is reclaimed in 4 BLK namespaces (for each
+ * dimm in the interleave set), "blk2.1", "blk3.1", "blk4.0", and
+ * "blk5.0".
+ *
+ * *) The portion of dimm2 and dimm3 that do not participate in the
+ * REGION1 interleaved SPA range (i.e. the DPA address below offset
+ * (b) are also included in the "blk4.0" and "blk5.0" namespaces.
+ * Note, that BLK namespaces need not be contiguous in DPA-space, and
+ * can consume aliased capacity from multiple interleave sets.
+ *
+ * BUS1: Legacy NVDIMM (single contiguous range)
+ *
+ * region2
+ * +---------------------+
+ * |---------------------|
+ * || pm2.0 ||
+ * |---------------------|
+ * +---------------------+
+ *
+ * *) A NFIT-table may describe a simple system-physical-address range
+ * with no BLK aliasing. This type of region may optionally
+ * reference an NVDIMM.
+ */
+enum {
+ NUM_PM = 2,
+ NUM_DCR = 4,
+ NUM_BDW = NUM_DCR,
+ NUM_SPA = NUM_PM + NUM_DCR + NUM_BDW,
+ NUM_MEM = NUM_DCR + NUM_BDW + 2 /* spa0 iset */ + 4 /* spa1 iset */,
+ DIMM_SIZE = SZ_32M,
+ LABEL_SIZE = SZ_128K,
+ SPA0_SIZE = DIMM_SIZE,
+ SPA1_SIZE = DIMM_SIZE*2,
+ SPA2_SIZE = DIMM_SIZE,
+ BDW_SIZE = 64 << 8,
+ DCR_SIZE = 12,
+ NUM_NFITS = 2, /* permit testing multiple NFITs per system */
+};
+
+struct nfit_test_dcr {
+ __le64 bdw_addr;
+ __le32 bdw_status;
+ __u8 aperature[BDW_SIZE];
+};
+
+static u32 handle[NUM_DCR] = {
+ [0] = NFIT_DIMM_HANDLE(0, 0, 0, 0, 0),
+ [1] = NFIT_DIMM_HANDLE(0, 0, 0, 0, 1),
+ [2] = NFIT_DIMM_HANDLE(0, 0, 1, 0, 0),
+ [3] = NFIT_DIMM_HANDLE(0, 0, 1, 0, 1),
+};
+
+struct nfit_test {
+ struct acpi_nfit_desc acpi_desc;
+ struct platform_device pdev;
+ struct list_head resources;
+ void *nfit_buf;
+ dma_addr_t nfit_dma;
+ size_t nfit_size;
+ int num_dcr;
+ int num_pm;
+ void **dimm;
+ dma_addr_t *dimm_dma;
+ void **label;
+ dma_addr_t *label_dma;
+ void **spa_set;
+ dma_addr_t *spa_set_dma;
+ struct nfit_test_dcr **dcr;
+ dma_addr_t *dcr_dma;
+ int (*alloc)(struct nfit_test *t);
+ void (*setup)(struct nfit_test *t);
+};
+
+static struct nfit_test *to_nfit_test(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+
+ return container_of(pdev, struct nfit_test, pdev);
+}
+
+static int nfit_test_ctl(struct nvdimm_bus_descriptor *nd_desc,
+ struct nvdimm *nvdimm, unsigned int cmd, void *buf,
+ unsigned int buf_len)
+{
+ struct acpi_nfit_desc *acpi_desc = to_acpi_desc(nd_desc);
+ struct nfit_test *t = container_of(acpi_desc, typeof(*t), acpi_desc);
+ struct nfit_mem *nfit_mem = nvdimm_provider_data(nvdimm);
+ int i, rc;
+
+ if (!nfit_mem || !test_bit(cmd, &nfit_mem->dsm_mask))
+ return -ENXIO;
+
+ /* lookup label space for the given dimm */
+ for (i = 0; i < ARRAY_SIZE(handle); i++)
+ if (__to_nfit_memdev(nfit_mem)->device_handle == handle[i])
+ break;
+ if (i >= ARRAY_SIZE(handle))
+ return -ENXIO;
+
+ switch (cmd) {
+ case ND_CMD_GET_CONFIG_SIZE: {
+ struct nd_cmd_get_config_size *nd_cmd = buf;
+
+ if (buf_len < sizeof(*nd_cmd))
+ return -EINVAL;
+ nd_cmd->status = 0;
+ nd_cmd->config_size = LABEL_SIZE;
+ nd_cmd->max_xfer = SZ_4K;
+ rc = 0;
+ break;
+ }
+ case ND_CMD_GET_CONFIG_DATA: {
+ struct nd_cmd_get_config_data_hdr *nd_cmd = buf;
+ unsigned int len, offset = nd_cmd->in_offset;
+
+ if (buf_len < sizeof(*nd_cmd))
+ return -EINVAL;
+ if (offset >= LABEL_SIZE)
+ return -EINVAL;
+ if (nd_cmd->in_length + sizeof(*nd_cmd) > buf_len)
+ return -EINVAL;
+
+ nd_cmd->status = 0;
+ len = min(nd_cmd->in_length, LABEL_SIZE - offset);
+ memcpy(nd_cmd->out_buf, t->label[i] + offset, len);
+ rc = buf_len - sizeof(*nd_cmd) - len;
+ break;
+ }
+ case ND_CMD_SET_CONFIG_DATA: {
+ struct nd_cmd_set_config_hdr *nd_cmd = buf;
+ unsigned int len, offset = nd_cmd->in_offset;
+ u32 *status;
+
+ if (buf_len < sizeof(*nd_cmd))
+ return -EINVAL;
+ if (offset >= LABEL_SIZE)
+ return -EINVAL;
+ if (nd_cmd->in_length + sizeof(*nd_cmd) + 4 > buf_len)
+ return -EINVAL;
+
+ status = buf + nd_cmd->in_length + sizeof(*nd_cmd);
+ *status = 0;
+ len = min(nd_cmd->in_length, LABEL_SIZE - offset);
+ memcpy(t->label[i] + offset, nd_cmd->in_buf, len);
+ rc = buf_len - sizeof(*nd_cmd) - (len + 4);
+ break;
+ }
+ default:
+ return -ENOTTY;
+ }
+
+ return rc;
+}
+
+static DEFINE_SPINLOCK(nfit_test_lock);
+static struct nfit_test *instances[NUM_NFITS];
+
+static void release_nfit_res(void *data)
+{
+ struct nfit_test_resource *nfit_res = data;
+ struct resource *res = nfit_res->res;
+
+ spin_lock(&nfit_test_lock);
+ list_del(&nfit_res->list);
+ spin_unlock(&nfit_test_lock);
+
+ if (is_vmalloc_addr(nfit_res->buf))
+ vfree(nfit_res->buf);
+ else
+ dma_free_coherent(nfit_res->dev, resource_size(res),
+ nfit_res->buf, res->start);
+ kfree(res);
+ kfree(nfit_res);
+}
+
+static void *__test_alloc(struct nfit_test *t, size_t size, dma_addr_t *dma,
+ void *buf)
+{
+ struct device *dev = &t->pdev.dev;
+ struct resource *res = kzalloc(sizeof(*res) * 2, GFP_KERNEL);
+ struct nfit_test_resource *nfit_res = kzalloc(sizeof(*nfit_res),
+ GFP_KERNEL);
+ int rc;
+
+ if (!res || !buf || !nfit_res)
+ goto err;
+ rc = devm_add_action(dev, release_nfit_res, nfit_res);
+ if (rc)
+ goto err;
+ INIT_LIST_HEAD(&nfit_res->list);
+ memset(buf, 0, size);
+ nfit_res->dev = dev;
+ nfit_res->buf = buf;
+ nfit_res->res = res;
+ res->start = *dma;
+ res->end = *dma + size - 1;
+ res->name = "NFIT";
+ spin_lock(&nfit_test_lock);
+ list_add(&nfit_res->list, &t->resources);
+ spin_unlock(&nfit_test_lock);
+
+ return nfit_res->buf;
+ err:
+ if (buf && !is_vmalloc_addr(buf))
+ dma_free_coherent(dev, size, buf, *dma);
+ else if (buf)
+ vfree(buf);
+ kfree(res);
+ kfree(nfit_res);
+ return NULL;
+}
+
+static void *test_alloc(struct nfit_test *t, size_t size, dma_addr_t *dma)
+{
+ void *buf = vmalloc(size);
+
+ *dma = (unsigned long) buf;
+ return __test_alloc(t, size, dma, buf);
+}
+
+static void *test_alloc_coherent(struct nfit_test *t, size_t size, dma_addr_t *dma)
+{
+ struct device *dev = &t->pdev.dev;
+ void *buf = dma_alloc_coherent(dev, size, dma, GFP_KERNEL);
+
+ return __test_alloc(t, size, dma, buf);
+}
+
+static struct nfit_test_resource *nfit_test_lookup(resource_size_t addr)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(instances); i++) {
+ struct nfit_test_resource *n, *nfit_res = NULL;
+ struct nfit_test *t = instances[i];
+
+ if (!t)
+ continue;
+ spin_lock(&nfit_test_lock);
+ list_for_each_entry(n, &t->resources, list) {
+ if (addr >= n->res->start && (addr < n->res->start
+ + resource_size(n->res))) {
+ nfit_res = n;
+ break;
+ } else if (addr >= (unsigned long) n->buf
+ && (addr < (unsigned long) n->buf
+ + resource_size(n->res))) {
+ nfit_res = n;
+ break;
+ }
+ }
+ spin_unlock(&nfit_test_lock);
+ if (nfit_res)
+ return nfit_res;
+ }
+
+ return NULL;
+}
+
+static int nfit_test0_alloc(struct nfit_test *t)
+{
+ size_t nfit_size = sizeof(struct acpi_table_nfit)
+ + sizeof(struct acpi_nfit_system_address) * NUM_SPA
+ + sizeof(struct acpi_nfit_memory_map) * NUM_MEM
+ + sizeof(struct acpi_nfit_control_region) * NUM_DCR
+ + sizeof(struct acpi_nfit_data_region) * NUM_BDW;
+ int i;
+
+ t->nfit_buf = test_alloc(t, nfit_size, &t->nfit_dma);
+ if (!t->nfit_buf)
+ return -ENOMEM;
+ t->nfit_size = nfit_size;
+
+ t->spa_set[0] = test_alloc_coherent(t, SPA0_SIZE, &t->spa_set_dma[0]);
+ if (!t->spa_set[0])
+ return -ENOMEM;
+
+ t->spa_set[1] = test_alloc_coherent(t, SPA1_SIZE, &t->spa_set_dma[1]);
+ if (!t->spa_set[1])
+ return -ENOMEM;
+
+ for (i = 0; i < NUM_DCR; i++) {
+ t->dimm[i] = test_alloc(t, DIMM_SIZE, &t->dimm_dma[i]);
+ if (!t->dimm[i])
+ return -ENOMEM;
+
+ t->label[i] = test_alloc(t, LABEL_SIZE, &t->label_dma[i]);
+ if (!t->label[i])
+ return -ENOMEM;
+ sprintf(t->label[i], "label%d", i);
+ }
+
+ for (i = 0; i < NUM_DCR; i++) {
+ t->dcr[i] = test_alloc(t, LABEL_SIZE, &t->dcr_dma[i]);
+ if (!t->dcr[i])
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+static int nfit_test1_alloc(struct nfit_test *t)
+{
+ size_t nfit_size = sizeof(struct acpi_table_nfit)
+ + sizeof(struct acpi_nfit_system_address) + sizeof(struct acpi_nfit_memory_map)
+ + sizeof(struct acpi_nfit_control_region);
+
+ t->nfit_buf = test_alloc(t, nfit_size, &t->nfit_dma);
+ if (!t->nfit_buf)
+ return -ENOMEM;
+ t->nfit_size = nfit_size;
+
+ t->spa_set[0] = test_alloc_coherent(t, SPA2_SIZE, &t->spa_set_dma[0]);
+ if (!t->spa_set[0])
+ return -ENOMEM;
+
+ return 0;
+}
+
+static void nfit_test_init_header(struct acpi_table_nfit *nfit, size_t size)
+{
+ memcpy(nfit->header.signature, ACPI_SIG_NFIT, 4);
+ nfit->header.length = size;
+ nfit->header.revision = 1;
+ memcpy(nfit->header.oem_id, "LIBND", 6);
+ memcpy(nfit->header.oem_table_id, "TEST", 5);
+ nfit->header.oem_revision = 1;
+ memcpy(nfit->header.asl_compiler_id, "TST", 4);
+ nfit->header.asl_compiler_revision = 1;
+}
+
+static void nfit_test0_setup(struct nfit_test *t)
+{
+ struct nvdimm_bus_descriptor *nd_desc;
+ struct acpi_nfit_desc *acpi_desc;
+ struct acpi_nfit_memory_map *memdev;
+ void *nfit_buf = t->nfit_buf;
+ size_t size = t->nfit_size;
+ struct acpi_nfit_system_address *spa;
+ struct acpi_nfit_control_region *dcr;
+ struct acpi_nfit_data_region *bdw;
+ unsigned int offset;
+
+ nfit_test_init_header(nfit_buf, size);
+
+ /*
+ * spa0 (interleave first half of dimm0 and dimm1, note storage
+ * does not actually alias the related block-data-window
+ * regions)
+ */
+ spa = nfit_buf + sizeof(struct acpi_table_nfit);
+ spa->header.type = ACPI_NFIT_TYPE_SYSTEM_ADDRESS;
+ spa->header.length = sizeof(*spa);
+ memcpy(spa->range_guid, to_nfit_uuid(NFIT_SPA_PM), 16);
+ spa->range_index = 0+1;
+ spa->address = t->spa_set_dma[0];
+ spa->length = SPA0_SIZE;
+
+ /*
+ * spa1 (interleave last half of the 4 DIMMS, note storage
+ * does not actually alias the related block-data-window
+ * regions)
+ */
+ spa = nfit_buf + sizeof(struct acpi_table_nfit) + sizeof(*spa);
+ spa->header.type = ACPI_NFIT_TYPE_SYSTEM_ADDRESS;
+ spa->header.length = sizeof(*spa);
+ memcpy(spa->range_guid, to_nfit_uuid(NFIT_SPA_PM), 16);
+ spa->range_index = 1+1;
+ spa->address = t->spa_set_dma[1];
+ spa->length = SPA1_SIZE;
+
+ /* spa2 (dcr0) dimm0 */
+ spa = nfit_buf + sizeof(struct acpi_table_nfit) + sizeof(*spa) * 2;
+ spa->header.type = ACPI_NFIT_TYPE_SYSTEM_ADDRESS;
+ spa->header.length = sizeof(*spa);
+ memcpy(spa->range_guid, to_nfit_uuid(NFIT_SPA_DCR), 16);
+ spa->range_index = 2+1;
+ spa->address = t->dcr_dma[0];
+ spa->length = DCR_SIZE;
+
+ /* spa3 (dcr1) dimm1 */
+ spa = nfit_buf + sizeof(struct acpi_table_nfit) + sizeof(*spa) * 3;
+ spa->header.type = ACPI_NFIT_TYPE_SYSTEM_ADDRESS;
+ spa->header.length = sizeof(*spa);
+ memcpy(spa->range_guid, to_nfit_uuid(NFIT_SPA_DCR), 16);
+ spa->range_index = 3+1;
+ spa->address = t->dcr_dma[1];
+ spa->length = DCR_SIZE;
+
+ /* spa4 (dcr2) dimm2 */
+ spa = nfit_buf + sizeof(struct acpi_table_nfit) + sizeof(*spa) * 4;
+ spa->header.type = ACPI_NFIT_TYPE_SYSTEM_ADDRESS;
+ spa->header.length = sizeof(*spa);
+ memcpy(spa->range_guid, to_nfit_uuid(NFIT_SPA_DCR), 16);
+ spa->range_index = 4+1;
+ spa->address = t->dcr_dma[2];
+ spa->length = DCR_SIZE;
+
+ /* spa5 (dcr3) dimm3 */
+ spa = nfit_buf + sizeof(struct acpi_table_nfit) + sizeof(*spa) * 5;
+ spa->header.type = ACPI_NFIT_TYPE_SYSTEM_ADDRESS;
+ spa->header.length = sizeof(*spa);
+ memcpy(spa->range_guid, to_nfit_uuid(NFIT_SPA_DCR), 16);
+ spa->range_index = 5+1;
+ spa->address = t->dcr_dma[3];
+ spa->length = DCR_SIZE;
+
+ /* spa6 (bdw for dcr0) dimm0 */
+ spa = nfit_buf + sizeof(struct acpi_table_nfit) + sizeof(*spa) * 6;
+ spa->header.type = ACPI_NFIT_TYPE_SYSTEM_ADDRESS;
+ spa->header.length = sizeof(*spa);
+ memcpy(spa->range_guid, to_nfit_uuid(NFIT_SPA_BDW), 16);
+ spa->range_index = 6+1;
+ spa->address = t->dimm_dma[0];
+ spa->length = DIMM_SIZE;
+
+ /* spa7 (bdw for dcr1) dimm1 */
+ spa = nfit_buf + sizeof(struct acpi_table_nfit) + sizeof(*spa) * 7;
+ spa->header.type = ACPI_NFIT_TYPE_SYSTEM_ADDRESS;
+ spa->header.length = sizeof(*spa);
+ memcpy(spa->range_guid, to_nfit_uuid(NFIT_SPA_BDW), 16);
+ spa->range_index = 7+1;
+ spa->address = t->dimm_dma[1];
+ spa->length = DIMM_SIZE;
+
+ /* spa8 (bdw for dcr2) dimm2 */
+ spa = nfit_buf + sizeof(struct acpi_table_nfit) + sizeof(*spa) * 8;
+ spa->header.type = ACPI_NFIT_TYPE_SYSTEM_ADDRESS;
+ spa->header.length = sizeof(*spa);
+ memcpy(spa->range_guid, to_nfit_uuid(NFIT_SPA_BDW), 16);
+ spa->range_index = 8+1;
+ spa->address = t->dimm_dma[2];
+ spa->length = DIMM_SIZE;
+
+ /* spa9 (bdw for dcr3) dimm3 */
+ spa = nfit_buf + sizeof(struct acpi_table_nfit) + sizeof(*spa) * 9;
+ spa->header.type = ACPI_NFIT_TYPE_SYSTEM_ADDRESS;
+ spa->header.length = sizeof(*spa);
+ memcpy(spa->range_guid, to_nfit_uuid(NFIT_SPA_BDW), 16);
+ spa->range_index = 9+1;
+ spa->address = t->dimm_dma[3];
+ spa->length = DIMM_SIZE;
+
+ offset = sizeof(struct acpi_table_nfit) + sizeof(*spa) * 10;
+ /* mem-region0 (spa0, dimm0) */
+ memdev = nfit_buf + offset;
+ memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;
+ memdev->header.length = sizeof(*memdev);
+ memdev->device_handle = handle[0];
+ memdev->physical_id = 0;
+ memdev->region_id = 0;
+ memdev->range_index = 0+1;
+ memdev->region_index = 0+1;
+ memdev->region_size = SPA0_SIZE/2;
+ memdev->region_offset = t->spa_set_dma[0];
+ memdev->address = 0;
+ memdev->interleave_index = 0;
+ memdev->interleave_ways = 2;
+
+ /* mem-region1 (spa0, dimm1) */
+ memdev = nfit_buf + offset + sizeof(struct acpi_nfit_memory_map);
+ memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;
+ memdev->header.length = sizeof(*memdev);
+ memdev->device_handle = handle[1];
+ memdev->physical_id = 1;
+ memdev->region_id = 0;
+ memdev->range_index = 0+1;
+ memdev->region_index = 1+1;
+ memdev->region_size = SPA0_SIZE/2;
+ memdev->region_offset = t->spa_set_dma[0] + SPA0_SIZE/2;
+ memdev->address = 0;
+ memdev->interleave_index = 0;
+ memdev->interleave_ways = 2;
+
+ /* mem-region2 (spa1, dimm0) */
+ memdev = nfit_buf + offset + sizeof(struct acpi_nfit_memory_map) * 2;
+ memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;
+ memdev->header.length = sizeof(*memdev);
+ memdev->device_handle = handle[0];
+ memdev->physical_id = 0;
+ memdev->region_id = 1;
+ memdev->range_index = 1+1;
+ memdev->region_index = 0+1;
+ memdev->region_size = SPA1_SIZE/4;
+ memdev->region_offset = t->spa_set_dma[1];
+ memdev->address = SPA0_SIZE/2;
+ memdev->interleave_index = 0;
+ memdev->interleave_ways = 4;
+
+ /* mem-region3 (spa1, dimm1) */
+ memdev = nfit_buf + offset + sizeof(struct acpi_nfit_memory_map) * 3;
+ memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;
+ memdev->header.length = sizeof(*memdev);
+ memdev->device_handle = handle[1];
+ memdev->physical_id = 1;
+ memdev->region_id = 1;
+ memdev->range_index = 1+1;
+ memdev->region_index = 1+1;
+ memdev->region_size = SPA1_SIZE/4;
+ memdev->region_offset = t->spa_set_dma[1] + SPA1_SIZE/4;
+ memdev->address = SPA0_SIZE/2;
+ memdev->interleave_index = 0;
+ memdev->interleave_ways = 4;
+
+ /* mem-region4 (spa1, dimm2) */
+ memdev = nfit_buf + offset + sizeof(struct acpi_nfit_memory_map) * 4;
+ memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;
+ memdev->header.length = sizeof(*memdev);
+ memdev->device_handle = handle[2];
+ memdev->physical_id = 2;
+ memdev->region_id = 0;
+ memdev->range_index = 1+1;
+ memdev->region_index = 2+1;
+ memdev->region_size = SPA1_SIZE/4;
+ memdev->region_offset = t->spa_set_dma[1] + 2*SPA1_SIZE/4;
+ memdev->address = SPA0_SIZE/2;
+ memdev->interleave_index = 0;
+ memdev->interleave_ways = 4;
+
+ /* mem-region5 (spa1, dimm3) */
+ memdev = nfit_buf + offset + sizeof(struct acpi_nfit_memory_map) * 5;
+ memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;
+ memdev->header.length = sizeof(*memdev);
+ memdev->device_handle = handle[3];
+ memdev->physical_id = 3;
+ memdev->region_id = 0;
+ memdev->range_index = 1+1;
+ memdev->region_index = 3+1;
+ memdev->region_size = SPA1_SIZE/4;
+ memdev->region_offset = t->spa_set_dma[1] + 3*SPA1_SIZE/4;
+ memdev->address = SPA0_SIZE/2;
+ memdev->interleave_index = 0;
+ memdev->interleave_ways = 4;
+
+ /* mem-region6 (spa/dcr0, dimm0) */
+ memdev = nfit_buf + offset + sizeof(struct acpi_nfit_memory_map) * 6;
+ memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;
+ memdev->header.length = sizeof(*memdev);
+ memdev->device_handle = handle[0];
+ memdev->physical_id = 0;
+ memdev->region_id = 0;
+ memdev->range_index = 2+1;
+ memdev->region_index = 0+1;
+ memdev->region_size = 0;
+ memdev->region_offset = 0;
+ memdev->address = 0;
+ memdev->interleave_index = 0;
+ memdev->interleave_ways = 1;
+
+ /* mem-region7 (spa/dcr1, dimm1) */
+ memdev = nfit_buf + offset + sizeof(struct acpi_nfit_memory_map) * 7;
+ memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;
+ memdev->header.length = sizeof(*memdev);
+ memdev->device_handle = handle[1];
+ memdev->physical_id = 1;
+ memdev->region_id = 0;
+ memdev->range_index = 3+1;
+ memdev->region_index = 1+1;
+ memdev->region_size = 0;
+ memdev->region_offset = 0;
+ memdev->address = 0;
+ memdev->interleave_index = 0;
+ memdev->interleave_ways = 1;
+
+ /* mem-region8 (spa/dcr2, dimm2) */
+ memdev = nfit_buf + offset + sizeof(struct acpi_nfit_memory_map) * 8;
+ memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;
+ memdev->header.length = sizeof(*memdev);
+ memdev->device_handle = handle[2];
+ memdev->physical_id = 2;
+ memdev->region_id = 0;
+ memdev->range_index = 4+1;
+ memdev->region_index = 2+1;
+ memdev->region_size = 0;
+ memdev->region_offset = 0;
+ memdev->address = 0;
+ memdev->interleave_index = 0;
+ memdev->interleave_ways = 1;
+
+ /* mem-region9 (spa/dcr3, dimm3) */
+ memdev = nfit_buf + offset + sizeof(struct acpi_nfit_memory_map) * 9;
+ memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;
+ memdev->header.length = sizeof(*memdev);
+ memdev->device_handle = handle[3];
+ memdev->physical_id = 3;
+ memdev->region_id = 0;
+ memdev->range_index = 5+1;
+ memdev->region_index = 3+1;
+ memdev->region_size = 0;
+ memdev->region_offset = 0;
+ memdev->address = 0;
+ memdev->interleave_index = 0;
+ memdev->interleave_ways = 1;
+
+ /* mem-region10 (spa/bdw0, dimm0) */
+ memdev = nfit_buf + offset + sizeof(struct acpi_nfit_memory_map) * 10;
+ memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;
+ memdev->header.length = sizeof(*memdev);
+ memdev->device_handle = handle[0];
+ memdev->physical_id = 0;
+ memdev->region_id = 0;
+ memdev->range_index = 6+1;
+ memdev->region_index = 0+1;
+ memdev->region_size = 0;
+ memdev->region_offset = 0;
+ memdev->address = 0;
+ memdev->interleave_index = 0;
+ memdev->interleave_ways = 1;
+
+ /* mem-region11 (spa/bdw1, dimm1) */
+ memdev = nfit_buf + offset + sizeof(struct acpi_nfit_memory_map) * 11;
+ memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;
+ memdev->header.length = sizeof(*memdev);
+ memdev->device_handle = handle[1];
+ memdev->physical_id = 1;
+ memdev->region_id = 0;
+ memdev->range_index = 7+1;
+ memdev->region_index = 1+1;
+ memdev->region_size = 0;
+ memdev->region_offset = 0;
+ memdev->address = 0;
+ memdev->interleave_index = 0;
+ memdev->interleave_ways = 1;
+
+ /* mem-region12 (spa/bdw2, dimm2) */
+ memdev = nfit_buf + offset + sizeof(struct acpi_nfit_memory_map) * 12;
+ memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;
+ memdev->header.length = sizeof(*memdev);
+ memdev->device_handle = handle[2];
+ memdev->physical_id = 2;
+ memdev->region_id = 0;
+ memdev->range_index = 8+1;
+ memdev->region_index = 2+1;
+ memdev->region_size = 0;
+ memdev->region_offset = 0;
+ memdev->address = 0;
+ memdev->interleave_index = 0;
+ memdev->interleave_ways = 1;
+
+ /* mem-region13 (spa/dcr3, dimm3) */
+ memdev = nfit_buf + offset + sizeof(struct acpi_nfit_memory_map) * 13;
+ memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;
+ memdev->header.length = sizeof(*memdev);
+ memdev->device_handle = handle[3];
+ memdev->physical_id = 3;
+ memdev->region_id = 0;
+ memdev->range_index = 9+1;
+ memdev->region_index = 3+1;
+ memdev->region_size = 0;
+ memdev->region_offset = 0;
+ memdev->address = 0;
+ memdev->interleave_index = 0;
+ memdev->interleave_ways = 1;
+
+ offset = offset + sizeof(struct acpi_nfit_memory_map) * 14;
+ /* dcr-descriptor0 */
+ dcr = nfit_buf + offset;
+ dcr->header.type = ACPI_NFIT_TYPE_CONTROL_REGION;
+ dcr->header.length = sizeof(struct acpi_nfit_control_region);
+ dcr->region_index = 0+1;
+ dcr->vendor_id = 0xabcd;
+ dcr->device_id = 0;
+ dcr->revision_id = 1;
+ dcr->serial_number = ~handle[0];
+ dcr->windows = 1;
+ dcr->window_size = DCR_SIZE;
+ dcr->command_offset = 0;
+ dcr->command_size = 8;
+ dcr->status_offset = 8;
+ dcr->status_size = 4;
+
+ /* dcr-descriptor1 */
+ dcr = nfit_buf + offset + sizeof(struct acpi_nfit_control_region);
+ dcr->header.type = ACPI_NFIT_TYPE_CONTROL_REGION;
+ dcr->header.length = sizeof(struct acpi_nfit_control_region);
+ dcr->region_index = 1+1;
+ dcr->vendor_id = 0xabcd;
+ dcr->device_id = 0;
+ dcr->revision_id = 1;
+ dcr->serial_number = ~handle[1];
+ dcr->windows = 1;
+ dcr->window_size = DCR_SIZE;
+ dcr->command_offset = 0;
+ dcr->command_size = 8;
+ dcr->status_offset = 8;
+ dcr->status_size = 4;
+
+ /* dcr-descriptor2 */
+ dcr = nfit_buf + offset + sizeof(struct acpi_nfit_control_region) * 2;
+ dcr->header.type = ACPI_NFIT_TYPE_CONTROL_REGION;
+ dcr->header.length = sizeof(struct acpi_nfit_control_region);
+ dcr->region_index = 2+1;
+ dcr->vendor_id = 0xabcd;
+ dcr->device_id = 0;
+ dcr->revision_id = 1;
+ dcr->serial_number = ~handle[2];
+ dcr->windows = 1;
+ dcr->window_size = DCR_SIZE;
+ dcr->command_offset = 0;
+ dcr->command_size = 8;
+ dcr->status_offset = 8;
+ dcr->status_size = 4;
+
+ /* dcr-descriptor3 */
+ dcr = nfit_buf + offset + sizeof(struct acpi_nfit_control_region) * 3;
+ dcr->header.type = ACPI_NFIT_TYPE_CONTROL_REGION;
+ dcr->header.length = sizeof(struct acpi_nfit_control_region);
+ dcr->region_index = 3+1;
+ dcr->vendor_id = 0xabcd;
+ dcr->device_id = 0;
+ dcr->revision_id = 1;
+ dcr->serial_number = ~handle[3];
+ dcr->windows = 1;
+ dcr->window_size = DCR_SIZE;
+ dcr->command_offset = 0;
+ dcr->command_size = 8;
+ dcr->status_offset = 8;
+ dcr->status_size = 4;
+
+ offset = offset + sizeof(struct acpi_nfit_control_region) * 4;
+ /* bdw0 (spa/dcr0, dimm0) */
+ bdw = nfit_buf + offset;
+ bdw->header.type = ACPI_NFIT_TYPE_DATA_REGION;
+ bdw->header.length = sizeof(struct acpi_nfit_data_region);
+ bdw->region_index = 0+1;
+ bdw->windows = 1;
+ bdw->offset = 0;
+ bdw->size = BDW_SIZE;
+ bdw->capacity = DIMM_SIZE;
+ bdw->start_address = 0;
+
+ /* bdw1 (spa/dcr1, dimm1) */
+ bdw = nfit_buf + offset + sizeof(struct acpi_nfit_data_region);
+ bdw->header.type = ACPI_NFIT_TYPE_DATA_REGION;
+ bdw->header.length = sizeof(struct acpi_nfit_data_region);
+ bdw->region_index = 1+1;
+ bdw->windows = 1;
+ bdw->offset = 0;
+ bdw->size = BDW_SIZE;
+ bdw->capacity = DIMM_SIZE;
+ bdw->start_address = 0;
+
+ /* bdw2 (spa/dcr2, dimm2) */
+ bdw = nfit_buf + offset + sizeof(struct acpi_nfit_data_region) * 2;
+ bdw->header.type = ACPI_NFIT_TYPE_DATA_REGION;
+ bdw->header.length = sizeof(struct acpi_nfit_data_region);
+ bdw->region_index = 2+1;
+ bdw->windows = 1;
+ bdw->offset = 0;
+ bdw->size = BDW_SIZE;
+ bdw->capacity = DIMM_SIZE;
+ bdw->start_address = 0;
+
+ /* bdw3 (spa/dcr3, dimm3) */
+ bdw = nfit_buf + offset + sizeof(struct acpi_nfit_data_region) * 3;
+ bdw->header.type = ACPI_NFIT_TYPE_DATA_REGION;
+ bdw->header.length = sizeof(struct acpi_nfit_data_region);
+ bdw->region_index = 3+1;
+ bdw->windows = 1;
+ bdw->offset = 0;
+ bdw->size = BDW_SIZE;
+ bdw->capacity = DIMM_SIZE;
+ bdw->start_address = 0;
+
+ acpi_desc = &t->acpi_desc;
+ set_bit(ND_CMD_GET_CONFIG_SIZE, &acpi_desc->dimm_dsm_force_en);
+ set_bit(ND_CMD_GET_CONFIG_DATA, &acpi_desc->dimm_dsm_force_en);
+ set_bit(ND_CMD_SET_CONFIG_DATA, &acpi_desc->dimm_dsm_force_en);
+ nd_desc = &acpi_desc->nd_desc;
+ nd_desc->ndctl = nfit_test_ctl;
+}
+
+static void nfit_test1_setup(struct nfit_test *t)
+{
+ size_t size = t->nfit_size, offset;
+ void *nfit_buf = t->nfit_buf;
+ struct acpi_nfit_memory_map *memdev;
+ struct acpi_nfit_control_region *dcr;
+ struct acpi_nfit_system_address *spa;
+
+ nfit_test_init_header(nfit_buf, size);
+
+ offset = sizeof(struct acpi_table_nfit);
+ /* spa0 (flat range with no bdw aliasing) */
+ spa = nfit_buf + offset;
+ spa->header.type = ACPI_NFIT_TYPE_SYSTEM_ADDRESS;
+ spa->header.length = sizeof(*spa);
+ memcpy(spa->range_guid, to_nfit_uuid(NFIT_SPA_PM), 16);
+ spa->range_index = 0+1;
+ spa->address = t->spa_set_dma[0];
+ spa->length = SPA2_SIZE;
+
+ offset += sizeof(*spa);
+ /* mem-region0 (spa0, dimm0) */
+ memdev = nfit_buf + offset;
+ memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;
+ memdev->header.length = sizeof(*memdev);
+ memdev->device_handle = 0;
+ memdev->physical_id = 0;
+ memdev->region_id = 0;
+ memdev->range_index = 0+1;
+ memdev->region_index = 0+1;
+ memdev->region_size = SPA2_SIZE;
+ memdev->region_offset = 0;
+ memdev->address = 0;
+ memdev->interleave_index = 0;
+ memdev->interleave_ways = 1;
+
+ offset += sizeof(*memdev);
+ /* dcr-descriptor0 */
+ dcr = nfit_buf + offset;
+ dcr->header.type = ACPI_NFIT_TYPE_CONTROL_REGION;
+ dcr->header.length = sizeof(struct acpi_nfit_control_region);
+ dcr->region_index = 0+1;
+ dcr->vendor_id = 0xabcd;
+ dcr->device_id = 0;
+ dcr->revision_id = 1;
+ dcr->serial_number = ~0;
+ dcr->code = 0x201;
+ dcr->windows = 0;
+ dcr->window_size = 0;
+ dcr->command_offset = 0;
+ dcr->command_size = 0;
+ dcr->status_offset = 0;
+ dcr->status_size = 0;
+}
+
+static int nfit_test_blk_do_io(struct nd_blk_region *ndbr, void *iobuf,
+ u64 len, int rw, resource_size_t dpa)
+{
+ struct nfit_blk *nfit_blk = ndbr->blk_provider_data;
+ struct nfit_blk_mmio *mmio = &nfit_blk->mmio[BDW];
+ struct nd_region *nd_region = &ndbr->nd_region;
+ unsigned int bw;
+
+ bw = nd_region_acquire_lane(nd_region);
+ if (rw)
+ memcpy(mmio->base + dpa, iobuf, len);
+ else
+ memcpy(iobuf, mmio->base + dpa, len);
+ nd_region_release_lane(nd_region, bw);
+
+ return 0;
+}
+
+extern const struct attribute_group *acpi_nfit_attribute_groups[];
+
+static int nfit_test_probe(struct platform_device *pdev)
+{
+ struct nvdimm_bus_descriptor *nd_desc;
+ struct acpi_nfit_desc *acpi_desc;
+ struct device *dev = &pdev->dev;
+ struct nfit_test *nfit_test;
+ int rc;
+
+ nfit_test = to_nfit_test(&pdev->dev);
+
+ /* common alloc */
+ if (nfit_test->num_dcr) {
+ int num = nfit_test->num_dcr;
+
+ nfit_test->dimm = devm_kcalloc(dev, num, sizeof(void *), GFP_KERNEL);
+ nfit_test->dimm_dma = devm_kcalloc(dev, num, sizeof(dma_addr_t), GFP_KERNEL);
+ nfit_test->label = devm_kcalloc(dev, num, sizeof(void *), GFP_KERNEL);
+ nfit_test->label_dma = devm_kcalloc(dev, num, sizeof(dma_addr_t), GFP_KERNEL);
+ nfit_test->dcr = devm_kcalloc(dev, num, sizeof(struct nfit_test_dcr *), GFP_KERNEL);
+ nfit_test->dcr_dma = devm_kcalloc(dev, num, sizeof(dma_addr_t), GFP_KERNEL);
+ if (nfit_test->dimm && nfit_test->dimm_dma && nfit_test->label
+ && nfit_test->label_dma && nfit_test->dcr
+ && nfit_test->dcr_dma)
+ /* pass */;
+ else
+ return -ENOMEM;
+ }
+
+ if (nfit_test->num_pm) {
+ int num = nfit_test->num_pm;
+
+ nfit_test->spa_set = devm_kcalloc(dev, num, sizeof(void *), GFP_KERNEL);
+ nfit_test->spa_set_dma = devm_kcalloc(dev, num,
+ sizeof(dma_addr_t), GFP_KERNEL);
+ if (nfit_test->spa_set && nfit_test->spa_set_dma)
+ /* pass */;
+ else
+ return -ENOMEM;
+ }
+
+ /* per-nfit specific alloc */
+ if (nfit_test->alloc(nfit_test))
+ return -ENOMEM;
+
+ nfit_test->setup(nfit_test);
+ acpi_desc = &nfit_test->acpi_desc;
+ acpi_desc->dev = &pdev->dev;
+ acpi_desc->nfit = nfit_test->nfit_buf;
+ acpi_desc->blk_do_io = nfit_test_blk_do_io;
+ nd_desc = &acpi_desc->nd_desc;
+ nd_desc->attr_groups = acpi_nfit_attribute_groups;
+ acpi_desc->nvdimm_bus = nvdimm_bus_register(&pdev->dev, nd_desc);
+ if (!acpi_desc->nvdimm_bus)
+ return -ENXIO;
+
+ rc = acpi_nfit_init(acpi_desc, nfit_test->nfit_size);
+ if (rc) {
+ nvdimm_bus_unregister(acpi_desc->nvdimm_bus);
+ return rc;
+ }
+
+ return 0;
+}
+
+static int nfit_test_remove(struct platform_device *pdev)
+{
+ struct nfit_test *nfit_test = to_nfit_test(&pdev->dev);
+ struct acpi_nfit_desc *acpi_desc = &nfit_test->acpi_desc;
+
+ nvdimm_bus_unregister(acpi_desc->nvdimm_bus);
+
+ return 0;
+}
+
+static void nfit_test_release(struct device *dev)
+{
+ struct nfit_test *nfit_test = to_nfit_test(dev);
+
+ kfree(nfit_test);
+}
+
+static const struct platform_device_id nfit_test_id[] = {
+ { KBUILD_MODNAME },
+ { },
+};
+
+static struct platform_driver nfit_test_driver = {
+ .probe = nfit_test_probe,
+ .remove = nfit_test_remove,
+ .driver = {
+ .name = KBUILD_MODNAME,
+ },
+ .id_table = nfit_test_id,
+};
+
+#ifdef CONFIG_CMA_SIZE_MBYTES
+#define CMA_SIZE_MBYTES CONFIG_CMA_SIZE_MBYTES
+#else
+#define CMA_SIZE_MBYTES 0
+#endif
+
+static __init int nfit_test_init(void)
+{
+ int rc, i;
+
+ nfit_test_setup(nfit_test_lookup);
+
+ for (i = 0; i < NUM_NFITS; i++) {
+ struct nfit_test *nfit_test;
+ struct platform_device *pdev;
+ static int once;
+
+ nfit_test = kzalloc(sizeof(*nfit_test), GFP_KERNEL);
+ if (!nfit_test) {
+ rc = -ENOMEM;
+ goto err_register;
+ }
+ INIT_LIST_HEAD(&nfit_test->resources);
+ switch (i) {
+ case 0:
+ nfit_test->num_pm = NUM_PM;
+ nfit_test->num_dcr = NUM_DCR;
+ nfit_test->alloc = nfit_test0_alloc;
+ nfit_test->setup = nfit_test0_setup;
+ break;
+ case 1:
+ nfit_test->num_pm = 1;
+ nfit_test->alloc = nfit_test1_alloc;
+ nfit_test->setup = nfit_test1_setup;
+ break;
+ default:
+ rc = -EINVAL;
+ goto err_register;
+ }
+ pdev = &nfit_test->pdev;
+ pdev->name = KBUILD_MODNAME;
+ pdev->id = i;
+ pdev->dev.release = nfit_test_release;
+ rc = platform_device_register(pdev);
+ if (rc) {
+ put_device(&pdev->dev);
+ goto err_register;
+ }
+
+ rc = dma_coerce_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64));
+ if (rc)
+ goto err_register;
+
+ instances[i] = nfit_test;
+
+ if (!once++) {
+ dma_addr_t dma;
+ void *buf;
+
+ buf = dma_alloc_coherent(&pdev->dev, SZ_128M, &dma,
+ GFP_KERNEL);
+ if (!buf) {
+ rc = -ENOMEM;
+ dev_warn(&pdev->dev, "need 128M of free cma\n");
+ goto err_register;
+ }
+ dma_free_coherent(&pdev->dev, SZ_128M, buf, dma);
+ }
+ }
+
+ rc = platform_driver_register(&nfit_test_driver);
+ if (rc)
+ goto err_register;
+ return 0;
+
+ err_register:
+ for (i = 0; i < NUM_NFITS; i++)
+ if (instances[i])
+ platform_device_unregister(&instances[i]->pdev);
+ nfit_test_teardown();
+ return rc;
+}
+
+static __exit void nfit_test_exit(void)
+{
+ int i;
+
+ platform_driver_unregister(&nfit_test_driver);
+ for (i = 0; i < NUM_NFITS; i++)
+ platform_device_unregister(&instances[i]->pdev);
+ nfit_test_teardown();
+}
+
+module_init(nfit_test_init);
+module_exit(nfit_test_exit);
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Intel Corporation");
diff --git a/tools/testing/nvdimm/test/nfit_test.h b/tools/testing/nvdimm/test/nfit_test.h
new file mode 100644
index 000000000000..4a1215ec45c0
--- /dev/null
+++ b/tools/testing/nvdimm/test/nfit_test.h
@@ -0,0 +1,28 @@
+/*
+ * Copyright(c) 2013-2015 Intel Corporation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ */
+#ifndef __NFIT_TEST_H__
+#define __NFIT_TEST_H__
+
+struct nfit_test_resource {
+ struct list_head list;
+ struct resource *res;
+ struct device *dev;
+ void *buf;
+};
+
+typedef struct nfit_test_resource *(*nfit_test_lookup_fn)(resource_size_t);
+void __iomem *__wrap_ioremap_nocache(resource_size_t offset, unsigned long size);
+void __wrap_iounmap(volatile void __iomem *addr);
+void nfit_test_setup(nfit_test_lookup_fn lookup);
+void nfit_test_teardown(void);
+#endif
^ permalink raw reply related
* [PATCH v5 19/21] libnvdimm, nfit, nd_blk: driver for BLK-mode access persistent memory
From: Dan Williams @ 2015-06-02 0:15 UTC (permalink / raw)
To: linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw
Cc: axboe-tSWWG44O7X1aa/9Udqfwiw, sfr-3FnU+UHB4dNDw9hX6IcOSA,
rafael-DgEjT+Ai2ygdnm+yROfE0A, neilb-l3A5Bk7waGM,
gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r,
linux-kernel-u79uwXL29TY76Z2rM5mHXA, mingo-DgEjT+Ai2ygdnm+yROfE0A,
linux-acpi-u79uwXL29TY76Z2rM5mHXA, jmoyer-H+wXaHxf7aLQT0dZR+AlfA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b, hch-jcswGhMUV9g
In-Reply-To: <20150602001134.4506.45867.stgit-p8uTFz9XbKj2zm6wflaqv1nYeNYlB/vhral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
From: Ross Zwisler <ross.zwisler-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
The libnvdimm implementation handles allocating dimm address space (DPA)
between PMEM and BLK mode interfaces. After DPA has been allocated from
a BLK-region to a BLK-namespace the nd_blk driver attaches to handle I/O
as a struct bio based block device. Unlike PMEM, BLK is required to
handle platform specific details like mmio register formats and memory
controller interleave. For this reason the libnvdimm generic nd_blk
driver calls back into the bus provider to carry out the I/O.
This initial implementation handles the BLK interface defined by the
ACPI 6 NFIT [1] and the NVDIMM DSM Interface Example [2] composed from
DCR (dimm control region), BDW (block data window), IDT (interleave
descriptor) NFIT structures and the hardware register format.
[1]: http://www.uefi.org/sites/default/files/resources/ACPI_6.0.pdf
[2]: http://pmem.io/documents/NVDIMM_DSM_Interface_Example.pdf
Cc: Andy Lutomirski <luto-kltTT9wpgjJwATOyAt5JVQ@public.gmane.org>
Cc: Boaz Harrosh <boaz-/8YdC2HfS5554TAoqtyWWQ@public.gmane.org>
Cc: H. Peter Anvin <hpa-YMNOUZJC4hwAvxtiuMwx3w@public.gmane.org>
Cc: Jens Axboe <axboe-b10kYP2dOMg@public.gmane.org>
Cc: Ingo Molnar <mingo-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Cc: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
Signed-off-by: Ross Zwisler <ross.zwisler-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
Signed-off-by: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/acpi/nfit.c | 441 +++++++++++++++++++++++++++++++++++++--
drivers/acpi/nfit.h | 49 ++++
drivers/nvdimm/Kconfig | 13 +
drivers/nvdimm/Makefile | 3
drivers/nvdimm/blk.c | 252 ++++++++++++++++++++++
drivers/nvdimm/dimm_devs.c | 9 +
drivers/nvdimm/namespace_devs.c | 47 ++++
drivers/nvdimm/nd-private.h | 3
drivers/nvdimm/nd.h | 13 +
drivers/nvdimm/region.c | 8 +
drivers/nvdimm/region_devs.c | 90 +++++++-
include/linux/libnvdimm.h | 19 ++
12 files changed, 908 insertions(+), 39 deletions(-)
create mode 100644 drivers/nvdimm/blk.c
diff --git a/drivers/acpi/nfit.c b/drivers/acpi/nfit.c
index e027f22014d2..1124c47f7b06 100644
--- a/drivers/acpi/nfit.c
+++ b/drivers/acpi/nfit.c
@@ -13,12 +13,16 @@
#include <linux/list_sort.h>
#include <linux/libnvdimm.h>
#include <linux/module.h>
+#include <linux/mutex.h>
#include <linux/ndctl.h>
#include <linux/list.h>
#include <linux/acpi.h>
#include <linux/sort.h>
+#include <linux/io.h>
#include "nfit.h"
+#include <asm-generic/io-64-nonatomic-hi-lo.h>
+
static bool force_enable_dimms;
module_param(force_enable_dimms, bool, S_IRUGO|S_IWUSR);
MODULE_PARM_DESC(force_enable_dimms, "Ignore _STA (ACPI DIMM device) status");
@@ -71,7 +75,7 @@ static int acpi_nfit_ctl(struct nvdimm_bus_descriptor *nd_desc,
if (!adev)
return -ENOTTY;
- dimm_name = dev_name(&adev->dev);
+ dimm_name = nvdimm_name(nvdimm);
cmd_name = nvdimm_cmd_name(cmd);
dsm_mask = nfit_mem->dsm_mask;
desc = nd_cmd_dimm_desc(cmd);
@@ -266,10 +270,20 @@ static void *add_table(struct acpi_nfit_desc *acpi_desc, void *table, const void
bdw->region_index, bdw->windows);
break;
}
- /* TODO */
- case ACPI_NFIT_TYPE_INTERLEAVE:
- dev_dbg(dev, "%s: idt\n", __func__);
+ case ACPI_NFIT_TYPE_INTERLEAVE: {
+ struct nfit_idt *nfit_idt = devm_kzalloc(dev, sizeof(*nfit_idt),
+ GFP_KERNEL);
+ struct acpi_nfit_interleave *idt = table;
+
+ if (!nfit_idt)
+ return err;
+ INIT_LIST_HEAD(&nfit_idt->list);
+ nfit_idt->idt = idt;
+ list_add_tail(&nfit_idt->list, &acpi_desc->idts);
+ dev_dbg(dev, "%s: idt index: %d num_lines: %d\n", __func__,
+ idt->interleave_index, idt->line_count);
break;
+ }
case ACPI_NFIT_TYPE_FLUSH_ADDRESS:
dev_dbg(dev, "%s: flush\n", __func__);
break;
@@ -321,8 +335,11 @@ static int nfit_mem_add(struct acpi_nfit_desc *acpi_desc,
struct nfit_mem *nfit_mem, struct acpi_nfit_system_address *spa)
{
u16 dcr_index = __to_nfit_memdev(nfit_mem)->region_index;
+ struct nfit_memdev *nfit_memdev;
struct nfit_dcr *nfit_dcr;
struct nfit_bdw *nfit_bdw;
+ struct nfit_idt *nfit_idt;
+ u16 idt_index, range_index;
list_for_each_entry(nfit_dcr, &acpi_desc->dcrs, list) {
if (nfit_dcr->dcr->region_index != dcr_index)
@@ -355,6 +372,26 @@ static int nfit_mem_add(struct acpi_nfit_desc *acpi_desc,
return 0;
nfit_mem_find_spa_bdw(acpi_desc, nfit_mem);
+
+ if (!nfit_mem->spa_bdw)
+ return 0;
+
+ range_index = nfit_mem->spa_bdw->range_index;
+ list_for_each_entry(nfit_memdev, &acpi_desc->memdevs, list) {
+ if (nfit_memdev->memdev->range_index != range_index ||
+ nfit_memdev->memdev->region_index != dcr_index)
+ continue;
+ nfit_mem->memdev_bdw = nfit_memdev->memdev;
+ idt_index = nfit_memdev->memdev->interleave_index;
+ list_for_each_entry(nfit_idt, &acpi_desc->idts, list) {
+ if (nfit_idt->idt->interleave_index != idt_index)
+ continue;
+ nfit_mem->idt_bdw = nfit_idt->idt;
+ break;
+ }
+ break;
+ }
+
return 0;
}
@@ -398,9 +435,19 @@ static int nfit_mem_dcr_init(struct acpi_nfit_desc *acpi_desc,
}
if (type == NFIT_SPA_DCR) {
+ struct nfit_idt *nfit_idt;
+ u16 idt_index;
+
/* multiple dimms may share a SPA when interleaved */
nfit_mem->spa_dcr = spa;
nfit_mem->memdev_dcr = nfit_memdev->memdev;
+ idt_index = nfit_memdev->memdev->interleave_index;
+ list_for_each_entry(nfit_idt, &acpi_desc->idts, list) {
+ if (nfit_idt->idt->interleave_index != idt_index)
+ continue;
+ nfit_mem->idt_dcr = nfit_idt->idt;
+ break;
+ }
} else {
/*
* A single dimm may belong to multiple SPA-PM
@@ -829,13 +876,362 @@ static int acpi_nfit_init_interleave_set(struct acpi_nfit_desc *acpi_desc,
return 0;
}
+static u64 to_interleave_offset(u64 offset, struct nfit_blk_mmio *mmio)
+{
+ struct acpi_nfit_interleave *idt = mmio->idt;
+ u32 sub_line_offset, line_index, line_offset;
+ u64 line_no, table_skip_count, table_offset;
+
+ line_no = div_u64_rem(offset, mmio->line_size, &sub_line_offset);
+ table_skip_count = div_u64_rem(line_no, mmio->num_lines, &line_index);
+ line_offset = idt->line_offset[line_index]
+ * mmio->line_size;
+ table_offset = table_skip_count * mmio->table_size;
+
+ return mmio->base_offset + line_offset + table_offset + sub_line_offset;
+}
+
+static u64 read_blk_stat(struct nfit_blk *nfit_blk, unsigned int bw)
+{
+ struct nfit_blk_mmio *mmio = &nfit_blk->mmio[DCR];
+ u64 offset = nfit_blk->stat_offset + mmio->size * bw;
+
+ if (mmio->num_lines)
+ offset = to_interleave_offset(offset, mmio);
+
+ return readq(mmio->base + offset);
+}
+
+static void write_blk_ctl(struct nfit_blk *nfit_blk, unsigned int bw,
+ resource_size_t dpa, unsigned int len, unsigned int write)
+{
+ u64 cmd, offset;
+ struct nfit_blk_mmio *mmio = &nfit_blk->mmio[DCR];
+
+ enum {
+ BCW_OFFSET_MASK = (1ULL << 48)-1,
+ BCW_LEN_SHIFT = 48,
+ BCW_LEN_MASK = (1ULL << 8) - 1,
+ BCW_CMD_SHIFT = 56,
+ };
+
+ cmd = (dpa >> L1_CACHE_SHIFT) & BCW_OFFSET_MASK;
+ len = len >> L1_CACHE_SHIFT;
+ cmd |= ((u64) len & BCW_LEN_MASK) << BCW_LEN_SHIFT;
+ cmd |= ((u64) write) << BCW_CMD_SHIFT;
+
+ offset = nfit_blk->cmd_offset + mmio->size * bw;
+ if (mmio->num_lines)
+ offset = to_interleave_offset(offset, mmio);
+
+ writeq(cmd, mmio->base + offset);
+ /* FIXME: conditionally perform read-back if mandated by firmware */
+}
+
+static int acpi_nfit_blk_single_io(struct nfit_blk *nfit_blk, void *iobuf,
+ unsigned int len, int write, resource_size_t dpa,
+ unsigned int bw)
+{
+ struct nfit_blk_mmio *mmio = &nfit_blk->mmio[BDW];
+ unsigned int copied = 0;
+ u64 base_offset;
+ int rc;
+
+ base_offset = nfit_blk->bdw_offset + dpa % L1_CACHE_BYTES + bw * mmio->size;
+ /* TODO: non-temporal access, flush hints, cache management etc... */
+ write_blk_ctl(nfit_blk, bw, dpa, len, write);
+ while (len) {
+ unsigned int c;
+ u64 offset;
+
+ if (mmio->num_lines) {
+ u32 line_offset;
+
+ offset = to_interleave_offset(base_offset + copied,
+ mmio);
+ div_u64_rem(offset, mmio->line_size, &line_offset);
+ c = min(len, mmio->line_size - line_offset);
+ } else {
+ offset = base_offset + nfit_blk->bdw_offset;
+ c = len;
+ }
+
+ if (write)
+ memcpy(mmio->aperture + offset, iobuf + copied, c);
+ else
+ memcpy(iobuf + copied, mmio->aperture + offset, c);
+
+ copied += c;
+ len -= c;
+ }
+ rc = read_blk_stat(nfit_blk, bw) ? -EIO : 0;
+ return rc;
+}
+
+static int acpi_nfit_blk_region_do_io(struct nd_blk_region *ndbr, void *iobuf,
+ u64 len, int write, resource_size_t dpa)
+{
+ struct nfit_blk *nfit_blk = nd_blk_region_provider_data(ndbr);
+ struct nfit_blk_mmio *mmio = &nfit_blk->mmio[BDW];
+ struct nd_region *nd_region = nfit_blk->nd_region;
+ unsigned int bw, copied = 0;
+ int rc = 0;
+
+ bw = nd_region_acquire_lane(nd_region);
+ while (len) {
+ u64 c = min(len, mmio->size);
+
+ rc = acpi_nfit_blk_single_io(nfit_blk, iobuf + copied, c, write,
+ dpa + copied, bw);
+ if (rc)
+ break;
+
+ copied += c;
+ len -= c;
+ }
+ nd_region_release_lane(nd_region, bw);
+
+ return rc;
+}
+
+static void nfit_spa_mapping_release(struct kref *kref)
+{
+ struct nfit_spa_mapping *spa_map = to_spa_map(kref);
+ struct acpi_nfit_system_address *spa = spa_map->spa;
+ struct acpi_nfit_desc *acpi_desc = spa_map->acpi_desc;
+
+ WARN_ON(!mutex_is_locked(&acpi_desc->spa_map_mutex));
+ dev_dbg(acpi_desc->dev, "%s: SPA%d\n", __func__, spa->range_index);
+ iounmap(spa_map->iomem);
+ release_mem_region(spa->address, spa->length);
+ list_del(&spa_map->list);
+ kfree(spa_map);
+}
+
+static struct nfit_spa_mapping *find_spa_mapping(struct acpi_nfit_desc *acpi_desc,
+ struct acpi_nfit_system_address *spa)
+{
+ struct nfit_spa_mapping *spa_map;
+
+ WARN_ON(!mutex_is_locked(&acpi_desc->spa_map_mutex));
+ list_for_each_entry(spa_map, &acpi_desc->spa_maps, list)
+ if (spa_map->spa == spa)
+ return spa_map;
+
+ return NULL;
+}
+
+static void nfit_spa_unmap(struct acpi_nfit_desc *acpi_desc,
+ struct acpi_nfit_system_address *spa)
+{
+ struct nfit_spa_mapping *spa_map;
+
+ mutex_lock(&acpi_desc->spa_map_mutex);
+ spa_map = find_spa_mapping(acpi_desc, spa);
+
+ if (spa_map)
+ kref_put(&spa_map->kref, nfit_spa_mapping_release);
+ mutex_unlock(&acpi_desc->spa_map_mutex);
+}
+
+static void __iomem *__nfit_spa_map(struct acpi_nfit_desc *acpi_desc,
+ struct acpi_nfit_system_address *spa)
+{
+ resource_size_t start = spa->address;
+ resource_size_t n = spa->length;
+ struct nfit_spa_mapping *spa_map;
+ struct resource *res;
+
+ WARN_ON(!mutex_is_locked(&acpi_desc->spa_map_mutex));
+
+ spa_map = find_spa_mapping(acpi_desc, spa);
+ if (spa_map) {
+ kref_get(&spa_map->kref);
+ return spa_map->iomem;
+ }
+
+ spa_map = kzalloc(sizeof(*spa_map), GFP_KERNEL);
+ if (!spa_map)
+ return NULL;
+
+ INIT_LIST_HEAD(&spa_map->list);
+ spa_map->spa = spa;
+ kref_init(&spa_map->kref);
+ spa_map->acpi_desc = acpi_desc;
+
+ res = request_mem_region(start, n, dev_name(acpi_desc->dev));
+ if (!res)
+ goto err_mem;
+
+ /* TODO: cacheability based on the spa type */
+ spa_map->iomem = ioremap_nocache(start, n);
+ if (!spa_map->iomem)
+ goto err_map;
+
+ list_add_tail(&spa_map->list, &acpi_desc->spa_maps);
+ return spa_map->iomem;
+
+ err_map:
+ release_mem_region(start, n);
+ err_mem:
+ kfree(spa_map);
+ return NULL;
+}
+
+/**
+ * nfit_spa_map - interleave-aware managed-mappings of acpi_nfit_system_address ranges
+ * @nvdimm_bus: NFIT-bus that provided the spa table entry
+ * @nfit_spa: spa table to map
+ *
+ * In the case where block-data-window apertures and
+ * dimm-control-regions are interleaved they will end up sharing a
+ * single request_mem_region() + ioremap() for the address range. In
+ * the style of devm nfit_spa_map() mappings are automatically dropped
+ * when all region devices referencing the same mapping are disabled /
+ * unbound.
+ */
+static void __iomem *nfit_spa_map(struct acpi_nfit_desc *acpi_desc,
+ struct acpi_nfit_system_address *spa)
+{
+ void __iomem *iomem;
+
+ mutex_lock(&acpi_desc->spa_map_mutex);
+ iomem = __nfit_spa_map(acpi_desc, spa);
+ mutex_unlock(&acpi_desc->spa_map_mutex);
+
+ return iomem;
+}
+
+static int nfit_blk_init_interleave(struct nfit_blk_mmio *mmio,
+ struct acpi_nfit_interleave *idt, u16 interleave_ways)
+{
+ if (idt) {
+ mmio->num_lines = idt->line_count;
+ mmio->line_size = idt->line_size;
+ if (interleave_ways == 0)
+ return -ENXIO;
+ mmio->table_size = mmio->num_lines * interleave_ways
+ * mmio->line_size;
+ }
+
+ return 0;
+}
+
+static int acpi_nfit_blk_region_enable(struct nvdimm_bus *nvdimm_bus, struct device *dev)
+{
+ struct nvdimm_bus_descriptor *nd_desc = to_nd_desc(nvdimm_bus);
+ struct acpi_nfit_desc *acpi_desc = to_acpi_desc(nd_desc);
+ struct nd_blk_region *ndbr = to_nd_blk_region(dev);
+ struct nfit_blk_mmio *mmio;
+ struct nfit_blk *nfit_blk;
+ struct nfit_mem *nfit_mem;
+ struct nvdimm *nvdimm;
+ int rc;
+
+ nvdimm = nd_blk_region_to_dimm(ndbr);
+ nfit_mem = nvdimm_provider_data(nvdimm);
+ if (!nfit_mem || !nfit_mem->dcr || !nfit_mem->bdw) {
+ dev_dbg(dev, "%s: missing%s%s%s\n", __func__,
+ nfit_mem ? "" : " nfit_mem",
+ nfit_mem->dcr ? "" : " dcr",
+ nfit_mem->bdw ? "" : " bdw");
+ return -ENXIO;
+ }
+
+ nfit_blk = devm_kzalloc(dev, sizeof(*nfit_blk), GFP_KERNEL);
+ if (!nfit_blk)
+ return -ENOMEM;
+ nd_blk_region_set_provider_data(ndbr, nfit_blk);
+ nfit_blk->nd_region = to_nd_region(dev);
+
+ /* map block aperture memory */
+ nfit_blk->bdw_offset = nfit_mem->bdw->offset;
+ mmio = &nfit_blk->mmio[BDW];
+ mmio->base = nfit_spa_map(acpi_desc, nfit_mem->spa_bdw);
+ if (!mmio->base) {
+ dev_dbg(dev, "%s: %s failed to map bdw\n", __func__,
+ nvdimm_name(nvdimm));
+ return -ENOMEM;
+ }
+ mmio->size = nfit_mem->bdw->size;
+ mmio->base_offset = nfit_mem->memdev_bdw->region_offset;
+ mmio->idt = nfit_mem->idt_bdw;
+ mmio->spa = nfit_mem->spa_bdw;
+ rc = nfit_blk_init_interleave(mmio, nfit_mem->idt_bdw,
+ nfit_mem->memdev_bdw->interleave_ways);
+ if (rc) {
+ dev_dbg(dev, "%s: %s failed to init bdw interleave\n",
+ __func__, nvdimm_name(nvdimm));
+ return rc;
+ }
+
+ /* map block control memory */
+ nfit_blk->cmd_offset = nfit_mem->dcr->command_offset;
+ nfit_blk->stat_offset = nfit_mem->dcr->status_offset;
+ mmio = &nfit_blk->mmio[DCR];
+ mmio->base = nfit_spa_map(acpi_desc, nfit_mem->spa_dcr);
+ if (!mmio->base) {
+ dev_dbg(dev, "%s: %s failed to map dcr\n", __func__,
+ nvdimm_name(nvdimm));
+ return -ENOMEM;
+ }
+ mmio->size = nfit_mem->dcr->window_size;
+ mmio->base_offset = nfit_mem->memdev_dcr->region_offset;
+ mmio->idt = nfit_mem->idt_dcr;
+ mmio->spa = nfit_mem->spa_dcr;
+ rc = nfit_blk_init_interleave(mmio, nfit_mem->idt_dcr,
+ nfit_mem->memdev_dcr->interleave_ways);
+ if (rc) {
+ dev_dbg(dev, "%s: %s failed to init dcr interleave\n",
+ __func__, nvdimm_name(nvdimm));
+ return rc;
+ }
+
+ if (mmio->line_size == 0)
+ return 0;
+
+ if ((u32) nfit_blk->cmd_offset % mmio->line_size + 8 > mmio->line_size) {
+ dev_dbg(dev, "cmd_offset crosses interleave boundary\n");
+ return -ENXIO;
+ } else if ((u32) nfit_blk->stat_offset % mmio->line_size + 8 > mmio->line_size) {
+ dev_dbg(dev, "stat_offset crosses interleave boundary\n");
+ return -ENXIO;
+ }
+
+ return 0;
+}
+
+static void acpi_nfit_blk_region_disable(struct nvdimm_bus *nvdimm_bus,
+ struct device *dev)
+{
+ struct nvdimm_bus_descriptor *nd_desc = to_nd_desc(nvdimm_bus);
+ struct acpi_nfit_desc *acpi_desc = to_acpi_desc(nd_desc);
+ struct nd_blk_region *ndbr = to_nd_blk_region(dev);
+ struct nfit_blk *nfit_blk = nd_blk_region_provider_data(ndbr);
+ int i;
+
+ if (!nfit_blk)
+ return; /* never enabled */
+
+ /* auto-free BLK spa mappings */
+ for (i = 0; i < 2; i++) {
+ struct nfit_blk_mmio *mmio = &nfit_blk->mmio[i];
+
+ if (mmio->base)
+ nfit_spa_unmap(acpi_desc, mmio->spa);
+ }
+ nd_blk_region_set_provider_data(ndbr, NULL);
+ /* devm will free nfit_blk */
+}
+
static int acpi_nfit_register_region(struct acpi_nfit_desc *acpi_desc,
struct nfit_spa *nfit_spa)
{
static struct nd_mapping nd_mappings[ND_MAX_MAPPINGS];
struct acpi_nfit_system_address *spa = nfit_spa->spa;
+ struct nd_blk_region_desc ndbr_desc;
+ struct nd_region_desc *ndr_desc;
struct nfit_memdev *nfit_memdev;
- struct nd_region_desc ndr_desc;
int spa_type, count = 0, rc;
struct resource res;
u16 range_index;
@@ -850,12 +1246,13 @@ static int acpi_nfit_register_region(struct acpi_nfit_desc *acpi_desc,
memset(&res, 0, sizeof(res));
memset(&nd_mappings, 0, sizeof(nd_mappings));
- memset(&ndr_desc, 0, sizeof(ndr_desc));
+ memset(&ndbr_desc, 0, sizeof(ndr_desc));
res.start = spa->address;
res.end = res.start + spa->length - 1;
- ndr_desc.res = &res;
- ndr_desc.provider_data = nfit_spa;
- ndr_desc.attr_groups = acpi_nfit_region_attribute_groups;
+ ndr_desc = &ndbr_desc.ndr_desc;
+ ndr_desc->res = &res;
+ ndr_desc->provider_data = nfit_spa;
+ ndr_desc->attr_groups = acpi_nfit_region_attribute_groups;
list_for_each_entry(nfit_memdev, &acpi_desc->memdevs, list) {
struct acpi_nfit_memory_map *memdev = nfit_memdev->memdev;
struct nd_mapping *nd_mapping;
@@ -891,26 +1288,31 @@ static int acpi_nfit_register_region(struct acpi_nfit_desc *acpi_desc,
} else {
nd_mapping->size = nfit_mem->bdw->capacity;
nd_mapping->start = nfit_mem->bdw->start_address;
- ndr_desc.num_lanes = nfit_mem->bdw->windows;
+ ndr_desc->num_lanes = nfit_mem->bdw->windows;
}
- ndr_desc.nd_mapping = nd_mapping;
- ndr_desc.num_mappings = blk_valid;
- if (!nvdimm_blk_region_create(acpi_desc->nvdimm_bus, &ndr_desc))
+ ndr_desc->nd_mapping = nd_mapping;
+ ndr_desc->num_mappings = blk_valid;
+ ndbr_desc.enable = acpi_nfit_blk_region_enable;
+ ndbr_desc.disable = acpi_nfit_blk_region_disable;
+ ndbr_desc.do_io = acpi_nfit_blk_region_do_io;
+ if (!nvdimm_blk_region_create(acpi_desc->nvdimm_bus,
+ ndr_desc))
return -ENOMEM;
}
}
- ndr_desc.nd_mapping = nd_mappings;
- ndr_desc.num_mappings = count;
- rc = acpi_nfit_init_interleave_set(acpi_desc, &ndr_desc, spa);
+ ndr_desc->nd_mapping = nd_mappings;
+ ndr_desc->num_mappings = count;
+ rc = acpi_nfit_init_interleave_set(acpi_desc, ndr_desc, spa);
if (rc)
return rc;
if (spa_type == NFIT_SPA_PM) {
- if (!nvdimm_pmem_region_create(acpi_desc->nvdimm_bus, &ndr_desc))
+ if (!nvdimm_pmem_region_create(acpi_desc->nvdimm_bus, ndr_desc))
return -ENOMEM;
} else if (spa_type == NFIT_SPA_VOLATILE) {
- if (!nvdimm_volatile_region_create(acpi_desc->nvdimm_bus, &ndr_desc))
+ if (!nvdimm_volatile_region_create(acpi_desc->nvdimm_bus,
+ ndr_desc))
return -ENOMEM;
}
return 0;
@@ -936,11 +1338,14 @@ static int acpi_nfit_init(struct acpi_nfit_desc *acpi_desc, acpi_size sz)
u8 *data;
int rc;
+ INIT_LIST_HEAD(&acpi_desc->spa_maps);
INIT_LIST_HEAD(&acpi_desc->spas);
INIT_LIST_HEAD(&acpi_desc->dcrs);
INIT_LIST_HEAD(&acpi_desc->bdws);
+ INIT_LIST_HEAD(&acpi_desc->idts);
INIT_LIST_HEAD(&acpi_desc->memdevs);
INIT_LIST_HEAD(&acpi_desc->dimms);
+ mutex_init(&acpi_desc->spa_map_mutex);
data = (u8 *) acpi_desc->nfit;
end = data + sz;
diff --git a/drivers/acpi/nfit.h b/drivers/acpi/nfit.h
index 3a87042aec77..b2e56becc8fd 100644
--- a/drivers/acpi/nfit.h
+++ b/drivers/acpi/nfit.h
@@ -52,6 +52,11 @@ struct nfit_bdw {
struct list_head list;
};
+struct nfit_idt {
+ struct acpi_nfit_interleave *idt;
+ struct list_head list;
+};
+
struct nfit_memdev {
struct acpi_nfit_memory_map *memdev;
struct list_head list;
@@ -62,10 +67,13 @@ struct nfit_mem {
struct nvdimm *nvdimm;
struct acpi_nfit_memory_map *memdev_dcr;
struct acpi_nfit_memory_map *memdev_pmem;
+ struct acpi_nfit_memory_map *memdev_bdw;
struct acpi_nfit_control_region *dcr;
struct acpi_nfit_data_region *bdw;
struct acpi_nfit_system_address *spa_dcr;
struct acpi_nfit_system_address *spa_bdw;
+ struct acpi_nfit_interleave *idt_dcr;
+ struct acpi_nfit_interleave *idt_bdw;
struct list_head list;
struct acpi_device *adev;
unsigned long dsm_mask;
@@ -74,16 +82,57 @@ struct nfit_mem {
struct acpi_nfit_desc {
struct nvdimm_bus_descriptor nd_desc;
struct acpi_table_nfit *nfit;
+ struct mutex spa_map_mutex;
+ struct list_head spa_maps;
struct list_head memdevs;
struct list_head dimms;
struct list_head spas;
struct list_head dcrs;
struct list_head bdws;
+ struct list_head idts;
struct nvdimm_bus *nvdimm_bus;
struct device *dev;
unsigned long dimm_dsm_force_en;
};
+enum nd_blk_mmio_selector {
+ BDW,
+ DCR,
+};
+
+struct nfit_blk {
+ struct nfit_blk_mmio {
+ union {
+ void __iomem *base;
+ void *aperture;
+ };
+ u64 size;
+ u64 base_offset;
+ u32 line_size;
+ u32 num_lines;
+ u32 table_size;
+ struct acpi_nfit_interleave *idt;
+ struct acpi_nfit_system_address *spa;
+ } mmio[2];
+ struct nd_region *nd_region;
+ u64 bdw_offset; /* post interleave offset */
+ u64 stat_offset;
+ u64 cmd_offset;
+};
+
+struct nfit_spa_mapping {
+ struct acpi_nfit_desc *acpi_desc;
+ struct acpi_nfit_system_address *spa;
+ struct list_head list;
+ struct kref kref;
+ void __iomem *iomem;
+};
+
+static inline struct nfit_spa_mapping *to_spa_map(struct kref *kref)
+{
+ return container_of(kref, struct nfit_spa_mapping, kref);
+}
+
static inline struct acpi_nfit_memory_map *__to_nfit_memdev(struct nfit_mem *nfit_mem)
{
if (nfit_mem->memdev_dcr)
diff --git a/drivers/nvdimm/Kconfig b/drivers/nvdimm/Kconfig
index da7037c018e6..b134f0a2f9a7 100644
--- a/drivers/nvdimm/Kconfig
+++ b/drivers/nvdimm/Kconfig
@@ -34,6 +34,19 @@ config BLK_DEV_PMEM
Say Y if you want to use a NVDIMM described by ACPI, E820, etc...
+config ND_BLK
+ tristate "BLK: Block data window (aperture) device support"
+ depends on LIBNVDIMM
+ default LIBNVDIMM
+ help
+ Support NVDIMMs, or other devices, that implement a BLK-mode
+ access capability. BLK-mode access uses memory-mapped-i/o
+ apertures to access persistent media.
+
+ Say Y if your platform firmware emits an ACPI.NFIT table
+ (CONFIG_ACPI_NFIT), or otherwise exposes BLK-mode
+ capabilities.
+
config ND_BTT_DEVS
bool
diff --git a/drivers/nvdimm/Makefile b/drivers/nvdimm/Makefile
index aa5bb1acf831..b5682e70904a 100644
--- a/drivers/nvdimm/Makefile
+++ b/drivers/nvdimm/Makefile
@@ -1,11 +1,14 @@
obj-$(CONFIG_LIBNVDIMM) += libnvdimm.o
obj-$(CONFIG_BLK_DEV_PMEM) += nd_pmem.o
obj-$(CONFIG_ND_BTT) += nd_btt.o
+obj-$(CONFIG_ND_BLK) += nd_blk.o
nd_pmem-y := pmem.o
nd_btt-y := btt.o
+nd_blk-y := blk.o
+
libnvdimm-y := core.o
libnvdimm-y += bus.o
libnvdimm-y += dimm_devs.o
diff --git a/drivers/nvdimm/blk.c b/drivers/nvdimm/blk.c
new file mode 100644
index 000000000000..37b6f52fe1c2
--- /dev/null
+++ b/drivers/nvdimm/blk.c
@@ -0,0 +1,252 @@
+/*
+ * NVDIMM Block Window Driver
+ * Copyright (c) 2014, Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ */
+
+#include <linux/blkdev.h>
+#include <linux/fs.h>
+#include <linux/genhd.h>
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/nd.h>
+#include <linux/sizes.h>
+#include "nd.h"
+
+struct nd_blk_device {
+ struct request_queue *queue;
+ struct gendisk *disk;
+ struct nd_namespace_blk *nsblk;
+ struct nd_blk_region *ndbr;
+ struct nd_io ndio;
+ size_t disk_size;
+};
+
+static int nd_blk_major;
+
+static resource_size_t to_dev_offset(struct nd_namespace_blk *nsblk,
+ resource_size_t ns_offset, unsigned int len)
+{
+ int i;
+
+ for (i = 0; i < nsblk->num_resources; i++) {
+ if (ns_offset < resource_size(nsblk->res[i])) {
+ if (ns_offset + len > resource_size(nsblk->res[i])) {
+ dev_WARN_ONCE(&nsblk->dev, 1,
+ "%s: illegal request\n", __func__);
+ return SIZE_MAX;
+ }
+ return nsblk->res[i]->start + ns_offset;
+ }
+ ns_offset -= resource_size(nsblk->res[i]);
+ }
+
+ dev_WARN_ONCE(&nsblk->dev, 1, "%s: request out of range\n", __func__);
+ return SIZE_MAX;
+}
+
+static void nd_blk_make_request(struct request_queue *q, struct bio *bio)
+{
+ struct block_device *bdev = bio->bi_bdev;
+ struct gendisk *disk = bdev->bd_disk;
+ struct nd_namespace_blk *nsblk;
+ struct nd_blk_device *blk_dev;
+ struct nd_blk_region *ndbr;
+ struct bvec_iter iter;
+ struct bio_vec bvec;
+ int err = 0, rw;
+ sector_t sector;
+
+ sector = bio->bi_iter.bi_sector;
+ if (bio_end_sector(bio) > get_capacity(disk)) {
+ err = -EIO;
+ goto out;
+ }
+
+ BUG_ON(bio->bi_rw & REQ_DISCARD);
+
+ rw = bio_data_dir(bio);
+
+ blk_dev = disk->private_data;
+ nsblk = blk_dev->nsblk;
+ ndbr = blk_dev->ndbr;
+ bio_for_each_segment(bvec, bio, iter) {
+ unsigned int len = bvec.bv_len;
+ resource_size_t dev_offset;
+ void *iobuf;
+
+ BUG_ON(len > PAGE_SIZE);
+
+ dev_offset = to_dev_offset(nsblk, sector << SECTOR_SHIFT, len);
+ if (dev_offset == SIZE_MAX) {
+ err = -EIO;
+ goto out;
+ }
+
+ iobuf = kmap_atomic(bvec.bv_page);
+ err = ndbr->do_io(ndbr, iobuf + bvec.bv_offset, len, rw, dev_offset);
+ kunmap_atomic(iobuf);
+ if (err)
+ goto out;
+
+ sector += len >> SECTOR_SHIFT;
+ }
+
+ out:
+ bio_endio(bio, err);
+}
+
+static int nd_blk_rw_bytes(struct nd_io *ndio, void *iobuf, size_t offset,
+ size_t n, unsigned long flags)
+{
+ struct nd_namespace_blk *nsblk;
+ struct nd_blk_device *blk_dev;
+ int rw = nd_data_dir(flags);
+ struct nd_blk_region *ndbr;
+ resource_size_t dev_offset;
+
+ blk_dev = container_of(ndio, typeof(*blk_dev), ndio);
+ ndbr = blk_dev->ndbr;
+ nsblk = blk_dev->nsblk;
+ dev_offset = to_dev_offset(nsblk, offset, n);
+
+ if (unlikely(offset + n > blk_dev->disk_size)) {
+ dev_WARN_ONCE(ndio->dev, 1, "%s: request out of range\n",
+ __func__);
+ return -EFAULT;
+ }
+
+ if (dev_offset == SIZE_MAX)
+ return -EIO;
+
+ return ndbr->do_io(ndbr, iobuf, n, rw, dev_offset);
+}
+
+static const struct block_device_operations nd_blk_fops = {
+ .owner = THIS_MODULE,
+};
+
+static int nd_blk_probe(struct device *dev)
+{
+ struct nd_namespace_blk *nsblk = to_nd_namespace_blk(dev);
+ struct nd_region *nd_region = to_nd_region(dev->parent);
+ struct nd_blk_device *blk_dev;
+ resource_size_t disk_size;
+ struct gendisk *disk;
+ int err;
+
+ disk_size = nd_namespace_blk_validate(nsblk);
+ if (disk_size < ND_MIN_NAMESPACE_SIZE)
+ return -ENXIO;
+
+ blk_dev = kzalloc(sizeof(*blk_dev), GFP_KERNEL);
+ if (!blk_dev)
+ return -ENOMEM;
+
+ blk_dev->disk_size = disk_size;
+
+ blk_dev->queue = blk_alloc_queue(GFP_KERNEL);
+ if (!blk_dev->queue) {
+ err = -ENOMEM;
+ goto err_alloc_queue;
+ }
+
+ blk_queue_make_request(blk_dev->queue, nd_blk_make_request);
+ blk_queue_max_hw_sectors(blk_dev->queue, 1024);
+ blk_queue_bounce_limit(blk_dev->queue, BLK_BOUNCE_ANY);
+ blk_queue_logical_block_size(blk_dev->queue, nsblk->lbasize);
+
+ disk = blk_dev->disk = alloc_disk(0);
+ if (!disk) {
+ err = -ENOMEM;
+ goto err_alloc_disk;
+ }
+
+ blk_dev->ndbr = to_nd_blk_region(nsblk->dev.parent);
+ blk_dev->nsblk = nsblk;
+
+ disk->driverfs_dev = dev;
+ disk->major = nd_blk_major;
+ disk->first_minor = 0;
+ disk->fops = &nd_blk_fops;
+ disk->private_data = blk_dev;
+ disk->queue = blk_dev->queue;
+ disk->flags = GENHD_FL_EXT_DEVT;
+ sprintf(disk->disk_name, "ndblk%d.%d", nd_region->id, nsblk->id);
+ set_capacity(disk, disk_size >> SECTOR_SHIFT);
+
+ nvdimm_bus_lock(dev);
+ dev_set_drvdata(dev, blk_dev);
+
+ add_disk(disk);
+ nd_init_ndio(&blk_dev->ndio, nd_blk_rw_bytes, dev, disk, 0);
+ nd_register_ndio(&blk_dev->ndio);
+ nvdimm_bus_unlock(dev);
+
+ return 0;
+
+ err_alloc_disk:
+ blk_cleanup_queue(blk_dev->queue);
+ err_alloc_queue:
+ kfree(blk_dev);
+ return err;
+}
+
+static int nd_blk_remove(struct device *dev)
+{
+ struct nd_blk_device *blk_dev = dev_get_drvdata(dev);
+
+ nd_unregister_ndio(&blk_dev->ndio);
+ del_gendisk(blk_dev->disk);
+ put_disk(blk_dev->disk);
+ blk_cleanup_queue(blk_dev->queue);
+ kfree(blk_dev);
+
+ return 0;
+}
+
+static struct nd_device_driver nd_blk_driver = {
+ .probe = nd_blk_probe,
+ .remove = nd_blk_remove,
+ .drv = {
+ .name = "nd_blk",
+ },
+ .type = ND_DRIVER_NAMESPACE_BLK,
+};
+
+static int __init nd_blk_init(void)
+{
+ int rc;
+
+ rc = register_blkdev(0, "nd_blk");
+ if (rc < 0)
+ return rc;
+
+ nd_blk_major = rc;
+ rc = nd_driver_register(&nd_blk_driver);
+
+ if (rc < 0)
+ unregister_blkdev(nd_blk_major, "nd_blk");
+
+ return rc;
+}
+
+static void __exit nd_blk_exit(void)
+{
+ driver_unregister(&nd_blk_driver.drv);
+ unregister_blkdev(nd_blk_major, "nd_blk");
+}
+
+MODULE_AUTHOR("Ross Zwisler <ross.zwisler-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>");
+MODULE_LICENSE("GPL v2");
+MODULE_ALIAS_ND_DEVICE(ND_DEVICE_NAMESPACE_BLK);
+module_init(nd_blk_init);
+module_exit(nd_blk_exit);
diff --git a/drivers/nvdimm/dimm_devs.c b/drivers/nvdimm/dimm_devs.c
index 38fb68af5ef9..ae5f7fb8ce38 100644
--- a/drivers/nvdimm/dimm_devs.c
+++ b/drivers/nvdimm/dimm_devs.c
@@ -209,6 +209,15 @@ struct nvdimm *to_nvdimm(struct device *dev)
}
EXPORT_SYMBOL_GPL(to_nvdimm);
+struct nvdimm *nd_blk_region_to_dimm(struct nd_blk_region *ndbr)
+{
+ struct nd_region *nd_region = &ndbr->nd_region;
+ struct nd_mapping *nd_mapping = &nd_region->mapping[0];
+
+ return nd_mapping->nvdimm;
+}
+EXPORT_SYMBOL_GPL(nd_blk_region_to_dimm);
+
struct nvdimm_drvdata *to_ndd(struct nd_mapping *nd_mapping)
{
struct nvdimm *nvdimm = nd_mapping->nvdimm;
diff --git a/drivers/nvdimm/namespace_devs.c b/drivers/nvdimm/namespace_devs.c
index 893eefafb940..ab71b454a020 100644
--- a/drivers/nvdimm/namespace_devs.c
+++ b/drivers/nvdimm/namespace_devs.c
@@ -151,6 +151,53 @@ static resource_size_t nd_namespace_blk_size(struct nd_namespace_blk *nsblk)
return size;
}
+resource_size_t nd_namespace_blk_validate(struct nd_namespace_blk *nsblk)
+{
+ struct nd_region *nd_region = to_nd_region(nsblk->dev.parent);
+ struct nd_mapping *nd_mapping = &nd_region->mapping[0];
+ struct nvdimm_drvdata *ndd = to_ndd(nd_mapping);
+ struct nd_label_id label_id;
+ struct resource *res;
+ int count, i;
+
+ if (!nsblk->uuid || !nsblk->lbasize)
+ return 0;
+
+ count = 0;
+ nd_label_gen_id(&label_id, nsblk->uuid, NSLABEL_FLAG_LOCAL);
+ for_each_dpa_resource(ndd, res) {
+ if (strcmp(res->name, label_id.id) != 0)
+ continue;
+ /*
+ * Resources with unacknoweldged adjustments indicate a
+ * failure to update labels
+ */
+ if (res->flags & DPA_RESOURCE_ADJUSTED)
+ return 0;
+ count++;
+ }
+
+ /* These values match after a successful label update */
+ if (count != nsblk->num_resources)
+ return 0;
+
+ for (i = 0; i < nsblk->num_resources; i++) {
+ struct resource *found = NULL;
+
+ for_each_dpa_resource(ndd, res)
+ if (res == nsblk->res[i]) {
+ found = res;
+ break;
+ }
+ /* stale resource */
+ if (!found)
+ return 0;
+ }
+
+ return nd_namespace_blk_size(nsblk);
+}
+EXPORT_SYMBOL(nd_namespace_blk_validate);
+
static int nd_namespace_label_update(struct nd_region *nd_region, struct device *dev)
{
dev_WARN_ONCE(dev, dev->driver,
diff --git a/drivers/nvdimm/nd-private.h b/drivers/nvdimm/nd-private.h
index 635af040023f..89633ce4774a 100644
--- a/drivers/nvdimm/nd-private.h
+++ b/drivers/nvdimm/nd-private.h
@@ -23,7 +23,6 @@ extern struct list_head nvdimm_bus_list;
extern struct mutex nvdimm_bus_list_mutex;
extern int nvdimm_major;
-struct block_device;
struct nd_io_claim;
struct nd_btt;
struct nd_io;
@@ -51,8 +50,8 @@ struct nvdimm {
struct nd_io *ndio_lookup(struct nvdimm_bus *nvdimm_bus, const char *diskname);
bool is_nvdimm(struct device *dev);
-bool is_nd_blk(struct device *dev);
bool is_nd_pmem(struct device *dev);
+bool is_nd_blk(struct device *dev);
#if IS_ENABLED(CONFIG_ND_BTT_DEVS)
bool is_nd_btt(struct device *dev);
struct nd_btt *nd_btt_create(struct nvdimm_bus *nvdimm_bus);
diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h
index 2bea91d270cc..c807379d9653 100644
--- a/drivers/nvdimm/nd.h
+++ b/drivers/nvdimm/nd.h
@@ -113,6 +113,15 @@ struct nd_region {
struct nd_mapping mapping[0];
};
+struct nd_blk_region {
+ int (*enable)(struct nvdimm_bus *nvdimm_bus, struct device *dev);
+ void (*disable)(struct nvdimm_bus *nvdimm_bus, struct device *dev);
+ int (*do_io)(struct nd_blk_region *ndbr, void *iobuf, u64 len,
+ int write, resource_size_t dpa);
+ void *blk_provider_data;
+ struct nd_region nd_region;
+};
+
/*
* Lookup next in the repeating sequence of 01, 10, and 11.
*/
@@ -236,8 +245,6 @@ struct nd_btt *to_nd_btt(struct device *dev);
struct btt_sb;
u64 nd_btt_sb_checksum(struct btt_sb *btt_sb);
struct nd_region *to_nd_region(struct device *dev);
-unsigned int nd_region_acquire_lane(struct nd_region *nd_region);
-void nd_region_release_lane(struct nd_region *nd_region, unsigned int lane);
int nd_region_to_namespace_type(struct nd_region *nd_region);
int nd_region_register_namespaces(struct nd_region *nd_region, int *err);
u64 nd_region_interleave_set_cookie(struct nd_region *nd_region);
@@ -249,4 +256,6 @@ void nvdimm_free_dpa(struct nvdimm_drvdata *ndd, struct resource *res);
struct resource *nvdimm_allocate_dpa(struct nvdimm_drvdata *ndd,
struct nd_label_id *label_id, resource_size_t start,
resource_size_t n);
+int nd_blk_region_init(struct nd_region *nd_region);
+resource_size_t nd_namespace_blk_validate(struct nd_namespace_blk *nsblk);
#endif /* __ND_H__ */
diff --git a/drivers/nvdimm/region.c b/drivers/nvdimm/region.c
index aa617bf86506..d9d82e7a90fa 100644
--- a/drivers/nvdimm/region.c
+++ b/drivers/nvdimm/region.c
@@ -94,11 +94,10 @@ EXPORT_SYMBOL(nd_region_release_lane);
static int nd_region_probe(struct device *dev)
{
- int err;
+ int err, rc;
static unsigned long once;
struct nd_region_namespaces *num_ns;
struct nd_region *nd_region = to_nd_region(dev);
- int rc = nd_region_register_namespaces(nd_region, &err);
if (nd_region->num_lanes > num_online_cpus()
&& nd_region->num_lanes < num_possible_cpus()
@@ -110,6 +109,11 @@ static int nd_region_probe(struct device *dev)
nd_region->num_lanes);
}
+ rc = nd_blk_region_init(nd_region);
+ if (rc)
+ return rc;
+
+ rc = nd_region_register_namespaces(nd_region, &err);
num_ns = devm_kzalloc(dev, sizeof(*num_ns), GFP_KERNEL);
if (!num_ns)
return -ENOMEM;
diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c
index 1274f9300002..5790aec081f4 100644
--- a/drivers/nvdimm/region_devs.c
+++ b/drivers/nvdimm/region_devs.c
@@ -11,6 +11,7 @@
* General Public License for more details.
*/
#include <linux/scatterlist.h>
+#include <linux/highmem.h>
#include <linux/sched.h>
#include <linux/slab.h>
#include <linux/sort.h>
@@ -33,7 +34,10 @@ static void nd_region_release(struct device *dev)
put_device(&nvdimm->dev);
}
ida_simple_remove(®ion_ida, nd_region->id);
- kfree(nd_region);
+ if (is_nd_blk(dev))
+ kfree(to_nd_blk_region(dev));
+ else
+ kfree(nd_region);
}
static struct device_type nd_blk_device_type = {
@@ -70,6 +74,33 @@ struct nd_region *to_nd_region(struct device *dev)
}
EXPORT_SYMBOL_GPL(to_nd_region);
+struct nd_blk_region *to_nd_blk_region(struct device *dev)
+{
+ struct nd_region *nd_region = to_nd_region(dev);
+
+ WARN_ON(!is_nd_blk(dev));
+ return container_of(nd_region, struct nd_blk_region, nd_region);
+}
+EXPORT_SYMBOL_GPL(to_nd_blk_region);
+
+void *nd_region_provider_data(struct nd_region *nd_region)
+{
+ return nd_region->provider_data;
+}
+EXPORT_SYMBOL_GPL(nd_region_provider_data);
+
+void *nd_blk_region_provider_data(struct nd_blk_region *ndbr)
+{
+ return ndbr->blk_provider_data;
+}
+EXPORT_SYMBOL_GPL(nd_blk_region_provider_data);
+
+void nd_blk_region_set_provider_data(struct nd_blk_region *ndbr, void *data)
+{
+ ndbr->blk_provider_data = data;
+}
+EXPORT_SYMBOL_GPL(nd_blk_region_set_provider_data);
+
/**
* nd_region_to_namespace_type() - region to an integer namespace type
* @nd_region: region-device to interrogate
@@ -344,14 +375,13 @@ u64 nd_region_interleave_set_cookie(struct nd_region *nd_region)
/*
* Upon successful probe/remove, take/release a reference on the
- * associated interleave set (if present)
+ * associated dimms in the interleave set, on successful probe of a BLK
+ * namespace check if we need a new seed, and on remove or failed probe
+ * of a BLK region notify the provider to disable the region.
*/
static void nd_region_notify_driver_action(struct nvdimm_bus *nvdimm_bus,
struct device *dev, int rc, bool probe)
{
- if (rc)
- return;
-
if (is_nd_pmem(dev) || is_nd_blk(dev)) {
struct nd_region *nd_region = to_nd_region(dev);
int i;
@@ -360,11 +390,16 @@ static void nd_region_notify_driver_action(struct nvdimm_bus *nvdimm_bus,
struct nd_mapping *nd_mapping = &nd_region->mapping[i];
struct nvdimm *nvdimm = nd_mapping->nvdimm;
- if (probe)
+ if (probe && rc == 0)
atomic_inc(&nvdimm->busy);
- else
+ else if (!probe)
atomic_dec(&nvdimm->busy);
}
+
+ if (is_nd_pmem(dev) || (probe && rc == 0))
+ return;
+
+ to_nd_blk_region(dev)->disable(nvdimm_bus, dev);
} else if (dev->parent && is_nd_blk(dev->parent) && probe && rc == 0) {
struct nd_region *nd_region = to_nd_region(dev->parent);
@@ -508,11 +543,21 @@ struct attribute_group nd_mapping_attribute_group = {
};
EXPORT_SYMBOL_GPL(nd_mapping_attribute_group);
-void *nd_region_provider_data(struct nd_region *nd_region)
+int nd_blk_region_init(struct nd_region *nd_region)
{
- return nd_region->provider_data;
+ struct device *dev = &nd_region->dev;
+ struct nvdimm_bus *nvdimm_bus = walk_to_nvdimm_bus(dev);
+
+ if (!is_nd_blk(dev))
+ return 0;
+
+ if (nd_region->ndr_mappings < 1) {
+ dev_err(dev, "invalid BLK region\n");
+ return -ENXIO;
+ }
+
+ return to_nd_blk_region(dev)->enable(nvdimm_bus, dev);
}
-EXPORT_SYMBOL_GPL(nd_region_provider_data);
static noinline struct nd_region *nd_region_create(struct nvdimm_bus *nvdimm_bus,
struct nd_region_desc *ndr_desc, struct device_type *dev_type)
@@ -534,9 +579,28 @@ static noinline struct nd_region *nd_region_create(struct nvdimm_bus *nvdimm_bus
}
}
- nd_region = kzalloc(sizeof(struct nd_region)
- + sizeof(struct nd_mapping) * ndr_desc->num_mappings,
- GFP_KERNEL);
+ if (dev_type == &nd_blk_device_type) {
+ struct nd_blk_region_desc *ndbr_desc;
+ struct nd_blk_region *ndbr;
+
+ ndbr_desc = container_of(ndr_desc, typeof(*ndbr_desc), ndr_desc);
+ ndbr = kzalloc(sizeof(*ndbr) + sizeof(struct nd_mapping)
+ * ndr_desc->num_mappings,
+ GFP_KERNEL);
+ if (ndbr) {
+ nd_region = &ndbr->nd_region;
+ ndbr->enable = ndbr_desc->enable;
+ ndbr->disable = ndbr_desc->disable;
+ ndbr->do_io = ndbr_desc->do_io;
+ } else
+ nd_region = NULL;
+ } else {
+ nd_region = kzalloc(sizeof(struct nd_region)
+ + sizeof(struct nd_mapping)
+ * ndr_desc->num_mappings,
+ GFP_KERNEL);
+ }
+
if (!nd_region)
return NULL;
nd_region->id = ida_simple_get(®ion_ida, 0, 0, GFP_KERNEL);
diff --git a/include/linux/libnvdimm.h b/include/linux/libnvdimm.h
index f97b6adf9653..96b950717fab 100644
--- a/include/linux/libnvdimm.h
+++ b/include/linux/libnvdimm.h
@@ -81,8 +81,17 @@ struct nd_region_desc {
};
struct nvdimm_bus;
-struct device;
struct module;
+struct device;
+struct nd_blk_region;
+struct nd_blk_region_desc {
+ int (*enable)(struct nvdimm_bus *nvdimm_bus, struct device *dev);
+ void (*disable)(struct nvdimm_bus *nvdimm_bus, struct device *dev);
+ int (*do_io)(struct nd_blk_region *ndbr, void *iobuf, u64 len,
+ int write, resource_size_t dpa);
+ struct nd_region_desc ndr_desc;
+};
+
struct nvdimm_bus *__nvdimm_bus_register(struct device *parent,
struct nvdimm_bus_descriptor *nfit_desc, struct module *module);
#define nvdimm_bus_register(parent, desc) \
@@ -91,10 +100,10 @@ void nvdimm_bus_unregister(struct nvdimm_bus *nvdimm_bus);
struct nvdimm_bus *to_nvdimm_bus(struct device *dev);
struct nvdimm *to_nvdimm(struct device *dev);
struct nd_region *to_nd_region(struct device *dev);
+struct nd_blk_region *to_nd_blk_region(struct device *dev);
struct nvdimm_bus_descriptor *to_nd_desc(struct nvdimm_bus *nvdimm_bus);
const char *nvdimm_name(struct nvdimm *nvdimm);
void *nvdimm_provider_data(struct nvdimm *nvdimm);
-void *nd_region_provider_data(struct nd_region *nd_region);
struct nvdimm *nvdimm_create(struct nvdimm_bus *nvdimm_bus, void *provider_data,
const struct attribute_group **groups, unsigned long flags,
unsigned long *dsm_mask);
@@ -112,5 +121,11 @@ struct nd_region *nvdimm_blk_region_create(struct nvdimm_bus *nvdimm_bus,
struct nd_region_desc *ndr_desc);
struct nd_region *nvdimm_volatile_region_create(struct nvdimm_bus *nvdimm_bus,
struct nd_region_desc *ndr_desc);
+void *nd_region_provider_data(struct nd_region *nd_region);
+void *nd_blk_region_provider_data(struct nd_blk_region *ndbr);
+void nd_blk_region_set_provider_data(struct nd_blk_region *ndbr, void *data);
+struct nvdimm *nd_blk_region_to_dimm(struct nd_blk_region *ndbr);
+unsigned int nd_region_acquire_lane(struct nd_region *nd_region);
+void nd_region_release_lane(struct nd_region *nd_region, unsigned int lane);
u64 nd_fletcher64(void *addr, size_t len, bool le);
#endif /* __LIBNVDIMM_H__ */
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox