* Re: [dpdk-dev] [PATCH 4/4] ci: reorganise Travis jobs
From: Thomas Monjalon @ 2020-02-20 10:42 UTC (permalink / raw)
To: David Marchand, Aaron Conole; +Cc: dev, Michael Santana, Bruce Richardson
In-Reply-To: <f7tpneauuhz.fsf@dhcp-25.97.bos.redhat.com>
19/02/2020 22:39, Aaron Conole:
> David Marchand <david.marchand@redhat.com> writes:
>
> > Let's prune the jobs list to limit the amount of time spent by the robot
> > in Travis.
> >
> > Since meson enables automatically the relevant components, there is not
> > much gain in testing with extra_packages vs required_packages only.
> >
> > For a given arch/compiler/env combination, compilation is first tested
> > in all jobs that run tests or build the docs or run the ABI checks.
> > In the same context, for jobs that accumulates running tests, building
> > the docs etc..., those steps are independent and can be split to save
> > some cpu on Travis.
> >
> > With this, we go down from 21 to 15 jobs.
> >
> > Note: this patch requires a flush of the existing caches in Travis.
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > ---
>
> In general, I think the idea with required vs. extra was to have a build
> that did the minimum required, and one that did all the packages (to
> allow a minimum vs. full DPDK).
>
> At least, that's from
> http://mails.dpdk.org/archives/dev/2019-January/124007.html
I think the benefit of a minimum build is to have a quick report,
and easy to setup.
> Not sure if that's still something anyone cares about.
Given that Travis knows how to satisfy the dependencies,
and that we must wait for all jobs to finish,
I don't see any benefit of a minimal setup.
^ permalink raw reply
* Re: [dpdk-dev] [PATCH] examples/ipsec-secgw: extend inline session to non AES-GCM
From: Akhil Goyal @ 2020-02-20 10:42 UTC (permalink / raw)
To: Ananyev, Konstantin, Iremonger, Bernard
Cc: dev@dpdk.org, stable@dpdk.org, Ankur Dwivedi,
Narayana Prasad Raju Athreya, Anoob Joseph
In-Reply-To: <SN6PR11MB2558380B223583D9E473111B9A130@SN6PR11MB2558.namprd11.prod.outlook.com>
>
> Hi Akhil,
>
> > > > This patch extends creation of inline session to all the algorithms.
> > > > Previously the inline session was enabled only for AES-GCM cipher.
> > > >
> > > > Fixes: 3a690d5a65e2 ("examples/ipsec-secgw: fix first packet with inline
> > > > crypto")
> > > >
> > > > Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
> > > > ---
> > > > examples/ipsec-secgw/sa.c | 26 ++++++++++++--------------
> > > > 1 file changed, 12 insertions(+), 14 deletions(-)
> > >
> > > Acked-by: Anoob Joseph <anoobj@marvell.com>
> > Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
> >
> > Hi Konstantin,
> >
> > I am about to merge this patch. Do you have any issues on this.
>
> No objections, LGTM.
> Konstantin
>
Applied to dpdk-next-crypto
Thanks.
^ permalink raw reply
* Re: [PATCH v4 5/6] driver core: Rename deferred_probe_timeout and make it global
From: Rafael J. Wysocki @ 2020-02-20 10:42 UTC (permalink / raw)
To: John Stultz
Cc: lkml, Rob Herring, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
Pavel Machek, Len Brown, Todd Kjos, Bjorn Andersson,
Liam Girdwood, Mark Brown, Thierry Reding, Linus Walleij,
Greg Kroah-Hartman, Linux PM
In-Reply-To: <20200220050440.45878-6-john.stultz@linaro.org>
On Thu, Feb 20, 2020 at 6:05 AM John Stultz <john.stultz@linaro.org> wrote:
>
> Since other subsystems (like regulator) have similar arbitrary
> timeouts for how long they try to resolve driver dependencies,
> rename deferred_probe_timeout to driver_deferred_probe_timeout
> and set it as global, so it can be shared.
Fair enough.
> Cc: Rob Herring <robh@kernel.org>
> Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>
> Cc: Kevin Hilman <khilman@kernel.org>
> Cc: Ulf Hansson <ulf.hansson@linaro.org>
> Cc: Pavel Machek <pavel@ucw.cz>
> Cc: Len Brown <len.brown@intel.com>
> Cc: Todd Kjos <tkjos@google.com>
> Cc: Bjorn Andersson <bjorn.andersson@linaro.org>
> Cc: Liam Girdwood <lgirdwood@gmail.com>
> Cc: Mark Brown <broonie@kernel.org>
> Cc: Thierry Reding <treding@nvidia.com>
> Cc: Linus Walleij <linus.walleij@linaro.org>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: linux-pm@vger.kernel.org
> Signed-off-by: John Stultz <john.stultz@linaro.org>
> Change-Id: I92ee3b392004ecc9217c5337b54eda48c2d7f3ee
Reviewed-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> ---
> v4:
> * Split out into its own patch as suggested by Mark
> * Renamed deferred_probe_timeout as suggested by Greg
> ---
> drivers/base/dd.c | 18 ++++++++++--------
> include/linux/device/driver.h | 1 +
> 2 files changed, 11 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/base/dd.c b/drivers/base/dd.c
> index 408e4da081da..39f1ce6d4f1c 100644
> --- a/drivers/base/dd.c
> +++ b/drivers/base/dd.c
> @@ -229,17 +229,19 @@ DEFINE_SHOW_ATTRIBUTE(deferred_devs);
> * In the case of modules, set the default probe timeout to
> * 30 seconds to give userland some time to load needed modules
> */
> -static int deferred_probe_timeout = 30;
> +int driver_deferred_probe_timeout = 30;
> #else
> /* In the case of !modules, no probe timeout needed */
> -static int deferred_probe_timeout = -1;
> +int driver_deferred_probe_timeout = -1;
> #endif
> +EXPORT_SYMBOL_GPL(driver_deferred_probe_timeout);
> +
> static int __init deferred_probe_timeout_setup(char *str)
> {
> int timeout;
>
> if (!kstrtoint(str, 10, &timeout))
> - deferred_probe_timeout = timeout;
> + driver_deferred_probe_timeout = timeout;
> return 1;
> }
> __setup("deferred_probe_timeout=", deferred_probe_timeout_setup);
> @@ -259,10 +261,10 @@ __setup("deferred_probe_timeout=", deferred_probe_timeout_setup);
> */
> int driver_deferred_probe_check_state(struct device *dev)
> {
> - if (!initcalls_done || deferred_probe_timeout > 0)
> + if (!initcalls_done || driver_deferred_probe_timeout > 0)
> return -EPROBE_DEFER;
>
> - if (!deferred_probe_timeout) {
> + if (!driver_deferred_probe_timeout) {
> dev_WARN(dev, "deferred probe timeout, ignoring dependency");
> return -ETIMEDOUT;
> }
> @@ -276,7 +278,7 @@ static void deferred_probe_timeout_work_func(struct work_struct *work)
> {
> struct device_private *private, *p;
>
> - deferred_probe_timeout = 0;
> + driver_deferred_probe_timeout = 0;
> driver_deferred_probe_trigger();
> flush_work(&deferred_probe_work);
>
> @@ -310,9 +312,9 @@ static int deferred_probe_initcall(void)
> driver_deferred_probe_trigger();
> flush_work(&deferred_probe_work);
>
> - if (deferred_probe_timeout > 0) {
> + if (driver_deferred_probe_timeout > 0) {
> schedule_delayed_work(&deferred_probe_timeout_work,
> - deferred_probe_timeout * HZ);
> + driver_deferred_probe_timeout * HZ);
> }
> return 0;
> }
> diff --git a/include/linux/device/driver.h b/include/linux/device/driver.h
> index 5242afabfaba..ee7ba5b5417e 100644
> --- a/include/linux/device/driver.h
> +++ b/include/linux/device/driver.h
> @@ -236,6 +236,7 @@ driver_find_device_by_acpi_dev(struct device_driver *drv, const void *adev)
> }
> #endif
>
> +extern int driver_deferred_probe_timeout;
> void driver_deferred_probe_add(struct device *dev);
> int driver_deferred_probe_check_state(struct device *dev);
> void driver_init(void);
> --
> 2.17.1
>
^ permalink raw reply
* Re: [PATCH v2] util/async: make bh_aio_poll() O(1)
From: Paolo Bonzini @ 2020-02-20 10:40 UTC (permalink / raw)
To: Stefan Hajnoczi, qemu-devel; +Cc: Kevin Wolf, Fam Zheng, qemu-block, Max Reitz
In-Reply-To: <20200219175348.1161536-1-stefanha@redhat.com>
On 19/02/20 18:53, Stefan Hajnoczi wrote:
> The ctx->first_bh list contains all created BHs, including those that
> are not scheduled. The list is iterated by the event loop and therefore
> has O(n) time complexity with respected to the number of created BHs.
>
> Rewrite BHs so that only scheduled or deleted BHs are enqueued.
> Only BHs that actually require action will be iterated.
>
> One semantic change is required: qemu_bh_delete() enqueues the BH and
> therefore invokes aio_notify(). The
> tests/test-aio.c:test_source_bh_delete_from_cb() test case assumed that
> g_main_context_iteration(NULL, false) returns false after
> qemu_bh_delete() but it now returns true for one iteration. Fix up the
> test case.
>
> This patch makes aio_compute_timeout() and aio_bh_poll() drop from a CPU
> profile reported by perf-top(1). Previously they combined to 9% CPU
> utilization when AioContext polling is commented out and the guest has 2
> virtio-blk,num-queues=1 and 99 virtio-blk,num-queues=32 devices.
>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
> v2:
> * Use QSLIST for BHs and QSIMPLEQ for BHListSlices [Paolo]
> (Note that I replaced bh = atomic_rcu_read(&first_bh) with
> QSLIST_FOREACH(&bh_list) so there is no memory ordering but I think
> this is safe.)
I have sent an implementation of QSLIST_FOREACH_RCU. I doubt you'd see
any problem in practice, since RCU accesses only need a compiler
barrier, but it's cleaner.
With that changed,
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Any further optimization of aio_compute_timeout() and aio_ctx_check()
can be done on top.
Paolo
> ---
> include/block/aio.h | 20 +++-
> tests/test-aio.c | 3 +-
> util/async.c | 236 +++++++++++++++++++++++++-------------------
> 3 files changed, 157 insertions(+), 102 deletions(-)
>
> diff --git a/include/block/aio.h b/include/block/aio.h
> index 7ba9bd7874..1a2ce9ca26 100644
> --- a/include/block/aio.h
> +++ b/include/block/aio.h
> @@ -51,6 +51,19 @@ struct ThreadPool;
> struct LinuxAioState;
> struct LuringState;
>
> +/*
> + * Each aio_bh_poll() call carves off a slice of the BH list, so that newly
> + * scheduled BHs are not processed until the next aio_bh_poll() call. All
> + * active aio_bh_poll() calls chain their slices together in a list, so that
> + * nested aio_bh_poll() calls process all scheduled bottom halves.
> + */
> +typedef QSLIST_HEAD(, QEMUBH) BHList;
> +typedef struct BHListSlice BHListSlice;
> +struct BHListSlice {
> + BHList bh_list;
> + QSIMPLEQ_ENTRY(BHListSlice) next;
> +};
> +
> struct AioContext {
> GSource source;
>
> @@ -91,8 +104,11 @@ struct AioContext {
> */
> QemuLockCnt list_lock;
>
> - /* Anchor of the list of Bottom Halves belonging to the context */
> - struct QEMUBH *first_bh;
> + /* Bottom Halves pending aio_bh_poll() processing */
> + BHList bh_list;
> +
> + /* Chained BH list slices for each nested aio_bh_poll() call */
> + QSIMPLEQ_HEAD(, BHListSlice) bh_slice_list;
>
> /* Used by aio_notify.
> *
> diff --git a/tests/test-aio.c b/tests/test-aio.c
> index 86fb73b3d5..8a46078463 100644
> --- a/tests/test-aio.c
> +++ b/tests/test-aio.c
> @@ -615,7 +615,8 @@ static void test_source_bh_delete_from_cb(void)
> g_assert_cmpint(data1.n, ==, data1.max);
> g_assert(data1.bh == NULL);
>
> - g_assert(!g_main_context_iteration(NULL, false));
> + assert(g_main_context_iteration(NULL, false));
> + assert(!g_main_context_iteration(NULL, false));
> }
>
> static void test_source_bh_delete_from_cb_many(void)
> diff --git a/util/async.c b/util/async.c
> index c192a24a61..2c8499cc29 100644
> --- a/util/async.c
> +++ b/util/async.c
> @@ -36,16 +36,76 @@
> /***********************************************************/
> /* bottom halves (can be seen as timers which expire ASAP) */
>
> +/* QEMUBH::flags values */
> +enum {
> + /* Already enqueued and waiting for aio_bh_poll() */
> + BH_PENDING = (1 << 0),
> +
> + /* Invoke the callback */
> + BH_SCHEDULED = (1 << 1),
> +
> + /* Delete without invoking callback */
> + BH_DELETED = (1 << 2),
> +
> + /* Delete after invoking callback */
> + BH_ONESHOT = (1 << 3),
> +
> + /* Schedule periodically when the event loop is idle */
> + BH_IDLE = (1 << 4),
> +};
> +
> struct QEMUBH {
> AioContext *ctx;
> QEMUBHFunc *cb;
> void *opaque;
> - QEMUBH *next;
> - bool scheduled;
> - bool idle;
> - bool deleted;
> + QSLIST_ENTRY(QEMUBH) next;
> + unsigned flags;
> };
>
> +/* Called concurrently from any thread */
> +static void aio_bh_enqueue(QEMUBH *bh, unsigned new_flags)
> +{
> + AioContext *ctx = bh->ctx;
> + unsigned old_flags;
> +
> + /*
> + * The memory barrier implicit in atomic_fetch_or makes sure that:
> + * 1. idle & any writes needed by the callback are done before the
> + * locations are read in the aio_bh_poll.
> + * 2. ctx is loaded before the callback has a chance to execute and bh
> + * could be freed.
> + */
> + old_flags = atomic_fetch_or(&bh->flags, BH_PENDING | new_flags);
> + if (!(old_flags & BH_PENDING)) {
> + QSLIST_INSERT_HEAD_ATOMIC(&ctx->bh_list, bh, next);
> + }
> +
> + aio_notify(ctx);
> +}
> +
> +/* Only called from aio_bh_poll() and aio_ctx_finalize() */
> +static QEMUBH *aio_bh_dequeue(BHList *head, unsigned *flags)
> +{
> + QEMUBH *bh = QSLIST_FIRST(head);
> +
> + if (!bh) {
> + return NULL;
> + }
> +
> + QSLIST_REMOVE_HEAD(head, next);
> +
> + /*
> + * The atomic_and is paired with aio_bh_enqueue(). The implicit memory
> + * barrier ensures that the callback sees all writes done by the scheduling
> + * thread. It also ensures that the scheduling thread sees the cleared
> + * flag before bh->cb has run, and thus will call aio_notify again if
> + * necessary.
> + */
> + *flags = atomic_fetch_and(&bh->flags,
> + ~(BH_PENDING | BH_SCHEDULED | BH_IDLE));
> + return bh;
> +}
> +
> void aio_bh_schedule_oneshot(AioContext *ctx, QEMUBHFunc *cb, void *opaque)
> {
> QEMUBH *bh;
> @@ -55,15 +115,7 @@ void aio_bh_schedule_oneshot(AioContext *ctx, QEMUBHFunc *cb, void *opaque)
> .cb = cb,
> .opaque = opaque,
> };
> - qemu_lockcnt_lock(&ctx->list_lock);
> - bh->next = ctx->first_bh;
> - bh->scheduled = 1;
> - bh->deleted = 1;
> - /* Make sure that the members are ready before putting bh into list */
> - smp_wmb();
> - ctx->first_bh = bh;
> - qemu_lockcnt_unlock(&ctx->list_lock);
> - aio_notify(ctx);
> + aio_bh_enqueue(bh, BH_SCHEDULED | BH_ONESHOT);
> }
>
> QEMUBH *aio_bh_new(AioContext *ctx, QEMUBHFunc *cb, void *opaque)
> @@ -75,12 +127,6 @@ QEMUBH *aio_bh_new(AioContext *ctx, QEMUBHFunc *cb, void *opaque)
> .cb = cb,
> .opaque = opaque,
> };
> - qemu_lockcnt_lock(&ctx->list_lock);
> - bh->next = ctx->first_bh;
> - /* Make sure that the members are ready before putting bh into list */
> - smp_wmb();
> - ctx->first_bh = bh;
> - qemu_lockcnt_unlock(&ctx->list_lock);
> return bh;
> }
>
> @@ -89,91 +135,56 @@ void aio_bh_call(QEMUBH *bh)
> bh->cb(bh->opaque);
> }
>
> -/* Multiple occurrences of aio_bh_poll cannot be called concurrently.
> - * The count in ctx->list_lock is incremented before the call, and is
> - * not affected by the call.
> - */
> +/* Multiple occurrences of aio_bh_poll cannot be called concurrently. */
> int aio_bh_poll(AioContext *ctx)
> {
> - QEMUBH *bh, **bhp, *next;
> - int ret;
> - bool deleted = false;
> -
> - ret = 0;
> - for (bh = atomic_rcu_read(&ctx->first_bh); bh; bh = next) {
> - next = atomic_rcu_read(&bh->next);
> - /* The atomic_xchg is paired with the one in qemu_bh_schedule. The
> - * implicit memory barrier ensures that the callback sees all writes
> - * done by the scheduling thread. It also ensures that the scheduling
> - * thread sees the zero before bh->cb has run, and thus will call
> - * aio_notify again if necessary.
> - */
> - if (atomic_xchg(&bh->scheduled, 0)) {
> + BHListSlice slice;
> + BHListSlice *s;
> + int ret = 0;
> +
> + QSLIST_MOVE_ATOMIC(&slice.bh_list, &ctx->bh_list);
> + QSIMPLEQ_INSERT_TAIL(&ctx->bh_slice_list, &slice, next);
> +
> + while ((s = QSIMPLEQ_FIRST(&ctx->bh_slice_list))) {
> + QEMUBH *bh;
> + unsigned flags;
> +
> + bh = aio_bh_dequeue(&s->bh_list, &flags);
> + if (!bh) {
> + QSIMPLEQ_REMOVE_HEAD(&ctx->bh_slice_list, next);
> + continue;
> + }
> +
> + if ((flags & (BH_SCHEDULED | BH_DELETED)) == BH_SCHEDULED) {
> /* Idle BHs don't count as progress */
> - if (!bh->idle) {
> + if (!(flags & BH_IDLE)) {
> ret = 1;
> }
> - bh->idle = 0;
> aio_bh_call(bh);
> }
> - if (bh->deleted) {
> - deleted = true;
> + if (flags & (BH_DELETED | BH_ONESHOT)) {
> + g_free(bh);
> }
> }
>
> - /* remove deleted bhs */
> - if (!deleted) {
> - return ret;
> - }
> -
> - if (qemu_lockcnt_dec_if_lock(&ctx->list_lock)) {
> - bhp = &ctx->first_bh;
> - while (*bhp) {
> - bh = *bhp;
> - if (bh->deleted && !bh->scheduled) {
> - *bhp = bh->next;
> - g_free(bh);
> - } else {
> - bhp = &bh->next;
> - }
> - }
> - qemu_lockcnt_inc_and_unlock(&ctx->list_lock);
> - }
> return ret;
> }
>
> void qemu_bh_schedule_idle(QEMUBH *bh)
> {
> - bh->idle = 1;
> - /* Make sure that idle & any writes needed by the callback are done
> - * before the locations are read in the aio_bh_poll.
> - */
> - atomic_mb_set(&bh->scheduled, 1);
> + aio_bh_enqueue(bh, BH_SCHEDULED | BH_IDLE);
> }
>
> void qemu_bh_schedule(QEMUBH *bh)
> {
> - AioContext *ctx;
> -
> - ctx = bh->ctx;
> - bh->idle = 0;
> - /* The memory barrier implicit in atomic_xchg makes sure that:
> - * 1. idle & any writes needed by the callback are done before the
> - * locations are read in the aio_bh_poll.
> - * 2. ctx is loaded before scheduled is set and the callback has a chance
> - * to execute.
> - */
> - if (atomic_xchg(&bh->scheduled, 1) == 0) {
> - aio_notify(ctx);
> - }
> + aio_bh_enqueue(bh, BH_SCHEDULED);
> }
>
> -
> /* This func is async.
> */
> void qemu_bh_cancel(QEMUBH *bh)
> {
> - atomic_mb_set(&bh->scheduled, 0);
> + atomic_and(&bh->flags, ~BH_SCHEDULED);
> }
>
> /* This func is async.The bottom half will do the delete action at the finial
> @@ -181,21 +192,16 @@ void qemu_bh_cancel(QEMUBH *bh)
> */
> void qemu_bh_delete(QEMUBH *bh)
> {
> - bh->scheduled = 0;
> - bh->deleted = 1;
> + aio_bh_enqueue(bh, BH_DELETED);
> }
>
> -int64_t
> -aio_compute_timeout(AioContext *ctx)
> +static int64_t aio_compute_bh_timeout(BHList *head, int timeout)
> {
> - int64_t deadline;
> - int timeout = -1;
> QEMUBH *bh;
>
> - for (bh = atomic_rcu_read(&ctx->first_bh); bh;
> - bh = atomic_rcu_read(&bh->next)) {
> - if (bh->scheduled) {
> - if (bh->idle) {
> + QSLIST_FOREACH(bh, head, next) {
> + if ((bh->flags & (BH_SCHEDULED | BH_DELETED)) == BH_SCHEDULED) {
> + if (bh->flags & BH_IDLE) {
> /* idle bottom halves will be polled at least
> * every 10ms */
> timeout = 10000000;
> @@ -207,6 +213,28 @@ aio_compute_timeout(AioContext *ctx)
> }
> }
>
> + return timeout;
> +}
> +
> +int64_t
> +aio_compute_timeout(AioContext *ctx)
> +{
> + BHListSlice *s;
> + int64_t deadline;
> + int timeout = -1;
> +
> + timeout = aio_compute_bh_timeout(&ctx->bh_list, timeout);
> + if (timeout == 0) {
> + return 0;
> + }
> +
> + QSIMPLEQ_FOREACH(s, &ctx->bh_slice_list, next) {
> + timeout = aio_compute_bh_timeout(&s->bh_list, timeout);
> + if (timeout == 0) {
> + return 0;
> + }
> + }
> +
> deadline = timerlistgroup_deadline_ns(&ctx->tlg);
> if (deadline == 0) {
> return 0;
> @@ -237,15 +265,24 @@ aio_ctx_check(GSource *source)
> {
> AioContext *ctx = (AioContext *) source;
> QEMUBH *bh;
> + BHListSlice *s;
>
> atomic_and(&ctx->notify_me, ~1);
> aio_notify_accept(ctx);
>
> - for (bh = ctx->first_bh; bh; bh = bh->next) {
> - if (bh->scheduled) {
> + QSLIST_FOREACH(bh, &ctx->bh_list, next) {
> + if ((bh->flags & (BH_SCHEDULED | BH_DELETED)) == BH_SCHEDULED) {
> return true;
> }
> }
> +
> + QSIMPLEQ_FOREACH(s, &ctx->bh_slice_list, next) {
> + QSLIST_FOREACH(bh, &s->bh_list, next) {
> + if ((bh->flags & (BH_SCHEDULED | BH_DELETED)) == BH_SCHEDULED) {
> + return true;
> + }
> + }
> + }
> return aio_pending(ctx) || (timerlistgroup_deadline_ns(&ctx->tlg) == 0);
> }
>
> @@ -265,6 +302,8 @@ static void
> aio_ctx_finalize(GSource *source)
> {
> AioContext *ctx = (AioContext *) source;
> + QEMUBH *bh;
> + unsigned flags;
>
> thread_pool_free(ctx->thread_pool);
>
> @@ -287,18 +326,15 @@ aio_ctx_finalize(GSource *source)
> assert(QSLIST_EMPTY(&ctx->scheduled_coroutines));
> qemu_bh_delete(ctx->co_schedule_bh);
>
> - qemu_lockcnt_lock(&ctx->list_lock);
> - assert(!qemu_lockcnt_count(&ctx->list_lock));
> - while (ctx->first_bh) {
> - QEMUBH *next = ctx->first_bh->next;
> + /* There must be no aio_bh_poll() calls going on */
> + assert(QSIMPLEQ_EMPTY(&ctx->bh_slice_list));
>
> + while ((bh = aio_bh_dequeue(&ctx->bh_list, &flags))) {
> /* qemu_bh_delete() must have been called on BHs in this AioContext */
> - assert(ctx->first_bh->deleted);
> + assert(flags & BH_DELETED);
>
> - g_free(ctx->first_bh);
> - ctx->first_bh = next;
> + g_free(bh);
> }
> - qemu_lockcnt_unlock(&ctx->list_lock);
>
> aio_set_event_notifier(ctx, &ctx->notifier, false, NULL, NULL);
> event_notifier_cleanup(&ctx->notifier);
> @@ -445,6 +481,8 @@ AioContext *aio_context_new(Error **errp)
> AioContext *ctx;
>
> ctx = (AioContext *) g_source_new(&aio_source_funcs, sizeof(AioContext));
> + QSLIST_INIT(&ctx->bh_list);
> + QSIMPLEQ_INIT(&ctx->bh_slice_list);
> aio_context_setup(ctx);
>
> ret = event_notifier_init(&ctx->notifier, false);
>
^ permalink raw reply
* Re: [dpdk-dev] [PATCH] examples/ipsec-secgw: extend inline session to non AES-GCM
From: Ananyev, Konstantin @ 2020-02-20 10:41 UTC (permalink / raw)
To: Akhil Goyal, Iremonger, Bernard
Cc: dev@dpdk.org, stable@dpdk.org, Ankur Dwivedi,
Narayana Prasad Raju Athreya, Anoob Joseph
In-Reply-To: <VE1PR04MB66398C838477113CC1FCD453E6130@VE1PR04MB6639.eurprd04.prod.outlook.com>
Hi Akhil,
> > > This patch extends creation of inline session to all the algorithms.
> > > Previously the inline session was enabled only for AES-GCM cipher.
> > >
> > > Fixes: 3a690d5a65e2 ("examples/ipsec-secgw: fix first packet with inline
> > > crypto")
> > >
> > > Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
> > > ---
> > > examples/ipsec-secgw/sa.c | 26 ++++++++++++--------------
> > > 1 file changed, 12 insertions(+), 14 deletions(-)
> >
> > Acked-by: Anoob Joseph <anoobj@marvell.com>
> Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
>
> Hi Konstantin,
>
> I am about to merge this patch. Do you have any issues on this.
No objections, LGTM.
Konstantin
^ permalink raw reply
* Re: [PATCH v2 RESEND] mm/sparsemem: pfn_to_page is not valid yet on SPARSEMEM
From: Michal Hocko @ 2020-02-20 10:41 UTC (permalink / raw)
To: Baoquan He
Cc: linux-kernel, linux-mm, akpm, richardw.yang, david, osalvador,
dan.j.williams
In-Reply-To: <20200219030454.4844-1-bhe@redhat.com>
On Wed 19-02-20 11:04:54, Baoquan He wrote:
> From: Wei Yang <richardw.yang@linux.intel.com>
>
> When we use SPARSEMEM instead of SPARSEMEM_VMEMMAP, pfn_to_page()
> doesn't work before sparse_init_one_section() is called. This leads to a
> crash when hotplug memory:
>
> [ 41.839170] BUG: unable to handle page fault for address: 0000000006400000
> [ 41.840663] #PF: supervisor write access in kernel mode
> [ 41.841822] #PF: error_code(0x0002) - not-present page
> [ 41.842970] PGD 0 P4D 0
> [ 41.843538] Oops: 0002 [#1] SMP PTI
> [ 41.844125] CPU: 3 PID: 221 Comm: kworker/u16:1 Tainted: G W 5.5.0-next-20200205+ #343
> [ 41.845659] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 0.0.0 02/06/2015
> [ 41.846977] Workqueue: kacpi_hotplug acpi_hotplug_work_fn
> [ 41.847904] RIP: 0010:__memset+0x24/0x30
> [ 41.848660] Code: cc cc cc cc cc cc 0f 1f 44 00 00 49 89 f9 48 89 d1 83 e2 07 48 c1 e9 03 40 0f b6 f6 48 b8 01 01 01 01 01 01 01 01 48 0f af c6 <f3> 48 ab 89 d1 f3 aa 4c 89 c8 c3 90 49 89 f9 40 88 f0 48 89 d1 f3
> [ 41.851836] RSP: 0018:ffffb43ac0373c80 EFLAGS: 00010a87
> [ 41.852686] RAX: ffffffffffffffff RBX: ffff8a1518800000 RCX: 0000000000050000
> [ 41.853824] RDX: 0000000000000000 RSI: 00000000000000ff RDI: 0000000006400000
> [ 41.854967] RBP: 0000000000140000 R08: 0000000000100000 R09: 0000000006400000
> [ 41.856107] R10: 0000000000000000 R11: 0000000000000002 R12: 0000000000000000
> [ 41.857255] R13: 0000000000000028 R14: 0000000000000000 R15: ffff8a153ffd9280
> [ 41.858414] FS: 0000000000000000(0000) GS:ffff8a153ab00000(0000) knlGS:0000000000000000
> [ 41.859703] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 41.860627] CR2: 0000000006400000 CR3: 0000000136fca000 CR4: 00000000000006e0
> [ 41.861716] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 41.862680] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [ 41.863628] Call Trace:
> [ 41.863983] sparse_add_section+0x1c9/0x26a
> [ 41.864570] __add_pages+0xbf/0x150
> [ 41.865057] add_pages+0x12/0x60
> [ 41.865489] add_memory_resource+0xc8/0x210
> [ 41.866017] ? wake_up_q+0xa0/0xa0
> [ 41.866416] __add_memory+0x62/0xb0
> [ 41.866825] acpi_memory_device_add+0x13f/0x300
> [ 41.867410] acpi_bus_attach+0xf6/0x200
> [ 41.867890] acpi_bus_scan+0x43/0x90
> [ 41.868448] acpi_device_hotplug+0x275/0x3d0
> [ 41.868972] acpi_hotplug_work_fn+0x1a/0x30
> [ 41.869473] process_one_work+0x1a7/0x370
> [ 41.869953] worker_thread+0x30/0x380
> [ 41.870396] ? flush_rcu_work+0x30/0x30
> [ 41.870846] kthread+0x112/0x130
> [ 41.871236] ? kthread_create_on_node+0x60/0x60
> [ 41.871770] ret_from_fork+0x35/0x40
>
> We should use memmap as it did.
>
> Fixes: ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug")
> Signed-off-by: Wei Yang <richardw.yang@linux.intel.com>
> Acked-by: David Hildenbrand <david@redhat.com>
> Reviewed-by: Baoquan He <bhe@redhat.com>
> CC: Dan Williams <dan.j.williams@intel.com>
Acked-by: Michal Hocko <mhocko@suse.com>
> ---
> v1->v2:
> Add trace printed by triggered bug to log.
>
> v1 can be found here:
> http://lkml.kernel.org/r/20200206125343.9070-1-richardw.yang@linux.intel.com
>
> mm/sparse.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/mm/sparse.c b/mm/sparse.c
> index c184b69460b7..596b2a45b100 100644
> --- a/mm/sparse.c
> +++ b/mm/sparse.c
> @@ -876,7 +876,7 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn,
> * Poison uninitialized struct pages in order to catch invalid flags
> * combinations.
> */
> - page_init_poison(pfn_to_page(start_pfn), sizeof(struct page) * nr_pages);
> + page_init_poison(memmap, sizeof(struct page) * nr_pages);
>
> ms = __nr_to_section(section_nr);
> set_section_nid(section_nr, nid);
> --
> 2.17.2
>
--
Michal Hocko
SUSE Labs
^ permalink raw reply
* Re: [PATCH v3 05/17] s390x: protvirt: Support unpack facility
From: Cornelia Huck @ 2020-02-20 10:39 UTC (permalink / raw)
To: Janosch Frank; +Cc: qemu-s390x, mihajlov, qemu-devel, david
In-Reply-To: <20200214151636.8764-6-frankja@linux.ibm.com>
On Fri, 14 Feb 2020 10:16:24 -0500
Janosch Frank <frankja@linux.ibm.com> wrote:
> When a guest has saved a ipib of type 5 and call diagnose308 with
s/call/calls/
> subcode 10, we have to setup the protected processing environment via
> Ultravisor calls. The calls are done by KVM and are exposed via an API.
>
> The following steps are necessary:
> 1. Create a VM (register it with the Ultravisor)
> 2. Create secure CPUs for all of our current cpus
> 3. Forward the secure header to the Ultravisor (has all information on
> how to decrypt the image and VM information)
> 4. Protect image pages from the host and decrypt them
> 5. Verify the image integrity
>
> Only after step 5 a protected VM is allowed to run.
>
> Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com> [Changes
> to machine]
> ---
> hw/s390x/Makefile.objs | 1 +
> hw/s390x/ipl.c | 32 ++++++
> hw/s390x/ipl.h | 2 +
> hw/s390x/pv.c | 154 ++++++++++++++++++++++++++++
> hw/s390x/pv.h | 38 +++++++
> hw/s390x/s390-virtio-ccw.c | 79 ++++++++++++++
> include/hw/s390x/s390-virtio-ccw.h | 1 +
> target/s390x/cpu.c | 4 +
> target/s390x/cpu.h | 1 +
> target/s390x/cpu_features_def.inc.h | 1 +
> 10 files changed, 313 insertions(+)
> create mode 100644 hw/s390x/pv.c
> create mode 100644 hw/s390x/pv.h
(...)
> diff --git a/hw/s390x/pv.c b/hw/s390x/pv.c
> new file mode 100644
> index 0000000000..5b6a26cba9
> --- /dev/null
> +++ b/hw/s390x/pv.c
> @@ -0,0 +1,154 @@
> +/*
> + * Secure execution functions
> + *
> + * Copyright IBM Corp. 2019
Update the year?
> + * Author(s):
> + * Janosch Frank <frankja@linux.ibm.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or (at
> + * your option) any later version. See the COPYING file in the top-level
> + * directory.
> + */
(...)
> +void s390_pv_vm_destroy(void)
> +{
> + s390_pv_cmd_exit(KVM_PV_VM_DESTROY, NULL);
Why does this exit()? Should Never Happen?
> +}
> +
> +int s390_pv_vcpu_create(CPUState *cs)
> +{
> + int rc;
> +
> + rc = s390_pv_cmd_vcpu(cs, KVM_PV_VCPU_CREATE, NULL);
> + if (!rc) {
> + S390_CPU(cs)->env.pv = true;
> + }
> +
> + return rc;
> +}
> +
> +void s390_pv_vcpu_destroy(CPUState *cs)
> +{
> + s390_pv_cmd_vcpu_exit(cs, KVM_PV_VCPU_DESTROY, NULL);
dito
> + S390_CPU(cs)->env.pv = false;
> +}
(...)
> +void s390_pv_perf_clear_reset(void)
> +{
> + s390_pv_cmd_exit(KVM_PV_VM_PREP_RESET, NULL);
And here. Or is that because the machine should not be left around in
an undefined state?
> +}
> +
> +int s390_pv_verify(void)
> +{
> + return s390_pv_cmd(KVM_PV_VM_VERIFY, NULL);
> +}
> +
> +void s390_pv_unshare(void)
> +{
> + s390_pv_cmd_exit(KVM_PV_VM_UNSHARE_ALL, NULL);
> +}
> diff --git a/hw/s390x/pv.h b/hw/s390x/pv.h
> new file mode 100644
> index 0000000000..7d20bdd12e
> --- /dev/null
> +++ b/hw/s390x/pv.h
> @@ -0,0 +1,38 @@
> +/*
> + * Protected Virtualization header
> + *
> + * Copyright IBM Corp. 2019
Year++
> + * Author(s):
> + * Janosch Frank <frankja@linux.ibm.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or (at
> + * your option) any later version. See the COPYING file in the top-level
> + * directory.
> + */
> +
> +#ifndef HW_S390_PV_H
> +#define HW_S390_PV_H
> +
> +#ifdef CONFIG_KVM
> +int s390_pv_vm_create(void);
> +void s390_pv_vm_destroy(void);
> +void s390_pv_vcpu_destroy(CPUState *cs);
> +int s390_pv_vcpu_create(CPUState *cs);
> +int s390_pv_set_sec_parms(uint64_t origin, uint64_t length);
> +int s390_pv_unpack(uint64_t addr, uint64_t size, uint64_t tweak);
> +void s390_pv_perf_clear_reset(void);
> +int s390_pv_verify(void);
> +void s390_pv_unshare(void);
> +#else
> +int s390_pv_vm_create(void) { return 0; }
I'm wondering why you return 0 here (and below). These function should
not be called for !KVM, but just to help catch logic error, use -EINVAL
or so?
> +void s390_pv_vm_destroy(void) {}
> +void s390_pv_vcpu_destroy(CPUState *cs) {}
> +int s390_pv_vcpu_create(CPUState *cs) { return 0; }
> +int s390_pv_set_sec_parms(uint64_t origin, uint64_t length) { return 0; }
> +int s390_pv_unpack(uint64_t addr, uint64_t size, uint64_t tweak) { return 0: }
> +void s390_pv_perf_clear_reset(void) {}
> +int s390_pv_verify(void) { return 0; }
> +void s390_pv_unshare(void) {}
> +#endif
> +
> +#endif /* HW_S390_PV_H */
> diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
> index e759eb5f83..5fa4372083 100644
> --- a/hw/s390x/s390-virtio-ccw.c
> +++ b/hw/s390x/s390-virtio-ccw.c
> @@ -41,6 +41,7 @@
> #include "hw/qdev-properties.h"
> #include "hw/s390x/tod.h"
> #include "sysemu/sysemu.h"
> +#include "hw/s390x/pv.h"
>
> S390CPU *s390_cpu_addr2state(uint16_t cpu_addr)
> {
> @@ -240,9 +241,11 @@ static void s390_create_sclpconsole(const char *type, Chardev *chardev)
> static void ccw_init(MachineState *machine)
> {
> int ret;
> + S390CcwMachineState *ms = S390_CCW_MACHINE(machine);
> VirtualCssBus *css_bus;
> DeviceState *dev;
>
> + ms->pv = false;
I'm wondering why you need to init this to false - isn't it already
zeroed out?
> s390_sclp_init();
> /* init memory + setup max page size. Required for the CPU model */
> s390_memory_init(machine->ram_size);
> @@ -318,10 +321,58 @@ static inline void s390_do_cpu_ipl(CPUState *cs, run_on_cpu_data arg)
> s390_cpu_set_state(S390_CPU_STATE_OPERATING, cpu);
> }
>
> +static int s390_machine_pv_secure(S390CcwMachineState *ms)
> +{
> + CPUState *t;
> + int rc;
> +
> + /* Create SE VM */
> + rc = s390_pv_vm_create();
> + if (rc) {
> + return rc;
> + }
> +
> + CPU_FOREACH(t) {
> + rc = s390_pv_vcpu_create(t);
> + if (rc) {
> + return rc;
No need to undo something on error?
> + }
> + }
> +
> + ms->pv = true;
> +
> + /* Set SE header and unpack */
> + rc = s390_ipl_prepare_pv_header();
> + if (rc) {
> + return rc;
Also here.
> + }
> +
> + /* Decrypt image */
> + rc = s390_ipl_pv_unpack();
> + if (rc) {
> + return rc;
And here.
> + }
> +
> + /* Verify integrity */
> + rc = s390_pv_verify();
> + return rc;
And here.
> +}
(...)
> diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
> index 8da1905485..1dbd84b9d7 100644
> --- a/target/s390x/cpu.c
> +++ b/target/s390x/cpu.c
> @@ -37,6 +37,8 @@
> #include "sysemu/hw_accel.h"
> #include "hw/qdev-properties.h"
> #ifndef CONFIG_USER_ONLY
> +#include "hw/s390x/s390-virtio-ccw.h"
> +#include "hw/s390x/pv.h"
> #include "hw/boards.h"
> #include "sysemu/arch_init.h"
> #include "sysemu/sysemu.h"
> @@ -191,6 +193,7 @@ static void s390_cpu_realizefn(DeviceState *dev, Error **errp)
>
> #if !defined(CONFIG_USER_ONLY)
> MachineState *ms = MACHINE(qdev_get_machine());
> + S390CcwMachineState *ccw = S390_CCW_MACHINE(ms);
I find the variable name a bit confusing... maybe ccw_ms?
> unsigned int max_cpus = ms->smp.max_cpus;
> if (cpu->env.core_id >= max_cpus) {
> error_setg(&err, "Unable to add CPU with core-id: %" PRIu32
> @@ -205,6 +208,7 @@ static void s390_cpu_realizefn(DeviceState *dev, Error **errp)
> goto out;
> }
>
> + cpu->env.pv = ccw->pv;
So, if you add a cpu, it will inherit the pv state of the machine...
doesn't it need any setup?
> /* sync cs->cpu_index and env->core_id. The latter is needed for TCG. */
> cs->cpu_index = cpu->env.core_id;
> #endif
(...)
^ permalink raw reply
* [PATCH v3 00/37] KVM: s390: Add support for protected VMs
From: Christian Borntraeger @ 2020-02-20 10:39 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank, Andrew Morton
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik, Andrea Arcangeli, linux-mm
mm people: This series contains a "pretty small" common code memory
management change that will allow paging, guest backing with files etc
almost just like normal VMs. It should be a no-op for all architectures
not opting in. And it should be usable for others that also try to get
notified on "the pages are in the process of being used for things like
I/O". This time I included error handling and an ACK from Will Deacon.
mm-related patches CCed on linux-mm, the complete list can be found on
the KVM and linux-s390 list.
Andrew, any chance to either take " mm:gup/writeback: add callbacks for
inaccessible pages" or ACK so that I can take it?
Overview
--------
Protected VMs (PVM) are KVM VMs, where KVM can't access the VM's state
like guest memory and guest registers anymore. Instead the PVMs are
mostly managed by a new entity called Ultravisor (UV), which provides
an API, so KVM and the PV can request management actions.
PVMs are encrypted at rest and protected from hypervisor access while
running. They switch from a normal operation into protected mode, so
we can still use the standard boot process to load a encrypted blob
and then move it into protected mode.
Rebooting is only possible by passing through the unprotected/normal
mode and switching to protected again.
All patches are in the protvirtv4 branch of the korg s390 kvm git
https://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux.git/log/?h=protvirtv5
Claudio presented the technology at his presentation at KVM Forum
2019.
https://static.sched.com/hosted_files/kvmforum2019/3b/ibm_protected_vms_s390x.pdf
v2 -> v3
- rebase against v5.6-rc2
- move some checks into the callers
- typo fixes
- extend UV query size
- do a tlb flush when entering/exiting protected mode
- more comments
- change interface to PV_ENABLE/DISABLE instead of vcpu/vm
create/destroy
- lockdep checks for *is_protected calls
- locking improments
- move facility 161 to qemu
- checkpatch fixes
- merged error handling in mm patch
- removed vcpu pv commands
- use mp_state for setting the IPL PSW
v1 -> v2
- rebase on top of kvm/master
- pipe through rc and rrc. This might have created some churn here and
there
- turn off sclp masking when rebooting into "unsecure"
- memory management simplification
- prefix page handling now via intercept 112
- io interrupt intervention request fix (do not use GISA)
- api.txt conversion to rst
- sample patches on top of mm/gup/writeback
- tons of review feedback
- kvm_uv debug feature fixes and unifications
- ultravisor information for /sys/firmware
-
RFCv2 -> v1 (you can diff the protvirtv2 and the protvirtv3 branch)
- tons of review feedback integrated (see mail thread)
- memory management now complete and working
- Documentation patches merged
- interrupt patches merged
- CONFIG_KVM_S390_PROTECTED_VIRTUALIZATION_HOST removed
- SIDA interface integrated into memop
- for merged patches I removed reviews that were not in all patches
Christian Borntraeger (5):
KVM: s390/mm: Make pages accessible before destroying the guest
KVM: s390: protvirt: Add SCLP interrupt handling
KVM: s390: protvirt: do not inject interrupts after start
KVM: s390: rstify new ioctls in api.rst
KVM: s390: protvirt: introduce and enable KVM_CAP_S390_PROTECTED
Claudio Imbrenda (3):
mm:gup/writeback: add callbacks for inaccessible pages
s390/mm: provide memory management functions for protected KVM guests
KVM: s390/mm: handle guest unpin events
Janosch Frank (24):
KVM: s390: protvirt: Add UV debug trace
KVM: s390: add new variants of UV CALL
KVM: s390: protvirt: Add initial vm and cpu lifecycle handling
KVM: s390: protvirt: Add KVM api documentation
KVM: s390: protvirt: Secure memory is not mergeable
KVM: s390: protvirt: Handle SE notification interceptions
KVM: s390: protvirt: Instruction emulation
KVM: s390: protvirt: Handle spec exception loops
KVM: s390: protvirt: Add new gprs location handling
KVM: S390: protvirt: Introduce instruction data area bounce buffer
KVM: s390: protvirt: handle secure guest prefix pages
KVM: s390: protvirt: Write sthyi data to instruction data area
KVM: s390: protvirt: STSI handling
KVM: s390: protvirt: disallow one_reg
KVM: s390: protvirt: Do only reset registers that are accessible
KVM: s390: protvirt: Only sync fmt4 registers
KVM: s390: protvirt: Add program exception injection
KVM: s390: protvirt: UV calls in support of diag308 0, 1
KVM: s390: protvirt: Report CPU state to Ultravisor
KVM: s390: protvirt: Support cmd 5 operation state
KVM: s390: protvirt: Mask PSW interrupt bits for interception 104 and
112
KVM: s390: protvirt: Add UV cpu reset calls
DOCUMENTATION: Protected virtual machine introduction and IPL
s390: protvirt: Add sysfs firmware interface for Ultravisor
information
Michael Mueller (1):
KVM: s390: protvirt: Implement interrupt injection
Ulrich Weigand (1):
KVM: s390/interrupt: do not pin adapter interrupt pages
Vasily Gorbik (3):
s390/protvirt: introduce host side setup
s390/protvirt: add ultravisor initialization
s390/mm: add (non)secure page access exceptions handlers
.../admin-guide/kernel-parameters.txt | 5 +
Documentation/virt/kvm/api.rst | 91 +++-
Documentation/virt/kvm/devices/s390_flic.rst | 11 +-
Documentation/virt/kvm/index.rst | 2 +
Documentation/virt/kvm/s390-pv-boot.rst | 83 +++
Documentation/virt/kvm/s390-pv.rst | 116 ++++
MAINTAINERS | 1 +
arch/s390/boot/Makefile | 2 +-
arch/s390/boot/uv.c | 21 +-
arch/s390/include/asm/gmap.h | 6 +
arch/s390/include/asm/kvm_host.h | 113 +++-
arch/s390/include/asm/mmu.h | 2 +
arch/s390/include/asm/mmu_context.h | 1 +
arch/s390/include/asm/page.h | 5 +
arch/s390/include/asm/pgtable.h | 35 +-
arch/s390/include/asm/uv.h | 252 ++++++++-
arch/s390/kernel/Makefile | 1 +
arch/s390/kernel/pgm_check.S | 4 +-
arch/s390/kernel/setup.c | 9 +-
arch/s390/kernel/uv.c | 413 ++++++++++++++
arch/s390/kvm/Makefile | 2 +-
arch/s390/kvm/diag.c | 4 +
arch/s390/kvm/intercept.c | 115 +++-
arch/s390/kvm/interrupt.c | 399 ++++++++------
arch/s390/kvm/kvm-s390.c | 509 +++++++++++++++---
arch/s390/kvm/kvm-s390.h | 51 +-
arch/s390/kvm/priv.c | 11 +-
arch/s390/kvm/pv.c | 286 ++++++++++
arch/s390/mm/fault.c | 78 +++
arch/s390/mm/gmap.c | 65 ++-
include/linux/gfp.h | 6 +
include/uapi/linux/kvm.h | 43 +-
mm/gup.c | 15 +-
mm/page-writeback.c | 5 +
34 files changed, 2442 insertions(+), 320 deletions(-)
create mode 100644 Documentation/virt/kvm/s390-pv-boot.rst
create mode 100644 Documentation/virt/kvm/s390-pv.rst
create mode 100644 arch/s390/kernel/uv.c
create mode 100644 arch/s390/kvm/pv.c
--
2.25.0
^ permalink raw reply
* [PATCH v3 34/37] DOCUMENTATION: Protected virtual machine introduction and IPL
From: Christian Borntraeger @ 2020-02-20 10:40 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik, Janosch Frank
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
From: Janosch Frank <frankja@linux.ibm.com>
Add documentation about protected KVM guests and description of changes
that are necessary to move a KVM VM into Protected Virtualization mode.
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
[borntraeger@de.ibm.com: fixing and conversion to rst]
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
Documentation/virt/kvm/index.rst | 2 +
Documentation/virt/kvm/s390-pv-boot.rst | 83 +++++++++++++++++
Documentation/virt/kvm/s390-pv.rst | 116 ++++++++++++++++++++++++
MAINTAINERS | 1 +
4 files changed, 202 insertions(+)
create mode 100644 Documentation/virt/kvm/s390-pv-boot.rst
create mode 100644 Documentation/virt/kvm/s390-pv.rst
diff --git a/Documentation/virt/kvm/index.rst b/Documentation/virt/kvm/index.rst
index 774deaebf7fa..dcc252634cf9 100644
--- a/Documentation/virt/kvm/index.rst
+++ b/Documentation/virt/kvm/index.rst
@@ -18,6 +18,8 @@ KVM
nested-vmx
ppc-pv
s390-diag
+ s390-pv
+ s390-pv-boot
timekeeping
vcpu-requests
diff --git a/Documentation/virt/kvm/s390-pv-boot.rst b/Documentation/virt/kvm/s390-pv-boot.rst
new file mode 100644
index 000000000000..b762df206ab7
--- /dev/null
+++ b/Documentation/virt/kvm/s390-pv-boot.rst
@@ -0,0 +1,83 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+======================================
+s390 (IBM Z) Boot/IPL of Protected VMs
+======================================
+
+Summary
+-------
+The memory of Protected Virtual Machines (PVMs) is not accessible to
+I/O or the hypervisor. In those cases where the hypervisor needs to
+access the memory of a PVM, that memory must be made accessible.
+Memory made accessible to the hypervisor will be encrypted. See
+:doc:`s390-pv` for details."
+
+On IPL (boot) a small plaintext bootloader is started, which provides
+information about the encrypted components and necessary metadata to
+KVM to decrypt the protected virtual machine.
+
+Based on this data, KVM will make the protected virtual machine known
+to the Ultravisor(UV) and instruct it to secure the memory of the PVM,
+decrypt the components and verify the data and address list hashes, to
+ensure integrity. Afterwards KVM can run the PVM via the SIE
+instruction which the UV will intercept and execute on KVM's behalf.
+
+As the guest image is just like an opaque kernel image that does the
+switch into PV mode itself, the user can load encrypted guest
+executables and data via every available method (network, dasd, scsi,
+direct kernel, ...) without the need to change the boot process.
+
+
+Diag308
+-------
+This diagnose instruction is the basic mechanism to handle IPL and
+related operations for virtual machines. The VM can set and retrieve
+IPL information blocks, that specify the IPL method/devices and
+request VM memory and subsystem resets, as well as IPLs.
+
+For PVMs this concept has been extended with new subcodes:
+
+Subcode 8: Set an IPL Information Block of type 5 (information block
+for PVMs)
+Subcode 9: Store the saved block in guest memory
+Subcode 10: Move into Protected Virtualization mode
+
+The new PV load-device-specific-parameters field specifies all data
+that is necessary to move into PV mode.
+
+* PV Header origin
+* PV Header length
+* List of Components composed of
+ * AES-XTS Tweak prefix
+ * Origin
+ * Size
+
+The PV header contains the keys and hashes, which the UV will use to
+decrypt and verify the PV, as well as control flags and a start PSW.
+
+The components are for instance an encrypted kernel, kernel parameters
+and initrd. The components are decrypted by the UV.
+
+After the initial import of the encrypted data, all defined pages will
+contain the guest content. All non-specified pages will start out as
+zero pages on first access.
+
+
+When running in protected virtualization mode, some subcodes will result in
+exceptions or return error codes.
+
+Subcodes 4 and 7, which specify operations that do not clear the guest
+memory, will result in specification exceptions. This is because the
+UV will clear all memory when a secure VM is removed, and therefore
+non-clearing IPL subcodes are not allowed."
+
+Subcodes 8, 9, 10 will result in specification exceptions.
+Re-IPL into a protected mode is only possible via a detour into non
+protected mode.
+
+Keys
+----
+Every CEC will have a unique public key to enable tooling to build
+encrypted images.
+See `s390-tools <https://github.com/ibm-s390-tools/s390-tools/>`_
+for the tooling.
diff --git a/Documentation/virt/kvm/s390-pv.rst b/Documentation/virt/kvm/s390-pv.rst
new file mode 100644
index 000000000000..27fe03eaeaad
--- /dev/null
+++ b/Documentation/virt/kvm/s390-pv.rst
@@ -0,0 +1,116 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=========================================
+s390 (IBM Z) Ultravisor and Protected VMs
+=========================================
+
+Summary
+-------
+Protected virtual machines (PVM) are KVM VMs that do not allow KVM to
+access VM state like guest memory or guest registers. Instead, the
+PVMs are mostly managed by a new entity called Ultravisor (UV). The UV
+provides an API that can be used by PVMs and KVM to request management
+actions.
+
+Each guest starts in the non-protected mode and then may make a
+request to transition into protected mode. On transition, KVM
+registers the guest and its VCPUs with the Ultravisor and prepares
+everything for running it.
+
+The Ultravisor will secure and decrypt the guest's boot memory
+(i.e. kernel/initrd). It will safeguard state changes like VCPU
+starts/stops and injected interrupts while the guest is running.
+
+As access to the guest's state, such as the SIE state description, is
+normally needed to be able to run a VM, some changes have been made in
+the behavior of the SIE instruction. A new format 4 state description
+has been introduced, where some fields have different meanings for a
+PVM. SIE exits are minimized as much as possible to improve speed and
+reduce exposed guest state.
+
+
+Interrupt injection
+-------------------
+Interrupt injection is safeguarded by the Ultravisor. As KVM doesn't
+have access to the VCPUs' lowcores, injection is handled via the
+format 4 state description.
+
+Machine check, external, IO and restart interruptions each can be
+injected on SIE entry via a bit in the interrupt injection control
+field (offset 0x54). If the guest cpu is not enabled for the interrupt
+at the time of injection, a validity interception is recognized. The
+format 4 state description contains fields in the interception data
+block where data associated with the interrupt can be transported.
+
+Program and Service Call exceptions have another layer of
+safeguarding; they can only be injected for instructions that have
+been intercepted into KVM. The exceptions need to be a valid outcome
+of an instruction emulation by KVM, e.g. we can never inject a
+addressing exception as they are reported by SIE since KVM has no
+access to the guest memory.
+
+
+Mask notification interceptions
+-------------------------------
+In order to be notified when a PVM enables a certain class of
+interrupt, KVM cannot intercept lctl(g) and lpsw(e) anymore. As a
+replacement, two new interception codes have been introduced: One
+indicating that the contents of CRs 0, 6, or 14 have been changed,
+indicating different interruption subclasses; and one indicating that
+PSW bit 13 has been changed, indicating that a machine check
+intervention was requested and those are now enabled.
+
+Instruction emulation
+---------------------
+With the format 4 state description for PVMs, the SIE instruction already
+interprets more instructions than it does with format 2. It is not able
+to interpret every instruction, but needs to hand some tasks to KVM;
+therefore, the SIE and the ultravisor safeguard emulation inputs and outputs.
+
+The control structures associated with SIE provide the Secure
+Instruction Data Area (SIDA), the Interception Parameters (IP) and the
+Secure Interception General Register Save Area. Guest GRs and most of
+the instruction data, such as I/O data structures, are filtered.
+Instruction data is copied to and from the Secure Instruction Data
+Area (SIDA) when needed. Guest GRs are put into / retrieved from the
+Secure Interception General Register Save Area.
+
+Only GR values needed to emulate an instruction will be copied into this
+save area and the real register numbers will be hidden.
+
+The Interception Parameters state description field still contains the
+the bytes of the instruction text, but with pre-set register values
+instead of the actual ones. I.e. each instruction always uses the same
+instruction text, in order not to leak guest instruction text.
+This also implies that the register content that a guest had in r<n>
+may be in r<m> from the hypervisor's point of view.
+
+The Secure Instruction Data Area contains instruction storage
+data. Instruction data, i.e. data being referenced by an instruction
+like the SCCB for sclp, is moved via the SIDA. When an instruction is
+intercepted, the SIE will only allow data and program interrupts for
+this instruction to be moved to the guest via the two data areas
+discussed before. Other data is either ignored or results in validity
+interceptions.
+
+
+Instruction emulation interceptions
+-----------------------------------
+There are two types of SIE secure instruction intercepts: the normal
+and the notification type. Normal secure instruction intercepts will
+make the guest pending for instruction completion of the intercepted
+instruction type, i.e. on SIE entry it is attempted to complete
+emulation of the instruction with the data provided by KVM. That might
+be a program exception or instruction completion.
+
+The notification type intercepts inform KVM about guest environment
+changes due to guest instruction interpretation. Such an interception
+is recognized, for example, for the store prefix instruction to provide
+the new lowcore location. On SIE reentry, any KVM data in the data areas
+is ignored and execution continues as if the guest instruction had
+completed. For that reason KVM is not allowed to inject a program
+interrupt.
+
+Links
+-----
+`KVM Forum 2019 presentation <https://static.sched.com/hosted_files/kvmforum2019/3b/ibm_protected_vms_s390x.pdf>`_
diff --git a/MAINTAINERS b/MAINTAINERS
index a0d86490c2c6..97a70647c93a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9209,6 +9209,7 @@ L: kvm@vger.kernel.org
W: http://www.ibm.com/developerworks/linux/linux390/
T: git git://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux.git
S: Supported
+F: Documentation/virt/kvm/s390*
F: arch/s390/include/uapi/asm/kvm*
F: arch/s390/include/asm/gmap.h
F: arch/s390/include/asm/kvm*
--
2.25.0
^ permalink raw reply related
* [PATCH v3 37/37] KVM: s390: protvirt: introduce and enable KVM_CAP_S390_PROTECTED
From: Christian Borntraeger @ 2020-02-20 10:40 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
Now that everything is in place, we can announce the feature.
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
arch/s390/kvm/kvm-s390.c | 3 +++
include/uapi/linux/kvm.h | 1 +
2 files changed, 4 insertions(+)
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index d79ccd34b5cb..e5f823840c29 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -574,6 +574,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
case KVM_CAP_S390_BPB:
r = test_facility(82);
break;
+ case KVM_CAP_S390_PROTECTED:
+ r = is_prot_virt_host();
+ break;
default:
r = 0;
}
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 60efbbc86209..2551de8bec57 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -1015,6 +1015,7 @@ struct kvm_ppc_resize_hpt {
#define KVM_CAP_ARM_NISV_TO_USER 177
#define KVM_CAP_ARM_INJECT_EXT_DABT 178
#define KVM_CAP_S390_VCPU_RESETS 179
+#define KVM_CAP_S390_PROTECTED 180
#ifdef KVM_CAP_IRQ_ROUTING
--
2.25.0
^ permalink raw reply related
* [PATCH v3 06/37] s390/mm: add (non)secure page access exceptions handlers
From: Christian Borntraeger @ 2020-02-20 10:39 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik, Andrea Arcangeli, linux-mm, Janosch Frank
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
From: Vasily Gorbik <gor@linux.ibm.com>
Add exceptions handlers performing transparent transition of non-secure
pages to secure (import) upon guest access and secure pages to
non-secure (export) upon hypervisor access.
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
[frankja@linux.ibm.com: adding checks for failures]
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
[imbrenda@linux.ibm.com: adding a check for gmap fault]
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
[borntraeger@de.ibm.com: patch merging, splitting, fixing]
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
arch/s390/kernel/pgm_check.S | 4 +-
arch/s390/mm/fault.c | 78 ++++++++++++++++++++++++++++++++++++
2 files changed, 80 insertions(+), 2 deletions(-)
diff --git a/arch/s390/kernel/pgm_check.S b/arch/s390/kernel/pgm_check.S
index eee3a482195a..2c27907a5ffc 100644
--- a/arch/s390/kernel/pgm_check.S
+++ b/arch/s390/kernel/pgm_check.S
@@ -78,8 +78,8 @@ PGM_CHECK(do_dat_exception) /* 39 */
PGM_CHECK(do_dat_exception) /* 3a */
PGM_CHECK(do_dat_exception) /* 3b */
PGM_CHECK_DEFAULT /* 3c */
-PGM_CHECK_DEFAULT /* 3d */
-PGM_CHECK_DEFAULT /* 3e */
+PGM_CHECK(do_secure_storage_access) /* 3d */
+PGM_CHECK(do_non_secure_storage_access) /* 3e */
PGM_CHECK_DEFAULT /* 3f */
PGM_CHECK(monitor_event_exception) /* 40 */
PGM_CHECK_DEFAULT /* 41 */
diff --git a/arch/s390/mm/fault.c b/arch/s390/mm/fault.c
index 7b0bb475c166..7bd86ebc882f 100644
--- a/arch/s390/mm/fault.c
+++ b/arch/s390/mm/fault.c
@@ -38,6 +38,7 @@
#include <asm/irq.h>
#include <asm/mmu_context.h>
#include <asm/facility.h>
+#include <asm/uv.h>
#include "../kernel/entry.h"
#define __FAIL_ADDR_MASK -4096L
@@ -816,3 +817,80 @@ static int __init pfault_irq_init(void)
early_initcall(pfault_irq_init);
#endif /* CONFIG_PFAULT */
+
+#if IS_ENABLED(CONFIG_PGSTE)
+void do_secure_storage_access(struct pt_regs *regs)
+{
+ unsigned long addr = regs->int_parm_long & __FAIL_ADDR_MASK;
+ struct vm_area_struct *vma;
+ struct mm_struct *mm;
+ struct page *page;
+ int rc;
+
+ switch (get_fault_type(regs)) {
+ case USER_FAULT:
+ mm = current->mm;
+ down_read(&mm->mmap_sem);
+ vma = find_vma(mm, addr);
+ if (!vma) {
+ up_read(&mm->mmap_sem);
+ do_fault_error(regs, VM_READ | VM_WRITE, VM_FAULT_BADMAP);
+ break;
+ }
+ page = follow_page(vma, addr, FOLL_WRITE | FOLL_GET);
+ if (IS_ERR_OR_NULL(page)) {
+ up_read(&mm->mmap_sem);
+ break;
+ }
+ if (arch_make_page_accessible(page))
+ send_sig(SIGSEGV, current, 0);
+ put_page(page);
+ up_read(&mm->mmap_sem);
+ break;
+ case KERNEL_FAULT:
+ page = phys_to_page(addr);
+ if (unlikely(!try_get_page(page)))
+ break;
+ rc = arch_make_page_accessible(page);
+ put_page(page);
+ if (rc)
+ BUG();
+ break;
+ case VDSO_FAULT:
+ /* fallthrough */
+ case GMAP_FAULT:
+ /* fallthrough */
+ default:
+ do_fault_error(regs, VM_READ | VM_WRITE, VM_FAULT_BADMAP);
+ WARN_ON_ONCE(1);
+ }
+}
+NOKPROBE_SYMBOL(do_secure_storage_access);
+
+void do_non_secure_storage_access(struct pt_regs *regs)
+{
+ unsigned long gaddr = regs->int_parm_long & __FAIL_ADDR_MASK;
+ struct gmap *gmap = (struct gmap *)S390_lowcore.gmap;
+
+ if (get_fault_type(regs) != GMAP_FAULT) {
+ do_fault_error(regs, VM_READ | VM_WRITE, VM_FAULT_BADMAP);
+ WARN_ON_ONCE(1);
+ return;
+ }
+
+ if (gmap_convert_to_secure(gmap, gaddr) == -EINVAL)
+ send_sig(SIGSEGV, current, 0);
+}
+NOKPROBE_SYMBOL(do_non_secure_storage_access);
+
+#else
+void do_secure_storage_access(struct pt_regs *regs)
+{
+ default_trap_handler(regs);
+}
+
+void do_non_secure_storage_access(struct pt_regs *regs)
+{
+ default_trap_handler(regs);
+}
+#endif
--
2.25.0
^ permalink raw reply related
* [PATCH v3 01/37] mm:gup/writeback: add callbacks for inaccessible pages
From: Christian Borntraeger @ 2020-02-20 10:39 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank, Andrew Morton
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik, Andrea Arcangeli, linux-mm, Will Deacon
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
From: Claudio Imbrenda <imbrenda@linux.ibm.com>
With the introduction of protected KVM guests on s390 there is now a
concept of inaccessible pages. These pages need to be made accessible
before the host can access them.
While cpu accesses will trigger a fault that can be resolved, I/O
accesses will just fail. We need to add a callback into architecture
code for places that will do I/O, namely when writeback is started or
when a page reference is taken.
This is not only to enable paging, file backing etc, it is also
necessary to protect the host against a malicious user space. For
example a bad QEMU could simply start direct I/O on such protected
memory. We do not want userspace to be able to trigger I/O errors and
thus we the logic is "whenever somebody accesses that page (gup) or
doing I/O, make sure that this page can be accessed. When the guest
tries to access that page we will wait in the page fault handler for
writeback to have finished and for the page_ref to be the expected
value.
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
include/linux/gfp.h | 6 ++++++
mm/gup.c | 15 ++++++++++++---
mm/page-writeback.c | 5 +++++
3 files changed, 23 insertions(+), 3 deletions(-)
diff --git a/include/linux/gfp.h b/include/linux/gfp.h
index e5b817cb86e7..be2754841369 100644
--- a/include/linux/gfp.h
+++ b/include/linux/gfp.h
@@ -485,6 +485,12 @@ static inline void arch_free_page(struct page *page, int order) { }
#ifndef HAVE_ARCH_ALLOC_PAGE
static inline void arch_alloc_page(struct page *page, int order) { }
#endif
+#ifndef HAVE_ARCH_MAKE_PAGE_ACCESSIBLE
+static inline int arch_make_page_accessible(struct page *page)
+{
+ return 0;
+}
+#endif
struct page *
__alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, int preferred_nid,
diff --git a/mm/gup.c b/mm/gup.c
index 1b521e0ac1de..354bcfbd844b 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -193,6 +193,7 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
struct page *page;
spinlock_t *ptl;
pte_t *ptep, pte;
+ int ret;
/* FOLL_GET and FOLL_PIN are mutually exclusive. */
if (WARN_ON_ONCE((flags & (FOLL_PIN | FOLL_GET)) ==
@@ -250,8 +251,6 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
if (is_zero_pfn(pte_pfn(pte))) {
page = pte_page(pte);
} else {
- int ret;
-
ret = follow_pfn_pte(vma, address, ptep, flags);
page = ERR_PTR(ret);
goto out;
@@ -259,7 +258,6 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
}
if (flags & FOLL_SPLIT && PageTransCompound(page)) {
- int ret;
get_page(page);
pte_unmap_unlock(ptep, ptl);
lock_page(page);
@@ -276,6 +274,12 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
page = ERR_PTR(-ENOMEM);
goto out;
}
+ ret = arch_make_page_accessible(page);
+ if (ret) {
+ put_page(page);
+ page = ERR_PTR(ret);
+ goto out;
+ }
}
if (flags & FOLL_TOUCH) {
if ((flags & FOLL_WRITE) &&
@@ -1919,6 +1923,11 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
VM_BUG_ON_PAGE(compound_head(page) != head, page);
+ ret = arch_make_page_accessible(page);
+ if (ret) {
+ put_page(head);
+ goto pte_unmap;
+ }
SetPageReferenced(page);
pages[*nr] = page;
(*nr)++;
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 2caf780a42e7..558d7063c117 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -2807,6 +2807,11 @@ int __test_set_page_writeback(struct page *page, bool keep_write)
inc_zone_page_state(page, NR_ZONE_WRITE_PENDING);
}
unlock_page_memcg(page);
+ /*
+ * If writeback has been triggered on a page that cannot be made
+ * accessible, it is too late.
+ */
+ WARN_ON(arch_make_page_accessible(page));
return ret;
}
--
2.25.0
^ permalink raw reply related
* [PATCH v3 05/37] s390/mm: provide memory management functions for protected KVM guests
From: Christian Borntraeger @ 2020-02-20 10:39 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik, Andrea Arcangeli, linux-mm
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
From: Claudio Imbrenda <imbrenda@linux.ibm.com>
This provides the basic ultravisor calls and page table handling to cope
with secure guests:
- provide arch_make_page_accessible
- make pages accessible after unmapping of secure guests
- provide the ultravisor commands convert to/from secure
- provide the ultravisor commands pin/unpin shared
- provide callbacks to make pages secure (inacccessible)
- we check for the expected pin count to only make pages secure if the
host is not accessing them
- we fence hugetlbfs for secure pages
Co-developed-by: Ulrich Weigand <Ulrich.Weigand@de.ibm.com>
Signed-off-by: Ulrich Weigand <Ulrich.Weigand@de.ibm.com>
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
[borntraeger@de.ibm.com: patch merging, splitting, fixing]
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
arch/s390/include/asm/gmap.h | 4 +
arch/s390/include/asm/mmu.h | 2 +
arch/s390/include/asm/mmu_context.h | 1 +
arch/s390/include/asm/page.h | 5 +
arch/s390/include/asm/pgtable.h | 35 ++++-
arch/s390/include/asm/uv.h | 31 ++++
arch/s390/kernel/uv.c | 227 ++++++++++++++++++++++++++++
7 files changed, 300 insertions(+), 5 deletions(-)
diff --git a/arch/s390/include/asm/gmap.h b/arch/s390/include/asm/gmap.h
index 37f96b6f0e61..3c4926aa78f4 100644
--- a/arch/s390/include/asm/gmap.h
+++ b/arch/s390/include/asm/gmap.h
@@ -9,6 +9,7 @@
#ifndef _ASM_S390_GMAP_H
#define _ASM_S390_GMAP_H
+#include <linux/radix-tree.h>
#include <linux/refcount.h>
/* Generic bits for GMAP notification on DAT table entry changes. */
@@ -31,6 +32,7 @@
* @table: pointer to the page directory
* @asce: address space control element for gmap page table
* @pfault_enabled: defines if pfaults are applicable for the guest
+ * @guest_handle: protected virtual machine handle for the ultravisor
* @host_to_rmap: radix tree with gmap_rmap lists
* @children: list of shadow gmap structures
* @pt_list: list of all page tables used in the shadow guest address space
@@ -54,6 +56,8 @@ struct gmap {
unsigned long asce_end;
void *private;
bool pfault_enabled;
+ /* only set for protected virtual machines */
+ unsigned long guest_handle;
/* Additional data for shadow guest address spaces */
struct radix_tree_root host_to_rmap;
struct list_head children;
diff --git a/arch/s390/include/asm/mmu.h b/arch/s390/include/asm/mmu.h
index bcfb6371086f..e21b618ad432 100644
--- a/arch/s390/include/asm/mmu.h
+++ b/arch/s390/include/asm/mmu.h
@@ -16,6 +16,8 @@ typedef struct {
unsigned long asce;
unsigned long asce_limit;
unsigned long vdso_base;
+ /* The mmu context belongs to a secure guest. */
+ atomic_t is_protected;
/*
* The following bitfields need a down_write on the mm
* semaphore when they are written to. As they are only
diff --git a/arch/s390/include/asm/mmu_context.h b/arch/s390/include/asm/mmu_context.h
index 8d04e6f3f796..afa836014076 100644
--- a/arch/s390/include/asm/mmu_context.h
+++ b/arch/s390/include/asm/mmu_context.h
@@ -23,6 +23,7 @@ static inline int init_new_context(struct task_struct *tsk,
INIT_LIST_HEAD(&mm->context.gmap_list);
cpumask_clear(&mm->context.cpu_attach_mask);
atomic_set(&mm->context.flush_count, 0);
+ atomic_set(&mm->context.is_protected, 0);
mm->context.gmap_asce = 0;
mm->context.flush_mm = 0;
mm->context.compat_mm = test_thread_flag(TIF_31BIT);
diff --git a/arch/s390/include/asm/page.h b/arch/s390/include/asm/page.h
index 85e944f04c70..4ebcf891ff3c 100644
--- a/arch/s390/include/asm/page.h
+++ b/arch/s390/include/asm/page.h
@@ -153,6 +153,11 @@ static inline int devmem_is_allowed(unsigned long pfn)
#define HAVE_ARCH_FREE_PAGE
#define HAVE_ARCH_ALLOC_PAGE
+#if IS_ENABLED(CONFIG_PGSTE)
+int arch_make_page_accessible(struct page *page);
+#define HAVE_ARCH_MAKE_PAGE_ACCESSIBLE
+#endif
+
#endif /* !__ASSEMBLY__ */
#define __PAGE_OFFSET 0x0UL
diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h
index 137a3920ca36..cc7a1adacb94 100644
--- a/arch/s390/include/asm/pgtable.h
+++ b/arch/s390/include/asm/pgtable.h
@@ -19,6 +19,7 @@
#include <linux/atomic.h>
#include <asm/bug.h>
#include <asm/page.h>
+#include <asm/uv.h>
extern pgd_t swapper_pg_dir[];
extern void paging_init(void);
@@ -520,6 +521,15 @@ static inline int mm_has_pgste(struct mm_struct *mm)
return 0;
}
+static inline int mm_is_protected(struct mm_struct *mm)
+{
+#ifdef CONFIG_PGSTE
+ if (unlikely(atomic_read(&mm->context.is_protected)))
+ return 1;
+#endif
+ return 0;
+}
+
static inline int mm_alloc_pgste(struct mm_struct *mm)
{
#ifdef CONFIG_PGSTE
@@ -1061,7 +1071,12 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma,
static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
unsigned long addr, pte_t *ptep)
{
- return ptep_xchg_lazy(mm, addr, ptep, __pte(_PAGE_INVALID));
+ pte_t res;
+
+ res = ptep_xchg_lazy(mm, addr, ptep, __pte(_PAGE_INVALID));
+ if (mm_is_protected(mm) && pte_present(res))
+ uv_convert_from_secure(pte_val(res) & PAGE_MASK);
+ return res;
}
#define __HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION
@@ -1073,7 +1088,12 @@ void ptep_modify_prot_commit(struct vm_area_struct *, unsigned long,
static inline pte_t ptep_clear_flush(struct vm_area_struct *vma,
unsigned long addr, pte_t *ptep)
{
- return ptep_xchg_direct(vma->vm_mm, addr, ptep, __pte(_PAGE_INVALID));
+ pte_t res;
+
+ res = ptep_xchg_direct(vma->vm_mm, addr, ptep, __pte(_PAGE_INVALID));
+ if (mm_is_protected(vma->vm_mm) && pte_present(res))
+ uv_convert_from_secure(pte_val(res) & PAGE_MASK);
+ return res;
}
/*
@@ -1088,12 +1108,17 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,
unsigned long addr,
pte_t *ptep, int full)
{
+ pte_t res;
+
if (full) {
- pte_t pte = *ptep;
+ res = *ptep;
*ptep = __pte(_PAGE_INVALID);
- return pte;
+ } else {
+ res = ptep_xchg_lazy(mm, addr, ptep, __pte(_PAGE_INVALID));
}
- return ptep_xchg_lazy(mm, addr, ptep, __pte(_PAGE_INVALID));
+ if (mm_is_protected(mm) && pte_present(res))
+ uv_convert_from_secure(pte_val(res) & PAGE_MASK);
+ return res;
}
#define __HAVE_ARCH_PTEP_SET_WRPROTECT
diff --git a/arch/s390/include/asm/uv.h b/arch/s390/include/asm/uv.h
index cad643b05d19..7956868340c1 100644
--- a/arch/s390/include/asm/uv.h
+++ b/arch/s390/include/asm/uv.h
@@ -15,6 +15,7 @@
#include <linux/errno.h>
#include <linux/bug.h>
#include <asm/page.h>
+#include <asm/gmap.h>
#define UVC_RC_EXECUTED 0x0001
#define UVC_RC_INV_CMD 0x0002
@@ -24,6 +25,10 @@
#define UVC_CMD_QUI 0x0001
#define UVC_CMD_INIT_UV 0x000f
+#define UVC_CMD_CONV_TO_SEC_STOR 0x0200
+#define UVC_CMD_CONV_FROM_SEC_STOR 0x0201
+#define UVC_CMD_PIN_PAGE_SHARED 0x0341
+#define UVC_CMD_UNPIN_PAGE_SHARED 0x0342
#define UVC_CMD_SET_SHARED_ACCESS 0x1000
#define UVC_CMD_REMOVE_SHARED_ACCESS 0x1001
@@ -31,8 +36,12 @@
enum uv_cmds_inst {
BIT_UVC_CMD_QUI = 0,
BIT_UVC_CMD_INIT_UV = 1,
+ BIT_UVC_CMD_CONV_TO_SEC_STOR = 6,
+ BIT_UVC_CMD_CONV_FROM_SEC_STOR = 7,
BIT_UVC_CMD_SET_SHARED_ACCESS = 8,
BIT_UVC_CMD_REMOVE_SHARED_ACCESS = 9,
+ BIT_UVC_CMD_PIN_PAGE_SHARED = 21,
+ BIT_UVC_CMD_UNPIN_PAGE_SHARED = 22,
};
struct uv_cb_header {
@@ -69,6 +78,19 @@ struct uv_cb_init {
u64 reserved28[4];
} __packed __aligned(8);
+struct uv_cb_cts {
+ struct uv_cb_header header;
+ u64 reserved08[2];
+ u64 guest_handle;
+ u64 gaddr;
+} __packed __aligned(8);
+
+struct uv_cb_cfs {
+ struct uv_cb_header header;
+ u64 reserved08[2];
+ u64 paddr;
+} __packed __aligned(8);
+
struct uv_cb_share {
struct uv_cb_header header;
u64 reserved08[3];
@@ -171,12 +193,21 @@ static inline int is_prot_virt_host(void)
return prot_virt_host;
}
+int gmap_make_secure(struct gmap *gmap, unsigned long gaddr, void *uvcb);
+int uv_convert_from_secure(unsigned long paddr);
+int gmap_convert_to_secure(struct gmap *gmap, unsigned long gaddr);
+
void setup_uv(void);
void adjust_to_uv_max(unsigned long *vmax);
#else
#define is_prot_virt_host() 0
static inline void setup_uv(void) {}
static inline void adjust_to_uv_max(unsigned long *vmax) {}
+
+static inline int uv_convert_from_secure(unsigned long paddr)
+{
+ return 0;
+}
#endif
#if defined(CONFIG_PROTECTED_VIRTUALIZATION_GUEST) || \
diff --git a/arch/s390/kernel/uv.c b/arch/s390/kernel/uv.c
index 1ddc42154ef6..4539003dac9d 100644
--- a/arch/s390/kernel/uv.c
+++ b/arch/s390/kernel/uv.c
@@ -12,6 +12,8 @@
#include <linux/sizes.h>
#include <linux/bitmap.h>
#include <linux/memblock.h>
+#include <linux/pagemap.h>
+#include <linux/swap.h>
#include <asm/facility.h>
#include <asm/sections.h>
#include <asm/uv.h>
@@ -97,4 +99,229 @@ void adjust_to_uv_max(unsigned long *vmax)
{
*vmax = min_t(unsigned long, *vmax, uv_info.max_sec_stor_addr);
}
+
+/*
+ * Requests the Ultravisor to pin the page in the shared state. This will
+ * cause an intercept when the guest attempts to unshare the pinned page.
+ */
+static int uv_pin_shared(unsigned long paddr)
+{
+ struct uv_cb_cfs uvcb = {
+ .header.cmd = UVC_CMD_PIN_PAGE_SHARED,
+ .header.len = sizeof(uvcb),
+ .paddr = paddr,
+ };
+
+ if (uv_call(0, (u64)&uvcb))
+ return -EINVAL;
+ return 0;
+}
+
+/*
+ * Requests the Ultravisor to encrypt a guest page and make it
+ * accessible to the host for paging (export).
+ *
+ * @paddr: Absolute host address of page to be exported
+ */
+int uv_convert_from_secure(unsigned long paddr)
+{
+ struct uv_cb_cfs uvcb = {
+ .header.cmd = UVC_CMD_CONV_FROM_SEC_STOR,
+ .header.len = sizeof(uvcb),
+ .paddr = paddr
+ };
+
+ if (uv_call(0, (u64)&uvcb))
+ return -EINVAL;
+ return 0;
+}
+
+/*
+ * Calculate the expected ref_count for a page that would otherwise have no
+ * further pins. This was cribbed from similar functions in other places in
+ * the kernel, but with some slight modifications. We know that a secure
+ * page can not be a huge page for example.
+ */
+static int expected_page_refs(struct page *page)
+{
+ int res;
+
+ res = page_mapcount(page);
+ if (PageSwapCache(page)) {
+ res++;
+ } else if (page_mapping(page)) {
+ res++;
+ if (page_has_private(page))
+ res++;
+ }
+ return res;
+}
+
+static int make_secure_pte(pte_t *ptep, unsigned long addr,
+ struct page *exp_page, struct uv_cb_header *uvcb)
+{
+ pte_t entry = READ_ONCE(*ptep);
+ struct page *page;
+ int expected, rc = 0;
+
+ if (!pte_present(entry))
+ return -ENXIO;
+ if (pte_val(entry) & _PAGE_INVALID)
+ return -ENXIO;
+
+ page = pte_page(entry);
+ if (page != exp_page)
+ return -ENXIO;
+ if (PageWriteback(page))
+ return -EAGAIN;
+ expected = expected_page_refs(page);
+ if (!page_ref_freeze(page, expected))
+ return -EBUSY;
+ set_bit(PG_arch_1, &page->flags);
+ rc = uv_call(0, (u64)uvcb);
+ page_ref_unfreeze(page, expected);
+ /* Return -ENXIO if the page was not mapped, -EINVAL otherwise */
+ if (rc)
+ rc = uvcb->rc == 0x10a ? -ENXIO : -EINVAL;
+ return rc;
+}
+
+/*
+ * Requests the Ultravisor to make a page accessible to a guest.
+ * If it's brought in the first time, it will be cleared. If
+ * it has been exported before, it will be decrypted and integrity
+ * checked.
+ */
+int gmap_make_secure(struct gmap *gmap, unsigned long gaddr, void *uvcb)
+{
+ struct vm_area_struct *vma;
+ bool local_drain = false;
+ spinlock_t *ptelock;
+ unsigned long uaddr;
+ struct page *page;
+ pte_t *ptep;
+ int rc;
+
+again:
+ rc = -EFAULT;
+ down_read(&gmap->mm->mmap_sem);
+
+ uaddr = __gmap_translate(gmap, gaddr);
+ if (IS_ERR_VALUE(uaddr))
+ goto out;
+ vma = find_vma(gmap->mm, uaddr);
+ if (!vma)
+ goto out;
+ /*
+ * Secure pages cannot be huge and userspace should not combine both.
+ * In case userspace does it anyway this will result in an -EFAULT for
+ * the unpack. The guest is thus never reaching secure mode. If
+ * userspace is playing dirty tricky with mapping huge pages later
+ * on this will result in a segmentation fault.
+ */
+ if (is_vm_hugetlb_page(vma))
+ goto out;
+
+ rc = -ENXIO;
+ page = follow_page(vma, uaddr, FOLL_WRITE);
+ if (IS_ERR_OR_NULL(page))
+ goto out;
+
+ lock_page(page);
+ ptep = get_locked_pte(gmap->mm, uaddr, &ptelock);
+ rc = make_secure_pte(ptep, uaddr, page, uvcb);
+ pte_unmap_unlock(ptep, ptelock);
+ unlock_page(page);
+out:
+ up_read(&gmap->mm->mmap_sem);
+
+ if (rc == -EAGAIN) {
+ wait_on_page_writeback(page);
+ } else if (rc == -EBUSY) {
+ /*
+ * If we have tried a local drain and the page refcount
+ * still does not match our expected safe value, try with a
+ * system wide drain. This is needed if the pagevecs holding
+ * the page are on a different CPU.
+ */
+ if (local_drain) {
+ lru_add_drain_all();
+ /* We give up here, and let the caller try again */
+ return -EAGAIN;
+ }
+ /*
+ * We are here if the page refcount does not match the
+ * expected safe value. The main culprits are usually
+ * pagevecs. With lru_add_drain() we drain the pagevecs
+ * on the local CPU so that hopefully the refcount will
+ * reach the expected safe value.
+ */
+ lru_add_drain();
+ local_drain = true;
+ /* And now we try again immediately after draining */
+ goto again;
+ } else if (rc == -ENXIO) {
+ if (gmap_fault(gmap, gaddr, FAULT_FLAG_WRITE))
+ return -EFAULT;
+ return -EAGAIN;
+ }
+ return rc;
+}
+EXPORT_SYMBOL_GPL(gmap_make_secure);
+
+int gmap_convert_to_secure(struct gmap *gmap, unsigned long gaddr)
+{
+ struct uv_cb_cts uvcb = {
+ .header.cmd = UVC_CMD_CONV_TO_SEC_STOR,
+ .header.len = sizeof(uvcb),
+ .guest_handle = gmap->guest_handle,
+ .gaddr = gaddr,
+ };
+
+ return gmap_make_secure(gmap, gaddr, &uvcb);
+}
+EXPORT_SYMBOL_GPL(gmap_convert_to_secure);
+
+/*
+ * To be called with the page locked or with an extra reference! This will
+ * prevent gmap_make_secure from touching the page concurrently. Having 2
+ * parallel make_page_accessible is fine, as the UV calls will become a
+ * no-op if the page is already exported.
+ */
+int arch_make_page_accessible(struct page *page)
+{
+ int rc = 0;
+
+ /* Hugepage cannot be protected, so nothing to do */
+ if (PageHuge(page))
+ return 0;
+
+ /*
+ * PG_arch_1 is used in 3 places:
+ * 1. for kernel page tables during early boot
+ * 2. for storage keys of huge pages and KVM
+ * 3. As an indication that this page might be secure. This can
+ * overindicate, e.g. we set the bit before calling
+ * convert_to_secure.
+ * As secure pages are never huge, all 3 variants can co-exists.
+ */
+ if (!test_bit(PG_arch_1, &page->flags))
+ return 0;
+
+ rc = uv_pin_shared(page_to_phys(page));
+ if (!rc) {
+ clear_bit(PG_arch_1, &page->flags);
+ return 0;
+ }
+
+ rc = uv_convert_from_secure(page_to_phys(page));
+ if (!rc) {
+ clear_bit(PG_arch_1, &page->flags);
+ return 0;
+ }
+
+ return rc;
+}
+EXPORT_SYMBOL_GPL(arch_make_page_accessible);
+
#endif
--
2.25.0
^ permalink raw reply related
* [PATCH v3 16/37] KVM: s390: protvirt: Add SCLP interrupt handling
From: Christian Borntraeger @ 2020-02-20 10:39 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik, Janosch Frank
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
The sclp interrupt is kind of special. The ultravisor polices that we
do not inject an sclp interrupt with payload if no sccb is outstanding.
On the other hand we have "asynchronous" event interrupts, e.g. for
console input.
We separate both variants into sclp interrupt and sclp event interrupt.
The sclp interrupt is masked until a previous servc instruction has
finished (sie exit 108).
[frankja@linux.ibm.com: factoring out write_sclp]
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
arch/s390/include/asm/kvm_host.h | 6 +-
arch/s390/kvm/intercept.c | 27 +++++++++
arch/s390/kvm/interrupt.c | 95 ++++++++++++++++++++++++++------
arch/s390/kvm/kvm-s390.c | 4 ++
4 files changed, 113 insertions(+), 19 deletions(-)
diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
index a13dc77f8b07..ba3364b37159 100644
--- a/arch/s390/include/asm/kvm_host.h
+++ b/arch/s390/include/asm/kvm_host.h
@@ -518,6 +518,7 @@ enum irq_types {
IRQ_PEND_PFAULT_INIT,
IRQ_PEND_EXT_HOST,
IRQ_PEND_EXT_SERVICE,
+ IRQ_PEND_EXT_SERVICE_EV,
IRQ_PEND_EXT_TIMING,
IRQ_PEND_EXT_CPU_TIMER,
IRQ_PEND_EXT_CLOCK_COMP,
@@ -562,6 +563,7 @@ enum irq_types {
(1UL << IRQ_PEND_EXT_TIMING) | \
(1UL << IRQ_PEND_EXT_HOST) | \
(1UL << IRQ_PEND_EXT_SERVICE) | \
+ (1UL << IRQ_PEND_EXT_SERVICE_EV) | \
(1UL << IRQ_PEND_VIRTIO) | \
(1UL << IRQ_PEND_PFAULT_INIT) | \
(1UL << IRQ_PEND_PFAULT_DONE))
@@ -582,7 +584,8 @@ enum irq_types {
(1UL << IRQ_PEND_EXT_CLOCK_COMP) | \
(1UL << IRQ_PEND_EXT_EMERGENCY) | \
(1UL << IRQ_PEND_EXT_EXTERNAL) | \
- (1UL << IRQ_PEND_EXT_SERVICE))
+ (1UL << IRQ_PEND_EXT_SERVICE) | \
+ (1UL << IRQ_PEND_EXT_SERVICE_EV))
struct kvm_s390_interrupt_info {
struct list_head list;
@@ -642,6 +645,7 @@ struct kvm_s390_local_interrupt {
struct kvm_s390_float_interrupt {
unsigned long pending_irqs;
+ unsigned long masked_irqs;
spinlock_t lock;
struct list_head lists[FIRQ_LIST_COUNT];
int counters[FIRQ_MAX_COUNT];
diff --git a/arch/s390/kvm/intercept.c b/arch/s390/kvm/intercept.c
index 6fdbac696f65..d50a0214eba1 100644
--- a/arch/s390/kvm/intercept.c
+++ b/arch/s390/kvm/intercept.c
@@ -444,8 +444,35 @@ static int handle_operexc(struct kvm_vcpu *vcpu)
return kvm_s390_inject_program_int(vcpu, PGM_OPERATION);
}
+static int handle_pv_sclp(struct kvm_vcpu *vcpu)
+{
+ struct kvm_s390_float_interrupt *fi = &vcpu->kvm->arch.float_int;
+
+ spin_lock(&fi->lock);
+ /*
+ * 2 cases:
+ * a: an sccb answering interrupt was already pending or in flight.
+ * As the sccb value is not known we can simply set some value to
+ * trigger delivery of a saved SCCB. UV will then use its saved
+ * copy of the SCCB value.
+ * b: an error SCCB interrupt needs to be injected so we also inject
+ * a fake SCCB address. Firmware will use the proper one.
+ * This makes sure, that both errors and real sccb returns will only
+ * be delivered after a notification intercept (instruction has
+ * finished) but not after others.
+ */
+ fi->srv_signal.ext_params |= 0x43000;
+ set_bit(IRQ_PEND_EXT_SERVICE, &fi->pending_irqs);
+ clear_bit(IRQ_PEND_EXT_SERVICE, &fi->masked_irqs);
+ spin_unlock(&fi->lock);
+ return 0;
+}
+
static int handle_pv_notification(struct kvm_vcpu *vcpu)
{
+ if (vcpu->arch.sie_block->ipa == 0xb220)
+ return handle_pv_sclp(vcpu);
+
return handle_instruction(vcpu);
}
diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
index 61310b1f5b62..aaff3dbd669e 100644
--- a/arch/s390/kvm/interrupt.c
+++ b/arch/s390/kvm/interrupt.c
@@ -324,8 +324,11 @@ static inline int gisa_tac_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisc)
static inline unsigned long pending_irqs_no_gisa(struct kvm_vcpu *vcpu)
{
- return vcpu->kvm->arch.float_int.pending_irqs |
- vcpu->arch.local_int.pending_irqs;
+ unsigned long pending = vcpu->kvm->arch.float_int.pending_irqs |
+ vcpu->arch.local_int.pending_irqs;
+
+ pending &= ~vcpu->kvm->arch.float_int.masked_irqs;
+ return pending;
}
static inline unsigned long pending_irqs(struct kvm_vcpu *vcpu)
@@ -383,8 +386,10 @@ static unsigned long deliverable_irqs(struct kvm_vcpu *vcpu)
__clear_bit(IRQ_PEND_EXT_CLOCK_COMP, &active_mask);
if (!(vcpu->arch.sie_block->gcr[0] & CR0_CPU_TIMER_SUBMASK))
__clear_bit(IRQ_PEND_EXT_CPU_TIMER, &active_mask);
- if (!(vcpu->arch.sie_block->gcr[0] & CR0_SERVICE_SIGNAL_SUBMASK))
+ if (!(vcpu->arch.sie_block->gcr[0] & CR0_SERVICE_SIGNAL_SUBMASK)) {
__clear_bit(IRQ_PEND_EXT_SERVICE, &active_mask);
+ __clear_bit(IRQ_PEND_EXT_SERVICE_EV, &active_mask);
+ }
if (psw_mchk_disabled(vcpu))
active_mask &= ~IRQ_PEND_MCHK_MASK;
/* PV guest cpus can have a single interruption injected at a time. */
@@ -946,20 +951,49 @@ static int __must_check __deliver_prog(struct kvm_vcpu *vcpu)
return rc ? -EFAULT : 0;
}
+#define SCCB_MASK 0xFFFFFFF8
+#define SCCB_EVENT_PENDING 0x3
+
+static int write_sclp(struct kvm_vcpu *vcpu, u32 parm)
+{
+ int rc;
+
+ if (kvm_s390_pv_cpu_get_handle(vcpu)) {
+ vcpu->arch.sie_block->iictl = IICTL_CODE_EXT;
+ vcpu->arch.sie_block->eic = EXT_IRQ_SERVICE_SIG;
+ vcpu->arch.sie_block->eiparams = parm;
+ return 0;
+ }
+
+ rc = put_guest_lc(vcpu, EXT_IRQ_SERVICE_SIG, (u16 *)__LC_EXT_INT_CODE);
+ rc |= put_guest_lc(vcpu, 0, (u16 *)__LC_EXT_CPU_ADDR);
+ rc |= write_guest_lc(vcpu, __LC_EXT_OLD_PSW,
+ &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
+ rc |= read_guest_lc(vcpu, __LC_EXT_NEW_PSW,
+ &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
+ rc |= put_guest_lc(vcpu, parm,
+ (u32 *)__LC_EXT_PARAMS);
+
+ return rc ? -EFAULT : 0;
+}
+
static int __must_check __deliver_service(struct kvm_vcpu *vcpu)
{
struct kvm_s390_float_interrupt *fi = &vcpu->kvm->arch.float_int;
struct kvm_s390_ext_info ext;
- int rc = 0;
spin_lock(&fi->lock);
- if (!(test_bit(IRQ_PEND_EXT_SERVICE, &fi->pending_irqs))) {
+ if (test_bit(IRQ_PEND_EXT_SERVICE, &fi->masked_irqs) ||
+ !(test_bit(IRQ_PEND_EXT_SERVICE, &fi->pending_irqs))) {
spin_unlock(&fi->lock);
return 0;
}
ext = fi->srv_signal;
memset(&fi->srv_signal, 0, sizeof(ext));
clear_bit(IRQ_PEND_EXT_SERVICE, &fi->pending_irqs);
+ clear_bit(IRQ_PEND_EXT_SERVICE_EV, &fi->pending_irqs);
+ if (kvm_s390_pv_cpu_is_protected(vcpu))
+ set_bit(IRQ_PEND_EXT_SERVICE, &fi->masked_irqs);
spin_unlock(&fi->lock);
VCPU_EVENT(vcpu, 4, "deliver: sclp parameter 0x%x",
@@ -968,16 +1002,31 @@ static int __must_check __deliver_service(struct kvm_vcpu *vcpu)
trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id, KVM_S390_INT_SERVICE,
ext.ext_params, 0);
- rc = put_guest_lc(vcpu, EXT_IRQ_SERVICE_SIG, (u16 *)__LC_EXT_INT_CODE);
- rc |= put_guest_lc(vcpu, 0, (u16 *)__LC_EXT_CPU_ADDR);
- rc |= write_guest_lc(vcpu, __LC_EXT_OLD_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
- rc |= read_guest_lc(vcpu, __LC_EXT_NEW_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
- rc |= put_guest_lc(vcpu, ext.ext_params,
- (u32 *)__LC_EXT_PARAMS);
+ return write_sclp(vcpu, ext.ext_params);
+}
- return rc ? -EFAULT : 0;
+static int __must_check __deliver_service_ev(struct kvm_vcpu *vcpu)
+{
+ struct kvm_s390_float_interrupt *fi = &vcpu->kvm->arch.float_int;
+ struct kvm_s390_ext_info ext;
+
+ spin_lock(&fi->lock);
+ if (!(test_bit(IRQ_PEND_EXT_SERVICE_EV, &fi->pending_irqs))) {
+ spin_unlock(&fi->lock);
+ return 0;
+ }
+ ext = fi->srv_signal;
+ /* only clear the event bit */
+ fi->srv_signal.ext_params &= ~SCCB_EVENT_PENDING;
+ clear_bit(IRQ_PEND_EXT_SERVICE_EV, &fi->pending_irqs);
+ spin_unlock(&fi->lock);
+
+ VCPU_EVENT(vcpu, 4, "%s", "deliver: sclp parameter event");
+ vcpu->stat.deliver_service_signal++;
+ trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id, KVM_S390_INT_SERVICE,
+ ext.ext_params, 0);
+
+ return write_sclp(vcpu, SCCB_EVENT_PENDING);
}
static int __must_check __deliver_pfault_done(struct kvm_vcpu *vcpu)
@@ -1382,6 +1431,9 @@ int __must_check kvm_s390_deliver_pending_interrupts(struct kvm_vcpu *vcpu)
case IRQ_PEND_EXT_SERVICE:
rc = __deliver_service(vcpu);
break;
+ case IRQ_PEND_EXT_SERVICE_EV:
+ rc = __deliver_service_ev(vcpu);
+ break;
case IRQ_PEND_PFAULT_DONE:
rc = __deliver_pfault_done(vcpu);
break;
@@ -1734,9 +1786,6 @@ struct kvm_s390_interrupt_info *kvm_s390_get_io_int(struct kvm *kvm,
return inti;
}
-#define SCCB_MASK 0xFFFFFFF8
-#define SCCB_EVENT_PENDING 0x3
-
static int __inject_service(struct kvm *kvm,
struct kvm_s390_interrupt_info *inti)
{
@@ -1745,6 +1794,11 @@ static int __inject_service(struct kvm *kvm,
kvm->stat.inject_service_signal++;
spin_lock(&fi->lock);
fi->srv_signal.ext_params |= inti->ext.ext_params & SCCB_EVENT_PENDING;
+
+ /* We always allow events, track them separately from the sccb ints */
+ if (fi->srv_signal.ext_params & SCCB_EVENT_PENDING)
+ set_bit(IRQ_PEND_EXT_SERVICE_EV, &fi->pending_irqs);
+
/*
* Early versions of the QEMU s390 bios will inject several
* service interrupts after another without handling a
@@ -2141,6 +2195,10 @@ void kvm_s390_clear_float_irqs(struct kvm *kvm)
struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int;
int i;
+ mutex_lock(&kvm->lock);
+ if (!kvm_s390_pv_is_protected(kvm))
+ fi->masked_irqs = 0;
+ mutex_unlock(&kvm->lock);
spin_lock(&fi->lock);
fi->pending_irqs = 0;
memset(&fi->srv_signal, 0, sizeof(fi->srv_signal));
@@ -2207,7 +2265,8 @@ static int get_all_floating_irqs(struct kvm *kvm, u8 __user *usrbuf, u64 len)
n++;
}
}
- if (test_bit(IRQ_PEND_EXT_SERVICE, &fi->pending_irqs)) {
+ if (test_bit(IRQ_PEND_EXT_SERVICE, &fi->pending_irqs) ||
+ test_bit(IRQ_PEND_EXT_SERVICE_EV, &fi->pending_irqs)) {
if (n == max_irqs) {
/* signal userspace to try again */
ret = -ENOMEM;
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index f99e4eb5c27b..f9e05b3f0289 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -2236,6 +2236,8 @@ static int kvm_s390_handle_pv(struct kvm *kvm, struct kvm_pv_cmd *cmd)
if (r)
kvm_s390_pv_destroy_vm(kvm, &dummy, &dummy);
+ /* we need to block service interrupts from now on */
+ set_bit(IRQ_PEND_EXT_SERVICE, &kvm->arch.float_int.masked_irqs);
break;
}
case KVM_PV_DISABLE: {
@@ -2247,6 +2249,8 @@ static int kvm_s390_handle_pv(struct kvm *kvm, struct kvm_pv_cmd *cmd)
r = kvm_s390_pv_destroy_vm(kvm, &cmd->rc, &cmd->rrc);
if (!r)
kvm_s390_pv_dealloc_vm(kvm);
+ /* no need to block service interrupts any more */
+ clear_bit(IRQ_PEND_EXT_SERVICE, &kvm->arch.float_int.masked_irqs);
break;
}
case KVM_PV_VM_SET_SEC_PARMS: {
--
2.25.0
^ permalink raw reply related
* [PATCH v3 29/37] KVM: s390: protvirt: Report CPU state to Ultravisor
From: Christian Borntraeger @ 2020-02-20 10:40 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik, Janosch Frank
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
From: Janosch Frank <frankja@linux.ibm.com>
VCPU states have to be reported to the ultravisor for SIGP
interpretation, kdump, kexec and reboot.
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
[borntraeger@de.ibm.com: patch merging, splitting, fixing]
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
arch/s390/include/asm/uv.h | 15 +++++++++++++++
arch/s390/kvm/diag.c | 4 ++++
arch/s390/kvm/intercept.c | 4 ++++
arch/s390/kvm/kvm-s390.c | 33 ++++++++++++++++++++++-----------
arch/s390/kvm/kvm-s390.h | 5 +++--
arch/s390/kvm/pv.c | 18 ++++++++++++++++++
6 files changed, 66 insertions(+), 13 deletions(-)
diff --git a/arch/s390/include/asm/uv.h b/arch/s390/include/asm/uv.h
index a7595c647759..99e1a14ef909 100644
--- a/arch/s390/include/asm/uv.h
+++ b/arch/s390/include/asm/uv.h
@@ -37,6 +37,7 @@
#define UVC_CMD_UNPACK_IMG 0x0301
#define UVC_CMD_VERIFY_IMG 0x0302
#define UVC_CMD_PREPARE_RESET 0x0320
+#define UVC_CMD_CPU_SET_STATE 0x0330
#define UVC_CMD_SET_UNSHARE_ALL 0x0340
#define UVC_CMD_PIN_PAGE_SHARED 0x0341
#define UVC_CMD_UNPIN_PAGE_SHARED 0x0342
@@ -58,6 +59,7 @@ enum uv_cmds_inst {
BIT_UVC_CMD_SET_SEC_PARMS = 11,
BIT_UVC_CMD_UNPACK_IMG = 13,
BIT_UVC_CMD_VERIFY_IMG = 14,
+ BIT_UVC_CMD_CPU_SET_STATE = 17,
BIT_UVC_CMD_PREPARE_RESET = 18,
BIT_UVC_CMD_UNSHARE_ALL = 20,
BIT_UVC_CMD_PIN_PAGE_SHARED = 21,
@@ -164,6 +166,19 @@ struct uv_cb_unp {
u64 reserved38[3];
} __packed __aligned(8);
+#define PV_CPU_STATE_OPR 1
+#define PV_CPU_STATE_STP 2
+#define PV_CPU_STATE_CHKSTP 3
+
+struct uv_cb_cpu_set_state {
+ struct uv_cb_header header;
+ u64 reserved08[2];
+ u64 cpu_handle;
+ u8 reserved20[7];
+ u8 state;
+ u64 reserved28[5];
+};
+
/*
* A common UV call struct for calls that take no payload
* Examples:
diff --git a/arch/s390/kvm/diag.c b/arch/s390/kvm/diag.c
index 3fb54ec2cf3e..130f0c51d162 100644
--- a/arch/s390/kvm/diag.c
+++ b/arch/s390/kvm/diag.c
@@ -201,6 +201,10 @@ static int __diag_ipl_functions(struct kvm_vcpu *vcpu)
return -EOPNOTSUPP;
}
+ /*
+ * no need to check the return value of vcpu_stop as it can only have
+ * an error for protvirt, but protvirt means user cpu state
+ */
if (!kvm_s390_user_cpu_state_ctrl(vcpu->kvm))
kvm_s390_vcpu_stop(vcpu);
vcpu->run->s390_reset_flags |= KVM_S390_RESET_SUBSYSTEM;
diff --git a/arch/s390/kvm/intercept.c b/arch/s390/kvm/intercept.c
index a5de0e1975bb..73cf2961ac4f 100644
--- a/arch/s390/kvm/intercept.c
+++ b/arch/s390/kvm/intercept.c
@@ -80,6 +80,10 @@ static int handle_stop(struct kvm_vcpu *vcpu)
return rc;
}
+ /*
+ * no need to check the return value of vcpu_stop as it can only have
+ * an error for protvirt, but protvirt means user cpu state
+ */
if (!kvm_s390_user_cpu_state_ctrl(vcpu->kvm))
kvm_s390_vcpu_stop(vcpu);
return -EOPNOTSUPP;
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 9ac73b717e25..036bff58e75a 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -2438,6 +2438,8 @@ long kvm_arch_vm_ioctl(struct file *filp,
case KVM_S390_PV_COMMAND: {
struct kvm_pv_cmd args;
+ /* protvirt means user sigp */
+ kvm->arch.user_cpu_state_ctrl = 1;
r = 0;
if (!is_prot_virt_host()) {
r = -EINVAL;
@@ -3699,10 +3701,10 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
switch (mp_state->mp_state) {
case KVM_MP_STATE_STOPPED:
- kvm_s390_vcpu_stop(vcpu);
+ rc = kvm_s390_vcpu_stop(vcpu);
break;
case KVM_MP_STATE_OPERATING:
- kvm_s390_vcpu_start(vcpu);
+ rc = kvm_s390_vcpu_start(vcpu);
break;
case KVM_MP_STATE_LOAD:
case KVM_MP_STATE_CHECK_STOP:
@@ -4287,6 +4289,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
kvm_sigset_activate(vcpu);
+ /*
+ * no need to check the return value of vcpu_start as it can only have
+ * an error for protvirt, but protvirt means user cpu state
+ */
if (!kvm_s390_user_cpu_state_ctrl(vcpu->kvm)) {
kvm_s390_vcpu_start(vcpu);
} else if (is_vcpu_stopped(vcpu)) {
@@ -4424,12 +4430,12 @@ static void __enable_ibs_on_vcpu(struct kvm_vcpu *vcpu)
kvm_s390_sync_request(KVM_REQ_ENABLE_IBS, vcpu);
}
-void kvm_s390_vcpu_start(struct kvm_vcpu *vcpu)
+int kvm_s390_vcpu_start(struct kvm_vcpu *vcpu)
{
- int i, online_vcpus, started_vcpus = 0;
+ int i, online_vcpus, r= 0, started_vcpus = 0;
if (!is_vcpu_stopped(vcpu))
- return;
+ return 0;
trace_kvm_s390_vcpu_start_stop(vcpu->vcpu_id, 1);
/* Only one cpu at a time may enter/leave the STOPPED state. */
@@ -4452,7 +4458,9 @@ void kvm_s390_vcpu_start(struct kvm_vcpu *vcpu)
*/
__disable_ibs_on_all_vcpus(vcpu->kvm);
}
-
+ /* Let's tell the UV that we want to start again */
+ if (kvm_s390_pv_cpu_is_protected(vcpu))
+ r = kvm_s390_pv_set_cpu_state(vcpu, PV_CPU_STATE_OPR);
kvm_s390_clear_cpuflags(vcpu, CPUSTAT_STOPPED);
/*
* Another VCPU might have used IBS while we were offline.
@@ -4460,16 +4468,16 @@ void kvm_s390_vcpu_start(struct kvm_vcpu *vcpu)
*/
kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu);
spin_unlock(&vcpu->kvm->arch.start_stop_lock);
- return;
+ return r;
}
-void kvm_s390_vcpu_stop(struct kvm_vcpu *vcpu)
+int kvm_s390_vcpu_stop(struct kvm_vcpu *vcpu)
{
- int i, online_vcpus, started_vcpus = 0;
+ int i, online_vcpus, r = 0, started_vcpus = 0;
struct kvm_vcpu *started_vcpu = NULL;
if (is_vcpu_stopped(vcpu))
- return;
+ return 0;
trace_kvm_s390_vcpu_start_stop(vcpu->vcpu_id, 0);
/* Only one cpu at a time may enter/leave the STOPPED state. */
@@ -4480,6 +4488,9 @@ void kvm_s390_vcpu_stop(struct kvm_vcpu *vcpu)
kvm_s390_clear_stop_irq(vcpu);
kvm_s390_set_cpuflags(vcpu, CPUSTAT_STOPPED);
+ /* Let's tell the UV that we successfully stopped the vcpu */
+ if (kvm_s390_pv_cpu_is_protected(vcpu))
+ r = kvm_s390_pv_set_cpu_state(vcpu, PV_CPU_STATE_STP);
__disable_ibs_on_vcpu(vcpu);
for (i = 0; i < online_vcpus; i++) {
@@ -4498,7 +4509,7 @@ void kvm_s390_vcpu_stop(struct kvm_vcpu *vcpu)
}
spin_unlock(&vcpu->kvm->arch.start_stop_lock);
- return;
+ return r;
}
static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
index d62de29b2d6c..e9e1996d643b 100644
--- a/arch/s390/kvm/kvm-s390.h
+++ b/arch/s390/kvm/kvm-s390.h
@@ -219,6 +219,7 @@ int kvm_s390_pv_set_sec_parms(struct kvm *kvm, void *hdr, u64 length, u16 *rc,
u16 *rrc);
int kvm_s390_pv_unpack(struct kvm *kvm, unsigned long addr, unsigned long size,
unsigned long tweak, u16 *rc, u16 *rrc);
+int kvm_s390_pv_set_cpu_state(struct kvm_vcpu *vcpu, u8 state);
static inline u64 kvm_s390_pv_get_handle(struct kvm *kvm)
{
@@ -332,8 +333,8 @@ void kvm_s390_set_tod_clock(struct kvm *kvm,
long kvm_arch_fault_in_page(struct kvm_vcpu *vcpu, gpa_t gpa, int writable);
int kvm_s390_store_status_unloaded(struct kvm_vcpu *vcpu, unsigned long addr);
int kvm_s390_vcpu_store_status(struct kvm_vcpu *vcpu, unsigned long addr);
-void kvm_s390_vcpu_start(struct kvm_vcpu *vcpu);
-void kvm_s390_vcpu_stop(struct kvm_vcpu *vcpu);
+int kvm_s390_vcpu_start(struct kvm_vcpu *vcpu);
+int kvm_s390_vcpu_stop(struct kvm_vcpu *vcpu);
void kvm_s390_vcpu_block(struct kvm_vcpu *vcpu);
void kvm_s390_vcpu_unblock(struct kvm_vcpu *vcpu);
bool kvm_s390_vcpu_sie_inhibited(struct kvm_vcpu *vcpu);
diff --git a/arch/s390/kvm/pv.c b/arch/s390/kvm/pv.c
index a9c7e97d98e1..e9863382ee15 100644
--- a/arch/s390/kvm/pv.c
+++ b/arch/s390/kvm/pv.c
@@ -266,3 +266,21 @@ int kvm_s390_pv_unpack(struct kvm *kvm, unsigned long addr, unsigned long size,
KVM_UV_EVENT(kvm, 3, "%s", "PROTVIRT VM UNPACK: successful");
return ret;
}
+
+int kvm_s390_pv_set_cpu_state(struct kvm_vcpu *vcpu, u8 state)
+{
+ struct uv_cb_cpu_set_state uvcb = {
+ .header.cmd = UVC_CMD_CPU_SET_STATE,
+ .header.len = sizeof(uvcb),
+ .cpu_handle = kvm_s390_pv_cpu_get_handle(vcpu),
+ .state = state,
+ };
+ int cc;
+
+ cc = uv_call(0, (u64)&uvcb);
+ KVM_UV_EVENT(vcpu->kvm, 3, "PROTVIRT SET CPU %d STATE %d rc %x rrc %x",
+ vcpu->vcpu_id, state, uvcb.header.rc, uvcb.header.rrc);
+ if (cc)
+ return -EINVAL;
+ return 0;
+}
--
2.25.0
^ permalink raw reply related
* [PATCH v3 36/37] KVM: s390: rstify new ioctls in api.rst
From: Christian Borntraeger @ 2020-02-20 10:40 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
We also need to rstify the new ioctls that we added in parallel to the
rstification of the kvm docs.
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
Documentation/virt/kvm/api.rst | 33 ++++++++++++++++++---------------
1 file changed, 18 insertions(+), 15 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 58be7aba0db2..faca9977cbe7 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -4613,35 +4613,38 @@ unpins the VPA pages and releases all the device pages that are used to
track the secure pages by hypervisor.
4.122 KVM_S390_NORMAL_RESET
+---------------------------
-Capability: KVM_CAP_S390_VCPU_RESETS
-Architectures: s390
-Type: vcpu ioctl
-Parameters: none
-Returns: 0
+:Capability: KVM_CAP_S390_VCPU_RESETS
+:Architectures: s390
+:Type: vcpu ioctl
+:Parameters: none
+:Returns: 0
This ioctl resets VCPU registers and control structures according to
the cpu reset definition in the POP (Principles Of Operation).
4.123 KVM_S390_INITIAL_RESET
+----------------------------
-Capability: none
-Architectures: s390
-Type: vcpu ioctl
-Parameters: none
-Returns: 0
+:Capability: none
+:Architectures: s390
+:Type: vcpu ioctl
+:Parameters: none
+:Returns: 0
This ioctl resets VCPU registers and control structures according to
the initial cpu reset definition in the POP. However, the cpu is not
put into ESA mode. This reset is a superset of the normal reset.
4.124 KVM_S390_CLEAR_RESET
+--------------------------
-Capability: KVM_CAP_S390_VCPU_RESETS
-Architectures: s390
-Type: vcpu ioctl
-Parameters: none
-Returns: 0
+:Capability: KVM_CAP_S390_VCPU_RESETS
+:Architectures: s390
+:Type: vcpu ioctl
+:Parameters: none
+:Returns: 0
This ioctl resets VCPU registers and control structures according to
the clear cpu reset definition in the POP. However, the cpu is not put
--
2.25.0
^ permalink raw reply related
* [PATCH v3 26/37] KVM: s390: protvirt: Only sync fmt4 registers
From: Christian Borntraeger @ 2020-02-20 10:40 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik, Janosch Frank
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
From: Janosch Frank <frankja@linux.ibm.com>
A lot of the registers are controlled by the Ultravisor and never
visible to KVM. Also some registers are overlayed, like gbea is with
sidad, which might leak data to userspace.
Hence we sync a minimal set of registers for both SIE formats and then
check and sync format 2 registers if necessary.
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
[borntraeger@de.ibm.com: patch merging, splitting, fixing]
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
arch/s390/kvm/kvm-s390.c | 110 +++++++++++++++++++++++++--------------
1 file changed, 70 insertions(+), 40 deletions(-)
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index db28a717a622..303f994f3d03 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -4084,7 +4084,7 @@ static int __vcpu_run(struct kvm_vcpu *vcpu)
return rc;
}
-static void sync_regs(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
+static void sync_regs_fmt2(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
{
struct runtime_instr_cb *riccb;
struct gs_cb *gscb;
@@ -4093,16 +4093,7 @@ static void sync_regs(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
gscb = (struct gs_cb *) &kvm_run->s.regs.gscb;
vcpu->arch.sie_block->gpsw.mask = kvm_run->psw_mask;
vcpu->arch.sie_block->gpsw.addr = kvm_run->psw_addr;
- if (kvm_run->kvm_dirty_regs & KVM_SYNC_PREFIX)
- kvm_s390_set_prefix(vcpu, kvm_run->s.regs.prefix);
- if (kvm_run->kvm_dirty_regs & KVM_SYNC_CRS) {
- memcpy(&vcpu->arch.sie_block->gcr, &kvm_run->s.regs.crs, 128);
- /* some control register changes require a tlb flush */
- kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu);
- }
if (kvm_run->kvm_dirty_regs & KVM_SYNC_ARCH0) {
- kvm_s390_set_cpu_timer(vcpu, kvm_run->s.regs.cputm);
- vcpu->arch.sie_block->ckc = kvm_run->s.regs.ckc;
vcpu->arch.sie_block->todpr = kvm_run->s.regs.todpr;
vcpu->arch.sie_block->pp = kvm_run->s.regs.pp;
vcpu->arch.sie_block->gbea = kvm_run->s.regs.gbea;
@@ -4143,6 +4134,36 @@ static void sync_regs(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
vcpu->arch.sie_block->fpf &= ~FPF_BPBC;
vcpu->arch.sie_block->fpf |= kvm_run->s.regs.bpbc ? FPF_BPBC : 0;
}
+ if (MACHINE_HAS_GS) {
+ preempt_disable();
+ __ctl_set_bit(2, 4);
+ if (current->thread.gs_cb) {
+ vcpu->arch.host_gscb = current->thread.gs_cb;
+ save_gs_cb(vcpu->arch.host_gscb);
+ }
+ if (vcpu->arch.gs_enabled) {
+ current->thread.gs_cb = (struct gs_cb *)
+ &vcpu->run->s.regs.gscb;
+ restore_gs_cb(current->thread.gs_cb);
+ }
+ preempt_enable();
+ }
+ /* SIE will load etoken directly from SDNX and therefore kvm_run */
+}
+
+static void sync_regs(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
+{
+ if (kvm_run->kvm_dirty_regs & KVM_SYNC_PREFIX)
+ kvm_s390_set_prefix(vcpu, kvm_run->s.regs.prefix);
+ if (kvm_run->kvm_dirty_regs & KVM_SYNC_CRS) {
+ memcpy(&vcpu->arch.sie_block->gcr, &kvm_run->s.regs.crs, 128);
+ /* some control register changes require a tlb flush */
+ kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu);
+ }
+ if (kvm_run->kvm_dirty_regs & KVM_SYNC_ARCH0) {
+ kvm_s390_set_cpu_timer(vcpu, kvm_run->s.regs.cputm);
+ vcpu->arch.sie_block->ckc = kvm_run->s.regs.ckc;
+ }
save_access_regs(vcpu->arch.host_acrs);
restore_access_regs(vcpu->run->s.regs.acrs);
/* save host (userspace) fprs/vrs */
@@ -4157,23 +4178,47 @@ static void sync_regs(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
if (test_fp_ctl(current->thread.fpu.fpc))
/* User space provided an invalid FPC, let's clear it */
current->thread.fpu.fpc = 0;
+
+ /* Sync fmt2 only data */
+ if (likely(!kvm_s390_pv_cpu_is_protected(vcpu))) {
+ sync_regs_fmt2(vcpu, kvm_run);
+ } else {
+ /*
+ * In several places we have to modify our internal view to
+ * not do things that are disallowed by the ultravisor. For
+ * example we must not inject interrupts after specific exits
+ * (e.g. 112 prefix page not secure). We do this by turning
+ * off the machine check, external and I/O interrupt bits
+ * of our PSW copy. To avoid getting validity intercepts, we
+ * do only accept the condition code from userspace.
+ */
+ vcpu->arch.sie_block->gpsw.mask &= ~PSW_MASK_CC;
+ vcpu->arch.sie_block->gpsw.mask |= kvm_run->psw_mask &
+ PSW_MASK_CC;
+ }
+
+ kvm_run->kvm_dirty_regs = 0;
+}
+
+static void store_regs_fmt2(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
+{
+ kvm_run->s.regs.todpr = vcpu->arch.sie_block->todpr;
+ kvm_run->s.regs.pp = vcpu->arch.sie_block->pp;
+ kvm_run->s.regs.gbea = vcpu->arch.sie_block->gbea;
+ kvm_run->s.regs.bpbc = (vcpu->arch.sie_block->fpf & FPF_BPBC) == FPF_BPBC;
if (MACHINE_HAS_GS) {
- preempt_disable();
__ctl_set_bit(2, 4);
- if (current->thread.gs_cb) {
- vcpu->arch.host_gscb = current->thread.gs_cb;
- save_gs_cb(vcpu->arch.host_gscb);
- }
- if (vcpu->arch.gs_enabled) {
- current->thread.gs_cb = (struct gs_cb *)
- &vcpu->run->s.regs.gscb;
- restore_gs_cb(current->thread.gs_cb);
- }
+ if (vcpu->arch.gs_enabled)
+ save_gs_cb(current->thread.gs_cb);
+ preempt_disable();
+ current->thread.gs_cb = vcpu->arch.host_gscb;
+ restore_gs_cb(vcpu->arch.host_gscb);
preempt_enable();
+ if (!vcpu->arch.host_gscb)
+ __ctl_clear_bit(2, 4);
+ vcpu->arch.host_gscb = NULL;
}
- /* SIE will load etoken directly from SDNX and therefore kvm_run */
-
- kvm_run->kvm_dirty_regs = 0;
+ /* SIE will save etoken directly into SDNX and therefore kvm_run */
}
static void store_regs(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
@@ -4184,13 +4229,9 @@ static void store_regs(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
memcpy(&kvm_run->s.regs.crs, &vcpu->arch.sie_block->gcr, 128);
kvm_run->s.regs.cputm = kvm_s390_get_cpu_timer(vcpu);
kvm_run->s.regs.ckc = vcpu->arch.sie_block->ckc;
- kvm_run->s.regs.todpr = vcpu->arch.sie_block->todpr;
- kvm_run->s.regs.pp = vcpu->arch.sie_block->pp;
- kvm_run->s.regs.gbea = vcpu->arch.sie_block->gbea;
kvm_run->s.regs.pft = vcpu->arch.pfault_token;
kvm_run->s.regs.pfs = vcpu->arch.pfault_select;
kvm_run->s.regs.pfc = vcpu->arch.pfault_compare;
- kvm_run->s.regs.bpbc = (vcpu->arch.sie_block->fpf & FPF_BPBC) == FPF_BPBC;
save_access_regs(vcpu->run->s.regs.acrs);
restore_access_regs(vcpu->arch.host_acrs);
/* Save guest register state */
@@ -4199,19 +4240,8 @@ static void store_regs(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
/* Restore will be done lazily at return */
current->thread.fpu.fpc = vcpu->arch.host_fpregs.fpc;
current->thread.fpu.regs = vcpu->arch.host_fpregs.regs;
- if (MACHINE_HAS_GS) {
- __ctl_set_bit(2, 4);
- if (vcpu->arch.gs_enabled)
- save_gs_cb(current->thread.gs_cb);
- preempt_disable();
- current->thread.gs_cb = vcpu->arch.host_gscb;
- restore_gs_cb(vcpu->arch.host_gscb);
- preempt_enable();
- if (!vcpu->arch.host_gscb)
- __ctl_clear_bit(2, 4);
- vcpu->arch.host_gscb = NULL;
- }
- /* SIE will save etoken directly into SDNX and therefore kvm_run */
+ if (likely(!kvm_s390_pv_cpu_is_protected(vcpu)))
+ store_regs_fmt2(vcpu, kvm_run);
}
int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
--
2.25.0
^ permalink raw reply related
* [PATCH v3 22/37] KVM: s390: protvirt: Write sthyi data to instruction data area
From: Christian Borntraeger @ 2020-02-20 10:40 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik, Janosch Frank
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
From: Janosch Frank <frankja@linux.ibm.com>
STHYI data has to go through the bounce buffer.
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
[borntraeger@de.ibm.com: patch merging, splitting, fixing]
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
arch/s390/kvm/intercept.c | 15 ++++++++++-----
1 file changed, 10 insertions(+), 5 deletions(-)
diff --git a/arch/s390/kvm/intercept.c b/arch/s390/kvm/intercept.c
index 850045034016..a5de0e1975bb 100644
--- a/arch/s390/kvm/intercept.c
+++ b/arch/s390/kvm/intercept.c
@@ -392,7 +392,7 @@ int handle_sthyi(struct kvm_vcpu *vcpu)
goto out;
}
- if (addr & ~PAGE_MASK)
+ if (!kvm_s390_pv_cpu_is_protected(vcpu) && (addr & ~PAGE_MASK))
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
sctns = (void *)get_zeroed_page(GFP_KERNEL);
@@ -403,10 +403,15 @@ int handle_sthyi(struct kvm_vcpu *vcpu)
out:
if (!cc) {
- r = write_guest(vcpu, addr, reg2, sctns, PAGE_SIZE);
- if (r) {
- free_page((unsigned long)sctns);
- return kvm_s390_inject_prog_cond(vcpu, r);
+ if (kvm_s390_pv_cpu_is_protected(vcpu)) {
+ memcpy((void *)(sida_origin(vcpu->arch.sie_block)),
+ sctns, PAGE_SIZE);
+ } else {
+ r = write_guest(vcpu, addr, reg2, sctns, PAGE_SIZE);
+ if (r) {
+ free_page((unsigned long)sctns);
+ return kvm_s390_inject_prog_cond(vcpu, r);
+ }
}
}
--
2.25.0
^ permalink raw reply related
* [PATCH v3 24/37] KVM: s390: protvirt: disallow one_reg
From: Christian Borntraeger @ 2020-02-20 10:40 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik, Janosch Frank
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
From: Janosch Frank <frankja@linux.ibm.com>
A lot of the registers are controlled by the Ultravisor and never
visible to KVM. Some fields in the sie control block are overlayed, like
gbea. As no known userspace uses the ONE_REG interface on s390 if sync
regs are available, no functionality is lost if it is disabled for
protected guests.
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
[borntraeger@de.ibm.com: patch merging, splitting, fixing]
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
Documentation/virt/kvm/api.rst | 6 ++++--
arch/s390/kvm/kvm-s390.c | 3 +++
2 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 77e1edfe5d4c..58be7aba0db2 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -2117,7 +2117,8 @@ Errors:
====== ============================================================
ENOENT no such register
- EINVAL invalid register ID, or no such register
+ EINVAL invalid register ID, or no such register or used with VMs in
+ protected virtualization mode on s390
EPERM (arm64) register access not allowed before vcpu finalization
====== ============================================================
@@ -2552,7 +2553,8 @@ Errors include:
======== ============================================================
ENOENT no such register
- EINVAL invalid register ID, or no such register
+ EINVAL invalid register ID, or no such register or used with VMs in
+ protected virtualization mode on s390
EPERM (arm64) register access not allowed before vcpu finalization
======== ============================================================
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 4f7d70157b2c..436778d3dc14 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -4649,6 +4649,9 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
case KVM_SET_ONE_REG:
case KVM_GET_ONE_REG: {
struct kvm_one_reg reg;
+ r = -EINVAL;
+ if (kvm_s390_pv_cpu_is_protected(vcpu))
+ break;
r = -EFAULT;
if (copy_from_user(®, argp, sizeof(reg)))
break;
--
2.25.0
^ permalink raw reply related
* [PATCH v3 21/37] KVM: s390/mm: handle guest unpin events
From: Christian Borntraeger @ 2020-02-20 10:40 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
From: Claudio Imbrenda <imbrenda@linux.ibm.com>
The current code tries to first pin shared pages, if that fails (e.g.
because the page is not shared) it will export them. For shared pages
this means that we get a new intercept telling us that the guest is
unsharing that page. We will make the page secure at that point in time
and revoke the host access. This is synchronized with other host events,
e.g. the code will wait until host I/O has finished.
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Acked-by: David Hildenbrand <david@redhat.com>
[borntraeger@de.ibm.com: patch merging, splitting, fixing]
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
arch/s390/kvm/intercept.c | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/arch/s390/kvm/intercept.c b/arch/s390/kvm/intercept.c
index f4d533554157..850045034016 100644
--- a/arch/s390/kvm/intercept.c
+++ b/arch/s390/kvm/intercept.c
@@ -16,6 +16,7 @@
#include <asm/asm-offsets.h>
#include <asm/irq.h>
#include <asm/sysinfo.h>
+#include <asm/uv.h>
#include "kvm-s390.h"
#include "gaccess.h"
@@ -484,12 +485,35 @@ static int handle_pv_sclp(struct kvm_vcpu *vcpu)
return 0;
}
+static int handle_pv_uvc(struct kvm_vcpu *vcpu)
+{
+ struct uv_cb_share *guest_uvcb = (void *)vcpu->arch.sie_block->sidad;
+ struct uv_cb_cts uvcb = {
+ .header.cmd = UVC_CMD_UNPIN_PAGE_SHARED,
+ .header.len = sizeof(uvcb),
+ .guest_handle = kvm_s390_pv_get_handle(vcpu->kvm),
+ .gaddr = guest_uvcb->paddr,
+ };
+ int rc;
+
+ if (guest_uvcb->header.cmd != UVC_CMD_REMOVE_SHARED_ACCESS) {
+ WARN_ONCE(1, "Unexpected UVC 0x%x!\n", guest_uvcb->header.cmd);
+ return 0;
+ }
+ rc = gmap_make_secure(vcpu->arch.gmap, uvcb.gaddr, &uvcb);
+ if (rc == -EINVAL)
+ return 0;
+ return rc;
+}
+
static int handle_pv_notification(struct kvm_vcpu *vcpu)
{
if (vcpu->arch.sie_block->ipa == 0xb210)
return handle_pv_spx(vcpu);
if (vcpu->arch.sie_block->ipa == 0xb220)
return handle_pv_sclp(vcpu);
+ if (vcpu->arch.sie_block->ipa == 0xb9a4)
+ return handle_pv_uvc(vcpu);
return handle_instruction(vcpu);
}
--
2.25.0
^ permalink raw reply related
* [PATCH v3 33/37] KVM: s390: protvirt: Add UV cpu reset calls
From: Christian Borntraeger @ 2020-02-20 10:40 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik, Janosch Frank
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
From: Janosch Frank <frankja@linux.ibm.com>
For protected VMs, the VCPU resets are done by the Ultravisor, as KVM
has no access to the VCPU registers.
Note that the ultravisor will only accept a call for the exact reset
that has been requested.
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
[borntraeger@de.ibm.com: patch merging, splitting, fixing]
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
arch/s390/include/asm/uv.h | 6 ++++++
arch/s390/kvm/kvm-s390.c | 20 ++++++++++++++++++++
2 files changed, 26 insertions(+)
diff --git a/arch/s390/include/asm/uv.h b/arch/s390/include/asm/uv.h
index 4945e44e1528..cb95763ffb30 100644
--- a/arch/s390/include/asm/uv.h
+++ b/arch/s390/include/asm/uv.h
@@ -36,7 +36,10 @@
#define UVC_CMD_SET_SEC_CONF_PARAMS 0x0300
#define UVC_CMD_UNPACK_IMG 0x0301
#define UVC_CMD_VERIFY_IMG 0x0302
+#define UVC_CMD_CPU_RESET 0x0310
+#define UVC_CMD_CPU_RESET_INITIAL 0x0311
#define UVC_CMD_PREPARE_RESET 0x0320
+#define UVC_CMD_CPU_RESET_CLEAR 0x0321
#define UVC_CMD_CPU_SET_STATE 0x0330
#define UVC_CMD_SET_UNSHARE_ALL 0x0340
#define UVC_CMD_PIN_PAGE_SHARED 0x0341
@@ -59,8 +62,11 @@ enum uv_cmds_inst {
BIT_UVC_CMD_SET_SEC_PARMS = 11,
BIT_UVC_CMD_UNPACK_IMG = 13,
BIT_UVC_CMD_VERIFY_IMG = 14,
+ BIT_UVC_CMD_CPU_RESET = 15,
+ BIT_UVC_CMD_CPU_RESET_INITIAL = 16,
BIT_UVC_CMD_CPU_SET_STATE = 17,
BIT_UVC_CMD_PREPARE_RESET = 18,
+ BIT_UVC_CMD_CPU_PERFORM_CLEAR_RESET = 19,
BIT_UVC_CMD_UNSHARE_ALL = 20,
BIT_UVC_CMD_PIN_PAGE_SHARED = 21,
BIT_UVC_CMD_UNPIN_PAGE_SHARED = 22,
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 76ab9e4c8f55..d79ccd34b5cb 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -4705,6 +4705,7 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
void __user *argp = (void __user *)arg;
int idx;
long r;
+ u16 rc, rrc;
vcpu_load(vcpu);
@@ -4726,14 +4727,33 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
case KVM_S390_CLEAR_RESET:
r = 0;
kvm_arch_vcpu_ioctl_clear_reset(vcpu);
+ if (kvm_s390_pv_cpu_is_protected(vcpu)) {
+ r = uv_cmd_nodata(kvm_s390_pv_cpu_get_handle(vcpu),
+ UVC_CMD_CPU_RESET_CLEAR, &rc, &rrc);
+ VCPU_EVENT(vcpu, 3, "PROTVIRT RESET CLEAR VCPU: rc %x rrc %x",
+ rc, rrc);
+ }
break;
case KVM_S390_INITIAL_RESET:
r = 0;
kvm_arch_vcpu_ioctl_initial_reset(vcpu);
+ if (kvm_s390_pv_cpu_is_protected(vcpu)) {
+ r = uv_cmd_nodata(kvm_s390_pv_cpu_get_handle(vcpu),
+ UVC_CMD_CPU_RESET_INITIAL,
+ &rc, &rrc);
+ VCPU_EVENT(vcpu, 3, "PROTVIRT RESET INITIAL VCPU: rc %x rrc %x",
+ rc, rrc);
+ }
break;
case KVM_S390_NORMAL_RESET:
r = 0;
kvm_arch_vcpu_ioctl_normal_reset(vcpu);
+ if (kvm_s390_pv_cpu_is_protected(vcpu)) {
+ r = uv_cmd_nodata(kvm_s390_pv_cpu_get_handle(vcpu),
+ UVC_CMD_CPU_RESET, &rc, &rrc);
+ VCPU_EVENT(vcpu, 3, "PROTVIRT RESET NORMAL VCPU: rc %x rrc %x",
+ rc, rrc);
+ }
break;
case KVM_SET_ONE_REG:
case KVM_GET_ONE_REG: {
--
2.25.0
^ permalink raw reply related
* [PATCH v3 30/37] KVM: s390: protvirt: Support cmd 5 operation state
From: Christian Borntraeger @ 2020-02-20 10:40 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik, Janosch Frank
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
From: Janosch Frank <frankja@linux.ibm.com>
Code 5 for the set cpu state UV call tells the UV to load a PSW from
the SE header (first IPL) or from guest location 0x0 (diag 308 subcode
0/1). Also it sets the cpu into operating state afterwards, so we can
start it.
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
[borntraeger@de.ibm.com: patch merging, splitting, fixing]
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
arch/s390/include/asm/uv.h | 1 +
arch/s390/kvm/kvm-s390.c | 6 ++++++
2 files changed, 7 insertions(+)
diff --git a/arch/s390/include/asm/uv.h b/arch/s390/include/asm/uv.h
index 99e1a14ef909..4945e44e1528 100644
--- a/arch/s390/include/asm/uv.h
+++ b/arch/s390/include/asm/uv.h
@@ -169,6 +169,7 @@ struct uv_cb_unp {
#define PV_CPU_STATE_OPR 1
#define PV_CPU_STATE_STP 2
#define PV_CPU_STATE_CHKSTP 3
+#define PV_CPU_STATE_OPR_LOAD 5
struct uv_cb_cpu_set_state {
struct uv_cb_header header;
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index 036bff58e75a..f0b6609b65b6 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -3707,6 +3707,12 @@ int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
rc = kvm_s390_vcpu_start(vcpu);
break;
case KVM_MP_STATE_LOAD:
+ if (!kvm_s390_pv_cpu_is_protected(vcpu)) {
+ rc = -ENXIO;
+ break;
+ }
+ rc = kvm_s390_pv_set_cpu_state(vcpu, PV_CPU_STATE_OPR_LOAD);
+ break;
case KVM_MP_STATE_CHECK_STOP:
/* fall through - CHECK_STOP and LOAD are not supported yet */
default:
--
2.25.0
^ permalink raw reply related
* [PATCH v3 35/37] s390: protvirt: Add sysfs firmware interface for Ultravisor information
From: Christian Borntraeger @ 2020-02-20 10:40 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik, Janosch Frank
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
From: Janosch Frank <frankja@linux.ibm.com>
That information, e.g. the maximum number of guests or installed
Ultravisor facilities, is interesting for QEMU, Libvirt and
administrators.
Let's provide an easily parsable API to get that information.
Signed-off-by: Janosch Frank <frankja@linux.ibm.com>
---
arch/s390/kernel/uv.c | 86 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 86 insertions(+)
diff --git a/arch/s390/kernel/uv.c b/arch/s390/kernel/uv.c
index 4539003dac9d..550e9617c459 100644
--- a/arch/s390/kernel/uv.c
+++ b/arch/s390/kernel/uv.c
@@ -323,5 +323,91 @@ int arch_make_page_accessible(struct page *page)
return rc;
}
EXPORT_SYMBOL_GPL(arch_make_page_accessible);
+#endif
+
+#if defined(CONFIG_PROTECTED_VIRTUALIZATION_GUEST) || IS_ENABLED(CONFIG_KVM)
+static ssize_t uv_query_facilities(struct kobject *kobj,
+ struct kobj_attribute *attr, char *page)
+{
+ return snprintf(page, PAGE_SIZE, "%lx\n%lx\n%lx\n%lx\n",
+ uv_info.inst_calls_list[0],
+ uv_info.inst_calls_list[1],
+ uv_info.inst_calls_list[2],
+ uv_info.inst_calls_list[3]);
+}
+
+static struct kobj_attribute uv_query_facilities_attr =
+ __ATTR(facilities, 0444, uv_query_facilities, NULL);
+
+static ssize_t uv_query_max_guest_cpus(struct kobject *kobj,
+ struct kobj_attribute *attr, char *page)
+{
+ return snprintf(page, PAGE_SIZE, "%d\n",
+ uv_info.max_guest_cpus);
+}
+
+static struct kobj_attribute uv_query_max_guest_cpus_attr =
+ __ATTR(max_cpus, 0444, uv_query_max_guest_cpus, NULL);
+
+static ssize_t uv_query_max_guest_vms(struct kobject *kobj,
+ struct kobj_attribute *attr, char *page)
+{
+ return snprintf(page, PAGE_SIZE, "%d\n",
+ uv_info.max_num_sec_conf);
+}
+
+static struct kobj_attribute uv_query_max_guest_vms_attr =
+ __ATTR(max_guests, 0444, uv_query_max_guest_vms, NULL);
+
+static ssize_t uv_query_max_guest_addr(struct kobject *kobj,
+ struct kobj_attribute *attr, char *page)
+{
+ return snprintf(page, PAGE_SIZE, "%lx\n",
+ uv_info.max_sec_stor_addr);
+}
+
+static struct kobj_attribute uv_query_max_guest_addr_attr =
+ __ATTR(max_address, 0444, uv_query_max_guest_addr, NULL);
+
+static struct attribute *uv_query_attrs[] = {
+ &uv_query_facilities_attr.attr,
+ &uv_query_max_guest_cpus_attr.attr,
+ &uv_query_max_guest_vms_attr.attr,
+ &uv_query_max_guest_addr_attr.attr,
+ NULL,
+};
+
+static struct attribute_group uv_query_attr_group = {
+ .attrs = uv_query_attrs,
+};
+static struct kset *uv_query_kset;
+struct kobject *uv_kobj;
+
+static int __init uv_info_init(void)
+{
+ int rc = -ENOMEM;
+
+ if (!test_facility(158))
+ return 0;
+
+ uv_kobj = kobject_create_and_add("uv", firmware_kobj);
+ if (!uv_kobj)
+ return -ENOMEM;
+
+ uv_query_kset = kset_create_and_add("query", NULL, uv_kobj);
+ if (!uv_query_kset)
+ goto out_kobj;
+
+ rc = sysfs_create_group(&uv_query_kset->kobj, &uv_query_attr_group);
+ if (!rc)
+ return 0;
+
+ kset_unregister(uv_query_kset);
+out_kobj:
+ kobject_del(uv_kobj);
+ kobject_put(uv_kobj);
+ return rc;
+}
+device_initcall(uv_info_init);
#endif
--
2.25.0
^ permalink raw reply related
* [PATCH v3 15/37] KVM: s390: protvirt: Implement interrupt injection
From: Christian Borntraeger @ 2020-02-20 10:39 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
From: Michael Mueller <mimu@linux.ibm.com>
This defines the necessary data structures in the SIE control block to
inject machine checks,external and I/O interrupts. We first define the
the interrupt injection control, which defines the next interrupt to
inject. Then we define the fields that contain the payload for machine
checks,external and I/O interrupts.
This is then used to implement interruption injection for the following
list of interruption types:
- I/O (uses inject io interruption)
__deliver_io
- External (uses inject external interruption)
__deliver_cpu_timer
__deliver_ckc
__deliver_emergency_signal
__deliver_external_call
- cpu restart (uses inject restart interruption)
__deliver_restart
- machine checks (uses mcic, failing address and external damage)
__write_machine_check
Please note that posted interrupts (GISA) are not used for protected
guests as of today.
The service interrupt is handled in a followup patch.
Signed-off-by: Michael Mueller <mimu@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
[borntraeger@de.ibm.com: patch merging, splitting, fixing]
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
arch/s390/include/asm/kvm_host.h | 62 +++++++++++++----
arch/s390/kvm/interrupt.c | 115 +++++++++++++++++++++++--------
2 files changed, 138 insertions(+), 39 deletions(-)
diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
index c6694f47b73b..a13dc77f8b07 100644
--- a/arch/s390/include/asm/kvm_host.h
+++ b/arch/s390/include/asm/kvm_host.h
@@ -222,7 +222,15 @@ struct kvm_s390_sie_block {
__u8 icptcode; /* 0x0050 */
__u8 icptstatus; /* 0x0051 */
__u16 ihcpu; /* 0x0052 */
- __u8 reserved54[2]; /* 0x0054 */
+ __u8 reserved54; /* 0x0054 */
+#define IICTL_CODE_NONE 0x00
+#define IICTL_CODE_MCHK 0x01
+#define IICTL_CODE_EXT 0x02
+#define IICTL_CODE_IO 0x03
+#define IICTL_CODE_RESTART 0x04
+#define IICTL_CODE_SPECIFICATION 0x10
+#define IICTL_CODE_OPERAND 0x11
+ __u8 iictl; /* 0x0055 */
__u16 ipa; /* 0x0056 */
__u32 ipb; /* 0x0058 */
__u32 scaoh; /* 0x005c */
@@ -259,24 +267,48 @@ struct kvm_s390_sie_block {
#define HPID_KVM 0x4
#define HPID_VSIE 0x5
__u8 hpid; /* 0x00b8 */
- __u8 reservedb9[11]; /* 0x00b9 */
- __u16 extcpuaddr; /* 0x00c4 */
- __u16 eic; /* 0x00c6 */
+ __u8 reservedb9[7]; /* 0x00b9 */
+ union {
+ struct {
+ __u32 eiparams; /* 0x00c0 */
+ __u16 extcpuaddr; /* 0x00c4 */
+ __u16 eic; /* 0x00c6 */
+ };
+ __u64 mcic; /* 0x00c0 */
+ } __packed;
__u32 reservedc8; /* 0x00c8 */
- __u16 pgmilc; /* 0x00cc */
- __u16 iprcc; /* 0x00ce */
- __u32 dxc; /* 0x00d0 */
- __u16 mcn; /* 0x00d4 */
- __u8 perc; /* 0x00d6 */
- __u8 peratmid; /* 0x00d7 */
+ union {
+ struct {
+ __u16 pgmilc; /* 0x00cc */
+ __u16 iprcc; /* 0x00ce */
+ };
+ __u32 edc; /* 0x00cc */
+ } __packed;
+ union {
+ struct {
+ __u32 dxc; /* 0x00d0 */
+ __u16 mcn; /* 0x00d4 */
+ __u8 perc; /* 0x00d6 */
+ __u8 peratmid; /* 0x00d7 */
+ };
+ __u64 faddr; /* 0x00d0 */
+ } __packed;
__u64 peraddr; /* 0x00d8 */
__u8 eai; /* 0x00e0 */
__u8 peraid; /* 0x00e1 */
__u8 oai; /* 0x00e2 */
__u8 armid; /* 0x00e3 */
__u8 reservede4[4]; /* 0x00e4 */
- __u64 tecmc; /* 0x00e8 */
- __u8 reservedf0[12]; /* 0x00f0 */
+ union {
+ __u64 tecmc; /* 0x00e8 */
+ struct {
+ __u16 subchannel_id; /* 0x00e8 */
+ __u16 subchannel_nr; /* 0x00ea */
+ __u32 io_int_parm; /* 0x00ec */
+ __u32 io_int_word; /* 0x00f0 */
+ };
+ } __packed;
+ __u8 reservedf4[8]; /* 0x00f4 */
#define CRYCB_FORMAT_MASK 0x00000003
#define CRYCB_FORMAT0 0x00000000
#define CRYCB_FORMAT1 0x00000001
@@ -546,6 +578,12 @@ enum irq_types {
#define IRQ_PEND_MCHK_MASK ((1UL << IRQ_PEND_MCHK_REP) | \
(1UL << IRQ_PEND_MCHK_EX))
+#define IRQ_PEND_EXT_II_MASK ((1UL << IRQ_PEND_EXT_CPU_TIMER) | \
+ (1UL << IRQ_PEND_EXT_CLOCK_COMP) | \
+ (1UL << IRQ_PEND_EXT_EMERGENCY) | \
+ (1UL << IRQ_PEND_EXT_EXTERNAL) | \
+ (1UL << IRQ_PEND_EXT_SERVICE))
+
struct kvm_s390_interrupt_info {
struct list_head list;
u64 type;
diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
index 0cebebf56515..61310b1f5b62 100644
--- a/arch/s390/kvm/interrupt.c
+++ b/arch/s390/kvm/interrupt.c
@@ -387,6 +387,12 @@ static unsigned long deliverable_irqs(struct kvm_vcpu *vcpu)
__clear_bit(IRQ_PEND_EXT_SERVICE, &active_mask);
if (psw_mchk_disabled(vcpu))
active_mask &= ~IRQ_PEND_MCHK_MASK;
+ /* PV guest cpus can have a single interruption injected at a time. */
+ if (kvm_s390_pv_cpu_is_protected(vcpu) &&
+ vcpu->arch.sie_block->iictl != IICTL_CODE_NONE)
+ active_mask &= ~(IRQ_PEND_EXT_II_MASK |
+ IRQ_PEND_IO_MASK |
+ IRQ_PEND_MCHK_MASK);
/*
* Check both floating and local interrupt's cr14 because
* bit IRQ_PEND_MCHK_REP could be set in both cases.
@@ -479,19 +485,23 @@ static void set_intercept_indicators(struct kvm_vcpu *vcpu)
static int __must_check __deliver_cpu_timer(struct kvm_vcpu *vcpu)
{
struct kvm_s390_local_interrupt *li = &vcpu->arch.local_int;
- int rc;
+ int rc = 0;
vcpu->stat.deliver_cputm++;
trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id, KVM_S390_INT_CPU_TIMER,
0, 0);
-
- rc = put_guest_lc(vcpu, EXT_IRQ_CPU_TIMER,
- (u16 *)__LC_EXT_INT_CODE);
- rc |= put_guest_lc(vcpu, 0, (u16 *)__LC_EXT_CPU_ADDR);
- rc |= write_guest_lc(vcpu, __LC_EXT_OLD_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
- rc |= read_guest_lc(vcpu, __LC_EXT_NEW_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
+ if (kvm_s390_pv_cpu_is_protected(vcpu)) {
+ vcpu->arch.sie_block->iictl = IICTL_CODE_EXT;
+ vcpu->arch.sie_block->eic = EXT_IRQ_CPU_TIMER;
+ } else {
+ rc = put_guest_lc(vcpu, EXT_IRQ_CPU_TIMER,
+ (u16 *)__LC_EXT_INT_CODE);
+ rc |= put_guest_lc(vcpu, 0, (u16 *)__LC_EXT_CPU_ADDR);
+ rc |= write_guest_lc(vcpu, __LC_EXT_OLD_PSW,
+ &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
+ rc |= read_guest_lc(vcpu, __LC_EXT_NEW_PSW,
+ &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
+ }
clear_bit(IRQ_PEND_EXT_CPU_TIMER, &li->pending_irqs);
return rc ? -EFAULT : 0;
}
@@ -499,19 +509,23 @@ static int __must_check __deliver_cpu_timer(struct kvm_vcpu *vcpu)
static int __must_check __deliver_ckc(struct kvm_vcpu *vcpu)
{
struct kvm_s390_local_interrupt *li = &vcpu->arch.local_int;
- int rc;
+ int rc = 0;
vcpu->stat.deliver_ckc++;
trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id, KVM_S390_INT_CLOCK_COMP,
0, 0);
-
- rc = put_guest_lc(vcpu, EXT_IRQ_CLK_COMP,
- (u16 __user *)__LC_EXT_INT_CODE);
- rc |= put_guest_lc(vcpu, 0, (u16 *)__LC_EXT_CPU_ADDR);
- rc |= write_guest_lc(vcpu, __LC_EXT_OLD_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
- rc |= read_guest_lc(vcpu, __LC_EXT_NEW_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
+ if (kvm_s390_pv_cpu_is_protected(vcpu)) {
+ vcpu->arch.sie_block->iictl = IICTL_CODE_EXT;
+ vcpu->arch.sie_block->eic = EXT_IRQ_CLK_COMP;
+ } else {
+ rc = put_guest_lc(vcpu, EXT_IRQ_CLK_COMP,
+ (u16 __user *)__LC_EXT_INT_CODE);
+ rc |= put_guest_lc(vcpu, 0, (u16 *)__LC_EXT_CPU_ADDR);
+ rc |= write_guest_lc(vcpu, __LC_EXT_OLD_PSW,
+ &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
+ rc |= read_guest_lc(vcpu, __LC_EXT_NEW_PSW,
+ &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
+ }
clear_bit(IRQ_PEND_EXT_CLOCK_COMP, &li->pending_irqs);
return rc ? -EFAULT : 0;
}
@@ -553,6 +567,20 @@ static int __write_machine_check(struct kvm_vcpu *vcpu,
union mci mci;
int rc;
+ /*
+ * All other possible payload for a machine check (e.g. the register
+ * contents in the save area) will be handled by the ultravisor, as
+ * the hypervisor does not not have the needed information for
+ * protected guests.
+ */
+ if (kvm_s390_pv_cpu_is_protected(vcpu)) {
+ vcpu->arch.sie_block->iictl = IICTL_CODE_MCHK;
+ vcpu->arch.sie_block->mcic = mchk->mcic;
+ vcpu->arch.sie_block->faddr = mchk->failing_storage_address;
+ vcpu->arch.sie_block->edc = mchk->ext_damage_code;
+ return 0;
+ }
+
mci.val = mchk->mcic;
/* take care of lazy register loading */
save_fpu_regs();
@@ -696,17 +724,21 @@ static int __must_check __deliver_machine_check(struct kvm_vcpu *vcpu)
static int __must_check __deliver_restart(struct kvm_vcpu *vcpu)
{
struct kvm_s390_local_interrupt *li = &vcpu->arch.local_int;
- int rc;
+ int rc = 0;
VCPU_EVENT(vcpu, 3, "%s", "deliver: cpu restart");
vcpu->stat.deliver_restart_signal++;
trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id, KVM_S390_RESTART, 0, 0);
- rc = write_guest_lc(vcpu,
- offsetof(struct lowcore, restart_old_psw),
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
- rc |= read_guest_lc(vcpu, offsetof(struct lowcore, restart_psw),
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
+ if (kvm_s390_pv_cpu_is_protected(vcpu)) {
+ vcpu->arch.sie_block->iictl = IICTL_CODE_RESTART;
+ } else {
+ rc = write_guest_lc(vcpu,
+ offsetof(struct lowcore, restart_old_psw),
+ &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
+ rc |= read_guest_lc(vcpu, offsetof(struct lowcore, restart_psw),
+ &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
+ }
clear_bit(IRQ_PEND_RESTART, &li->pending_irqs);
return rc ? -EFAULT : 0;
}
@@ -748,6 +780,12 @@ static int __must_check __deliver_emergency_signal(struct kvm_vcpu *vcpu)
vcpu->stat.deliver_emergency_signal++;
trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id, KVM_S390_INT_EMERGENCY,
cpu_addr, 0);
+ if (kvm_s390_pv_cpu_is_protected(vcpu)) {
+ vcpu->arch.sie_block->iictl = IICTL_CODE_EXT;
+ vcpu->arch.sie_block->eic = EXT_IRQ_EMERGENCY_SIG;
+ vcpu->arch.sie_block->extcpuaddr = cpu_addr;
+ return 0;
+ }
rc = put_guest_lc(vcpu, EXT_IRQ_EMERGENCY_SIG,
(u16 *)__LC_EXT_INT_CODE);
@@ -776,6 +814,12 @@ static int __must_check __deliver_external_call(struct kvm_vcpu *vcpu)
trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id,
KVM_S390_INT_EXTERNAL_CALL,
extcall.code, 0);
+ if (kvm_s390_pv_cpu_is_protected(vcpu)) {
+ vcpu->arch.sie_block->iictl = IICTL_CODE_EXT;
+ vcpu->arch.sie_block->eic = EXT_IRQ_EXTERNAL_CALL;
+ vcpu->arch.sie_block->extcpuaddr = extcall.code;
+ return 0;
+ }
rc = put_guest_lc(vcpu, EXT_IRQ_EXTERNAL_CALL,
(u16 *)__LC_EXT_INT_CODE);
@@ -1028,6 +1072,15 @@ static int __do_deliver_io(struct kvm_vcpu *vcpu, struct kvm_s390_io_info *io)
{
int rc;
+ if (kvm_s390_pv_cpu_is_protected(vcpu)) {
+ vcpu->arch.sie_block->iictl = IICTL_CODE_IO;
+ vcpu->arch.sie_block->subchannel_id = io->subchannel_id;
+ vcpu->arch.sie_block->subchannel_nr = io->subchannel_nr;
+ vcpu->arch.sie_block->io_int_parm = io->io_int_parm;
+ vcpu->arch.sie_block->io_int_word = io->io_int_word;
+ return 0;
+ }
+
rc = put_guest_lc(vcpu, io->subchannel_id, (u16 *)__LC_SUBCHANNEL_ID);
rc |= put_guest_lc(vcpu, io->subchannel_nr, (u16 *)__LC_SUBCHANNEL_NR);
rc |= put_guest_lc(vcpu, io->io_int_parm, (u32 *)__LC_IO_INT_PARM);
@@ -1421,7 +1474,7 @@ static int __inject_extcall(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq)
if (kvm_get_vcpu_by_id(vcpu->kvm, src_id) == NULL)
return -EINVAL;
- if (sclp.has_sigpif)
+ if (sclp.has_sigpif && !kvm_s390_pv_cpu_get_handle(vcpu))
return sca_inject_ext_call(vcpu, src_id);
if (test_and_set_bit(IRQ_PEND_EXT_EXTERNAL, &li->pending_irqs))
@@ -1773,7 +1826,14 @@ static int __inject_io(struct kvm *kvm, struct kvm_s390_interrupt_info *inti)
kvm->stat.inject_io++;
isc = int_word_to_isc(inti->io.io_int_word);
- if (gi->origin && inti->type & KVM_S390_INT_IO_AI_MASK) {
+ /*
+ * Do not make use of gisa in protected mode. We do not use the lock
+ * checking variant as this is just a performance optimization and we
+ * do not hold the lock here. This is ok as the code will pick
+ * interrupts from both "lists" for delivery.
+ */
+ if (!kvm_s390_pv_get_handle(kvm) &&
+ gi->origin && inti->type & KVM_S390_INT_IO_AI_MASK) {
VM_EVENT(kvm, 4, "%s isc %1u", "inject: I/O (AI/gisa)", isc);
gisa_set_ipm_gisc(gi->origin, isc);
kfree(inti);
@@ -1834,7 +1894,8 @@ static void __floating_irq_kick(struct kvm *kvm, u64 type)
break;
case KVM_S390_INT_IO_MIN...KVM_S390_INT_IO_MAX:
if (!(type & KVM_S390_INT_IO_AI_MASK &&
- kvm->arch.gisa_int.origin))
+ kvm->arch.gisa_int.origin) ||
+ kvm_s390_pv_cpu_get_handle(dst_vcpu))
kvm_s390_set_cpuflags(dst_vcpu, CPUSTAT_IO_INT);
break;
default:
--
2.25.0
^ permalink raw reply related
* [PATCH v3 03/37] s390/protvirt: introduce host side setup
From: Christian Borntraeger @ 2020-02-20 10:39 UTC (permalink / raw)
To: Christian Borntraeger, Janosch Frank
Cc: KVM, Cornelia Huck, David Hildenbrand, Thomas Huth,
Ulrich Weigand, Claudio Imbrenda, linux-s390, Michael Mueller,
Vasily Gorbik
In-Reply-To: <20200220104020.5343-1-borntraeger@de.ibm.com>
From: Vasily Gorbik <gor@linux.ibm.com>
Add "prot_virt" command line option which controls if the kernel
protected VMs support is enabled at early boot time. This has to be
done early, because it needs large amounts of memory and will disable
some features like STP time sync for the lpar.
Extend ultravisor info definitions and expose it via uv_info struct
filled in during startup.
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: David Hildenbrand <david@redhat.com>
[borntraeger@de.ibm.com: patch merging, splitting, fixing]
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
.../admin-guide/kernel-parameters.txt | 5 ++
arch/s390/boot/Makefile | 2 +-
arch/s390/boot/uv.c | 21 +++++++-
arch/s390/include/asm/uv.h | 46 +++++++++++++++-
arch/s390/kernel/Makefile | 1 +
arch/s390/kernel/setup.c | 4 --
arch/s390/kernel/uv.c | 52 +++++++++++++++++++
7 files changed, 123 insertions(+), 8 deletions(-)
create mode 100644 arch/s390/kernel/uv.c
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index dbc22d684627..b0beae9b9e36 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -3795,6 +3795,11 @@
before loading.
See Documentation/admin-guide/blockdev/ramdisk.rst.
+ prot_virt= [S390] enable hosting protected virtual machines
+ isolated from the hypervisor (if hardware supports
+ that).
+ Format: <bool>
+
psi= [KNL] Enable or disable pressure stall information
tracking.
Format: <bool>
diff --git a/arch/s390/boot/Makefile b/arch/s390/boot/Makefile
index e2c47d3a1c89..30f1811540c5 100644
--- a/arch/s390/boot/Makefile
+++ b/arch/s390/boot/Makefile
@@ -37,7 +37,7 @@ CFLAGS_sclp_early_core.o += -I$(srctree)/drivers/s390/char
obj-y := head.o als.o startup.o mem_detect.o ipl_parm.o ipl_report.o
obj-y += string.o ebcdic.o sclp_early_core.o mem.o ipl_vmparm.o cmdline.o
obj-y += version.o pgm_check_info.o ctype.o text_dma.o
-obj-$(CONFIG_PROTECTED_VIRTUALIZATION_GUEST) += uv.o
+obj-$(findstring y, $(CONFIG_PROTECTED_VIRTUALIZATION_GUEST) $(CONFIG_PGSTE)) += uv.o
obj-$(CONFIG_RELOCATABLE) += machine_kexec_reloc.o
obj-$(CONFIG_RANDOMIZE_BASE) += kaslr.o
targets := bzImage startup.a section_cmp.boot.data section_cmp.boot.preserved.data $(obj-y)
diff --git a/arch/s390/boot/uv.c b/arch/s390/boot/uv.c
index 3f501159ee9f..eeb121108242 100644
--- a/arch/s390/boot/uv.c
+++ b/arch/s390/boot/uv.c
@@ -3,7 +3,13 @@
#include <asm/facility.h>
#include <asm/sections.h>
+/* will be used in arch/s390/kernel/uv.c */
+#ifdef CONFIG_PROTECTED_VIRTUALIZATION_GUEST
int __bootdata_preserved(prot_virt_guest);
+#endif
+#if IS_ENABLED(CONFIG_KVM)
+struct uv_info __bootdata_preserved(uv_info);
+#endif
void uv_query_info(void)
{
@@ -19,7 +25,20 @@ void uv_query_info(void)
if (uv_call(0, (uint64_t)&uvcb) && uvcb.header.rc != 0x100)
return;
- if (test_bit_inv(BIT_UVC_CMD_SET_SHARED_ACCESS, (unsigned long *)uvcb.inst_calls_list) &&
+ if (IS_ENABLED(CONFIG_KVM)) {
+ memcpy(uv_info.inst_calls_list, uvcb.inst_calls_list, sizeof(uv_info.inst_calls_list));
+ uv_info.uv_base_stor_len = uvcb.uv_base_stor_len;
+ uv_info.guest_base_stor_len = uvcb.conf_base_phys_stor_len;
+ uv_info.guest_virt_base_stor_len = uvcb.conf_base_virt_stor_len;
+ uv_info.guest_virt_var_stor_len = uvcb.conf_virt_var_stor_len;
+ uv_info.guest_cpu_stor_len = uvcb.cpu_stor_len;
+ uv_info.max_sec_stor_addr = ALIGN(uvcb.max_guest_stor_addr, PAGE_SIZE);
+ uv_info.max_num_sec_conf = uvcb.max_num_sec_conf;
+ uv_info.max_guest_cpus = uvcb.max_guest_cpus;
+ }
+
+ if (IS_ENABLED(CONFIG_PROTECTED_VIRTUALIZATION_GUEST) &&
+ test_bit_inv(BIT_UVC_CMD_SET_SHARED_ACCESS, (unsigned long *)uvcb.inst_calls_list) &&
test_bit_inv(BIT_UVC_CMD_REMOVE_SHARED_ACCESS, (unsigned long *)uvcb.inst_calls_list))
prot_virt_guest = 1;
}
diff --git a/arch/s390/include/asm/uv.h b/arch/s390/include/asm/uv.h
index 4093a2856929..422aee15c9cc 100644
--- a/arch/s390/include/asm/uv.h
+++ b/arch/s390/include/asm/uv.h
@@ -44,7 +44,19 @@ struct uv_cb_qui {
struct uv_cb_header header;
u64 reserved08;
u64 inst_calls_list[4];
- u64 reserved30[15];
+ u64 reserved30[2];
+ u64 uv_base_stor_len;
+ u64 reserved48;
+ u64 conf_base_phys_stor_len;
+ u64 conf_base_virt_stor_len;
+ u64 conf_virt_var_stor_len;
+ u64 cpu_stor_len;
+ u32 reserved70[3];
+ u32 max_num_sec_conf;
+ u64 max_guest_stor_addr;
+ u8 reserved88[158 - 136];
+ u16 max_guest_cpus;
+ u8 reserveda0[200 - 160];
} __packed __aligned(8);
struct uv_cb_share {
@@ -69,6 +81,20 @@ static inline int uv_call(unsigned long r1, unsigned long r2)
return cc;
}
+struct uv_info {
+ unsigned long inst_calls_list[4];
+ unsigned long uv_base_stor_len;
+ unsigned long guest_base_stor_len;
+ unsigned long guest_virt_base_stor_len;
+ unsigned long guest_virt_var_stor_len;
+ unsigned long guest_cpu_stor_len;
+ unsigned long max_sec_stor_addr;
+ unsigned int max_num_sec_conf;
+ unsigned short max_guest_cpus;
+};
+
+extern struct uv_info uv_info;
+
#ifdef CONFIG_PROTECTED_VIRTUALIZATION_GUEST
extern int prot_virt_guest;
@@ -121,11 +147,27 @@ static inline int uv_remove_shared(unsigned long addr)
return share(addr, UVC_CMD_REMOVE_SHARED_ACCESS);
}
-void uv_query_info(void);
#else
#define is_prot_virt_guest() 0
static inline int uv_set_shared(unsigned long addr) { return 0; }
static inline int uv_remove_shared(unsigned long addr) { return 0; }
+#endif
+
+#if IS_ENABLED(CONFIG_KVM)
+extern int prot_virt_host;
+
+static inline int is_prot_virt_host(void)
+{
+ return prot_virt_host;
+}
+#else
+#define is_prot_virt_host() 0
+#endif
+
+#if defined(CONFIG_PROTECTED_VIRTUALIZATION_GUEST) || \
+ IS_ENABLED(CONFIG_KVM)
+void uv_query_info(void);
+#else
static inline void uv_query_info(void) {}
#endif
diff --git a/arch/s390/kernel/Makefile b/arch/s390/kernel/Makefile
index 2b1203cf7be6..22bfb8d5084e 100644
--- a/arch/s390/kernel/Makefile
+++ b/arch/s390/kernel/Makefile
@@ -78,6 +78,7 @@ obj-$(CONFIG_PERF_EVENTS) += perf_cpum_cf_events.o perf_regs.o
obj-$(CONFIG_PERF_EVENTS) += perf_cpum_cf_diag.o
obj-$(CONFIG_TRACEPOINTS) += trace.o
+obj-$(findstring y, $(CONFIG_PROTECTED_VIRTUALIZATION_GUEST) $(CONFIG_PGSTE)) += uv.o
# vdso
obj-y += vdso64/
diff --git a/arch/s390/kernel/setup.c b/arch/s390/kernel/setup.c
index b2c2f75860e8..a2496382175e 100644
--- a/arch/s390/kernel/setup.c
+++ b/arch/s390/kernel/setup.c
@@ -92,10 +92,6 @@ char elf_platform[ELF_PLATFORM_SIZE];
unsigned long int_hwcap = 0;
-#ifdef CONFIG_PROTECTED_VIRTUALIZATION_GUEST
-int __bootdata_preserved(prot_virt_guest);
-#endif
-
int __bootdata(noexec_disabled);
int __bootdata(memory_end_set);
unsigned long __bootdata(memory_end);
diff --git a/arch/s390/kernel/uv.c b/arch/s390/kernel/uv.c
new file mode 100644
index 000000000000..b1f936710360
--- /dev/null
+++ b/arch/s390/kernel/uv.c
@@ -0,0 +1,52 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Common Ultravisor functions and initialization
+ *
+ * Copyright IBM Corp. 2019, 2020
+ */
+#define KMSG_COMPONENT "prot_virt"
+#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt
+
+#include <linux/kernel.h>
+#include <linux/types.h>
+#include <linux/sizes.h>
+#include <linux/bitmap.h>
+#include <linux/memblock.h>
+#include <asm/facility.h>
+#include <asm/sections.h>
+#include <asm/uv.h>
+
+/* the bootdata_preserved fields come from ones in arch/s390/boot/uv.c */
+#ifdef CONFIG_PROTECTED_VIRTUALIZATION_GUEST
+int __bootdata_preserved(prot_virt_guest);
+#endif
+
+#if IS_ENABLED(CONFIG_KVM)
+int prot_virt_host;
+EXPORT_SYMBOL(prot_virt_host);
+struct uv_info __bootdata_preserved(uv_info);
+EXPORT_SYMBOL(uv_info);
+
+static int __init prot_virt_setup(char *val)
+{
+ bool enabled;
+ int rc;
+
+ rc = kstrtobool(val, &enabled);
+ if (!rc && enabled)
+ prot_virt_host = 1;
+
+ if (is_prot_virt_guest() && prot_virt_host) {
+ prot_virt_host = 0;
+ pr_warn("Protected virtualization not available in protected guests.");
+ }
+
+ if (prot_virt_host && !test_facility(158)) {
+ prot_virt_host = 0;
+ pr_warn("Protected virtualization not supported by the hardware.");
+ }
+
+ return rc;
+}
+early_param("prot_virt", prot_virt_setup);
+#endif
--
2.25.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.