Linux userland API discussions
 help / color / mirror / Atom feed
* Re: [PATCH 1/2] xen: Implement ioctl to restrict privcmd to a specific domain
From: Frediano Ziglio @ 2014-07-31 14:43 UTC (permalink / raw)
  To: David Vrabel
  Cc: xen-devel, Boris Ostrovsky, Ian Campbell, linux-kernel, linux-api
In-Reply-To: <E1XCr4q-0007Nh-3J@lists.xen.org>

On Thu, 2014-07-31 at 15:11 +0100, David Vrabel wrote:
> On 31/07/14 14:53, Ian Campbell wrote:
> > On Thu, 2014-07-31 at 14:16 +0100, Frediano Ziglio wrote:
> > 
> >>  include/xen/interface/domctl.h     | 1090 ++++++++++++++++++++++++++++++++++++
> > 
> > domctl is an stable toolstack only hypervisor interface, so the kernel
> > cannot use it because it would then break.
> 
> Ok.  I guess we'll have to resurrect the idea to do something with XSM.
> 
> David
> 

The code just require that:
- sizeof(struct xen_domctl) does not increase;
- position and size of cmd, domain and interface_version does not
change;
- XEN_DOMCTL_createdomain is 1.

For safety there is a check on interface_version.

Frediano

^ permalink raw reply

* Re: [PATCH 1/2] xen: Implement ioctl to restrict privcmd to a specific domain
From: David Vrabel @ 2014-07-31 14:11 UTC (permalink / raw)
  To: Ian Campbell, Frediano Ziglio
  Cc: xen-devel, Boris Ostrovsky, linux-api, linux-kernel
In-Reply-To: <1406814787.10395.2.camel@kazak.uk.xensource.com>

On 31/07/14 14:53, Ian Campbell wrote:
> On Thu, 2014-07-31 at 14:16 +0100, Frediano Ziglio wrote:
> 
>>  include/xen/interface/domctl.h     | 1090 ++++++++++++++++++++++++++++++++++++
> 
> domctl is an stable toolstack only hypervisor interface, so the kernel
> cannot use it because it would then break.

Ok.  I guess we'll have to resurrect the idea to do something with XSM.

David

^ permalink raw reply

* Re: [PATCH 1/2] xen: Implement ioctl to restrict privcmd to a specific domain
From: Ian Campbell @ 2014-07-31 13:53 UTC (permalink / raw)
  To: Frediano Ziglio
  Cc: linux-api, linux-kernel, David Vrabel, xen-devel, Boris Ostrovsky
In-Reply-To: <E1XCqEY-0005zn-RQ@lists.xen.org>

On Thu, 2014-07-31 at 14:16 +0100, Frediano Ziglio wrote:

>  include/xen/interface/domctl.h     | 1090 ++++++++++++++++++++++++++++++++++++

domctl is an stable toolstack only hypervisor interface, so the kernel
cannot use it because it would then break.

Ian.

^ permalink raw reply

* [PATCH 2/2] xen: Implement ioctl to restrict event channels to a specific domain
From: Frediano Ziglio @ 2014-07-31 13:18 UTC (permalink / raw)
  To: Konrad Rzeszutek Wilk, Boris Ostrovsky, David Vrabel
  Cc: xen-devel, linux-kernel, linux-api

Add a RESTRICT ioctl to /dev/xen/evtchn, which allows an event channel
file descriptor to be restricted to only working with a particular domain.

Signed-off-by: Frediano Ziglio <frediano.ziglio@citrix.com>
---
 drivers/xen/evtchn.c      | 40 ++++++++++++++++++++++++++++++++++++++++
 include/uapi/xen/evtchn.h | 13 +++++++++++++
 2 files changed, 53 insertions(+)

diff --git a/drivers/xen/evtchn.c b/drivers/xen/evtchn.c
index 00f40f0..a9934b7 100644
--- a/drivers/xen/evtchn.c
+++ b/drivers/xen/evtchn.c
@@ -71,8 +71,12 @@ struct per_user_data {
 	wait_queue_head_t evtchn_wait;
 	struct fasync_struct *evtchn_async_queue;
 	const char *name;
+
+	domid_t restrict_domid;
 };
 
+#define UNRESTRICTED_DOMID ((domid_t)-1)
+
 struct user_evtchn {
 	struct rb_node node;
 	struct per_user_data *user;
@@ -349,6 +353,10 @@ static long evtchn_ioctl(struct file *file,
 		struct ioctl_evtchn_bind_virq bind;
 		struct evtchn_bind_virq bind_virq;
 
+		rc = -EACCES;
+		if (u->restrict_domid != UNRESTRICTED_DOMID)
+			break;
+
 		rc = -EFAULT;
 		if (copy_from_user(&bind, uarg, sizeof(bind)))
 			break;
@@ -374,6 +382,11 @@ static long evtchn_ioctl(struct file *file,
 		if (copy_from_user(&bind, uarg, sizeof(bind)))
 			break;
 
+		rc = -EACCES;
+		if (u->restrict_domid != UNRESTRICTED_DOMID &&
+		    u->restrict_domid != bind.remote_domain)
+			break;
+
 		bind_interdomain.remote_dom  = bind.remote_domain;
 		bind_interdomain.remote_port = bind.remote_port;
 		rc = HYPERVISOR_event_channel_op(EVTCHNOP_bind_interdomain,
@@ -391,6 +404,10 @@ static long evtchn_ioctl(struct file *file,
 		struct ioctl_evtchn_bind_unbound_port bind;
 		struct evtchn_alloc_unbound alloc_unbound;
 
+		rc = -EACCES;
+		if (u->restrict_domid != UNRESTRICTED_DOMID)
+			break;
+
 		rc = -EFAULT;
 		if (copy_from_user(&bind, uarg, sizeof(bind)))
 			break;
@@ -459,6 +476,27 @@ static long evtchn_ioctl(struct file *file,
 		break;
 	}
 
+	case IOCTL_EVTCHN_RESTRICT_DOMID: {
+		struct ioctl_evtchn_restrict_domid ierd;
+
+		rc = -EACCES;
+		if (u->restrict_domid != UNRESTRICTED_DOMID)
+			break;
+
+		rc = -EFAULT;
+		if (copy_from_user(&ierd, uarg, sizeof(ierd)))
+			break;
+
+		rc = -EINVAL;
+		if (ierd.domid == 0 || ierd.domid >= DOMID_FIRST_RESERVED)
+			break;
+
+		u->restrict_domid = ierd.domid;
+		rc = 0;
+
+		break;
+	}
+
 	default:
 		rc = -ENOSYS;
 		break;
@@ -514,6 +552,8 @@ static int evtchn_open(struct inode *inode, struct file *filp)
 	mutex_init(&u->ring_cons_mutex);
 	spin_lock_init(&u->ring_prod_lock);
 
+	u->restrict_domid = UNRESTRICTED_DOMID;
+
 	filp->private_data = u;
 
 	return nonseekable_open(inode, filp);
diff --git a/include/uapi/xen/evtchn.h b/include/uapi/xen/evtchn.h
index 14e833e..72f5492 100644
--- a/include/uapi/xen/evtchn.h
+++ b/include/uapi/xen/evtchn.h
@@ -85,4 +85,17 @@ struct ioctl_evtchn_notify {
 #define IOCTL_EVTCHN_RESET				\
 	_IOC(_IOC_NONE, 'E', 5, 0)
 
+/* Restrict this file descriptor so that it can only be applied to a
+ * nominated domain.  Once a file descriptor has been restricted it
+ * cannot be de-restricted, and must be closed and re-openned.  Event
+ * channels which were bound before restricting remain bound
+ * afterwards, and can be notified as usual.
+ */
+#define IOCTL_EVTCHN_RESTRICT_DOMID			\
+	_IOC(_IOC_NONE, 'E', 6, sizeof(struct ioctl_evtchn_restrict_domid))
+struct ioctl_evtchn_restrict_domid {
+	domid_t domid;
+};
+
+
 #endif /* __LINUX_PUBLIC_EVTCHN_H__ */
-- 
1.9.1

^ permalink raw reply related

* [PATCH 1/2] xen: Implement ioctl to restrict privcmd to a specific domain
From: Frediano Ziglio @ 2014-07-31 13:16 UTC (permalink / raw)
  To: Konrad Rzeszutek Wilk, Boris Ostrovsky, David Vrabel
  Cc: xen-devel, linux-kernel, linux-api

Add a RESTRICT ioctl to /dev/xen/privcmd, which allows privileged commands
file descriptor to be restricted to only working with a particular domain.

Signed-off-by: Frediano Ziglio <frediano.ziglio@citrix.com>
---
 drivers/xen/privcmd.c              |  209 ++++++-
 include/uapi/xen/privcmd.h         |    6 +
 include/xen/interface/domctl.h     | 1090 ++++++++++++++++++++++++++++++++++++
 include/xen/interface/hvm/hvm_op.h |   66 +++
 include/xen/interface/memory.h     |    8 +
 include/xen/interface/xen.h        |    1 +
 6 files changed, 1373 insertions(+), 7 deletions(-)
 create mode 100644 include/xen/interface/domctl.h

diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
index 569a13b..c177850 100644
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -32,6 +32,10 @@
 #include <xen/xen.h>
 #include <xen/privcmd.h>
 #include <xen/interface/xen.h>
+#include <xen/interface/sched.h>
+#include <xen/interface/memory.h>
+#include <xen/interface/domctl.h>
+#include <xen/interface/hvm/hvm_op.h>
 #include <xen/features.h>
 #include <xen/page.h>
 #include <xen/xen-ops.h>
@@ -43,24 +47,173 @@ MODULE_LICENSE("GPL");
 
 #define PRIV_VMA_LOCKED ((void *)1)
 
+#define UNRESTRICTED_DOMID ((domid_t)-1)
+
 static int privcmd_vma_range_is_mapped(
                struct vm_area_struct *vma,
                unsigned long addr,
                unsigned long nr_pages);
 
-static long privcmd_ioctl_hypercall(void __user *udata)
+struct privcmd_check_buf {
+	unsigned copy_back;
+	void __user *copy_ptr;
+	union {
+		struct sched_remote_shutdown remote_shutdown;
+		struct xen_memory_exchange mem_exchange;
+		struct xen_domctl domctl;
+		unsigned char buf[1];
+	} u;
+};
+
+static long privcmd_check_hypercall(struct privcmd_hypercall *hypercall,
+				    struct privcmd_check *check,
+				    domid_t restrict_domid)
+{
+#define DOMID_AT(type, field) do { \
+	BUILD_BUG_ON(sizeof(type) > sizeof(check->u)); \
+	BUILD_BUG_ON(sizeof(((type *) 0)->field) != sizeof(domid_t)); \
+	check->copy_back = sizeof(type); \
+	domid_offset = offsetof(type, field); \
+	} while (0)
+
+/* we copy from userspace and replace arguments to avoid unsafe data */
+#define FETCH_ARG(dest, arg_num, size) do { \
+	check->copy_ptr = (void *) (long) hypercall->arg[arg_num]; \
+	if (copy_from_user(dest, check->copy_ptr, size)) \
+		return -EFAULT; \
+	hypercall->arg[arg_num] = (long) dest; \
+	} while (0)
+
+	unsigned domid_offset;
+
+	/* default to invalid so on cases not handled we fail */
+	domid_t domid = UNRESTRICTED_DOMID;
+
+	switch (hypercall->op) {
+	case __HYPERVISOR_sched_op:
+		if (hypercall->arg[0] == SCHEDOP_remote_shutdown) {
+			FETCH_ARG(&check->u.remote_shutdown, 1,
+				  sizeof(check->u.remote_shutdown));
+			domid = check->u.remote_shutdown.domain_id;
+		}
+		break;
+
+	case __HYPERVISOR_domctl:
+		FETCH_ARG(&check->u.domctl, 0, sizeof(check->u.domctl));
+		check->copy_back = sizeof(check->u.domctl);
+		/* avoid to create a domain */
+		if (check->u.domctl.cmd == XEN_DOMCTL_createdomain)
+			return -EACCES;
+		/* limit versions to avoid possible future bigger buffer */
+		if (check->u.domctl.interface_version > XEN_DOMCTL_INTERFACE_VERSION)
+			return -EACCES;
+		domid = check->u.domctl.domain;
+		break;
+
+	case __HYPERVISOR_memory_op:
+		switch (hypercall->arg[0]) {
+		case XENMEM_increase_reservation:
+		case XENMEM_decrease_reservation:
+		case XENMEM_populate_physmap:
+			DOMID_AT(struct xen_memory_reservation, domid);
+			break;
+		case XENMEM_exchange:
+			DOMID_AT(struct xen_memory_exchange, in.domid);
+			break;
+		case XENMEM_current_reservation:
+		case XENMEM_maximum_reservation:
+		case XENMEM_maximum_gpfn:
+			check->copy_back = sizeof(domid);
+			domid_offset = 0;
+			break;
+		case XENMEM_add_to_physmap:
+			DOMID_AT(struct xen_add_to_physmap, domid);
+			break;
+		case XENMEM_set_memory_map:
+			DOMID_AT(struct xen_foreign_memory_map, domid);
+			break;
+		default:
+			return -EACCES;
+		}
+		FETCH_ARG(&check->u, 1, check->copy_back);
+		domid = *((domid_t *) &check->u.buf[domid_offset]);
+
+		/* extra check for XENMEM_exchange, exchange in the same
+		 * domain */
+		if (hypercall->arg[0] == XENMEM_exchange &&
+		    check->u.mem_exchange.in.domid != check->u.mem_exchange.out.domid)
+			return -EACCES;
+		break;
+
+	case __HYPERVISOR_hvm_op:
+		switch (hypercall->arg[0]) {
+		case HVMOP_set_param:
+		case HVMOP_get_param:
+			DOMID_AT(struct xen_hvm_param, domid);
+			break;
+		case HVMOP_set_pci_intx_level:
+			DOMID_AT(struct xen_hvm_set_pci_intx_level, domid);
+			break;
+		case HVMOP_set_isa_irq_level:
+			DOMID_AT(struct xen_hvm_set_isa_irq_level, domid);
+			break;
+		case HVMOP_set_pci_link_route:
+			DOMID_AT(struct xen_hvm_set_pci_link_route, domid);
+			break;
+		case HVMOP_modified_memory:
+			DOMID_AT(struct xen_hvm_modified_memory, domid);
+			break;
+		case HVMOP_set_mem_type:
+			DOMID_AT(struct xen_hvm_set_mem_type, domid);
+			break;
+		case HVMOP_track_dirty_vram:
+			DOMID_AT(struct xen_hvm_track_dirty_vram, domid);
+			break;
+		default:
+			return -EACCES;
+		}
+		FETCH_ARG(&check->u, 1, check->copy_back);
+		domid = *((domid_t *) &check->u.buf[domid_offset]);
+		break;
+	}
+
+	if (domid != restrict_domid)
+		return -EACCES;
+
+	return 0;
+}
+
+static long privcmd_ioctl_hypercall(void __user *udata,
+				    domid_t restrict_domid)
 {
 	struct privcmd_hypercall hypercall;
+	struct privcmd_check_buf check_buf;
 	long ret;
 
+	check_buf.copy_back = 0;
+	check_buf.copy_ptr = NULL;
+
 	if (copy_from_user(&hypercall, udata, sizeof(hypercall)))
 		return -EFAULT;
 
+	/* we must check domain we are using */
+	if (restrict_domid != UNRESTRICTED_DOMID) {
+		ret = privcmd_check_hypercall(&hypercall, &check_buf,
+					      restrict_domid);
+		if (ret)
+			return ret;
+	}
+
 	ret = privcmd_call(hypercall.op,
 			   hypercall.arg[0], hypercall.arg[1],
 			   hypercall.arg[2], hypercall.arg[3],
 			   hypercall.arg[4]);
 
+	if (check_buf.copy_back && check_buf.copy_ptr && ret >= 0)
+		if (copy_to_user(check_buf.copy_ptr, &check_buf.u,
+				 check_buf.copy_back))
+			ret = -EFAULT;
+
 	return ret;
 }
 
@@ -193,7 +346,7 @@ static int mmap_mfn_range(void *data, void *state)
 	return 0;
 }
 
-static long privcmd_ioctl_mmap(void __user *udata)
+static long privcmd_ioctl_mmap(void __user *udata, domid_t restrict_domid)
 {
 	struct privcmd_mmap mmapcmd;
 	struct mm_struct *mm = current->mm;
@@ -209,6 +362,10 @@ static long privcmd_ioctl_mmap(void __user *udata)
 	if (copy_from_user(&mmapcmd, udata, sizeof(mmapcmd)))
 		return -EFAULT;
 
+	if (restrict_domid != UNRESTRICTED_DOMID &&
+	    restrict_domid != mmapcmd.dom)
+		return -EACCES;
+
 	rc = gather_array(&pagelist,
 			  mmapcmd.num, sizeof(struct privcmd_mmap_entry),
 			  mmapcmd.entry);
@@ -367,7 +524,8 @@ static int alloc_empty_pages(struct vm_area_struct *vma, int numpgs)
 
 static struct vm_operations_struct privcmd_vm_ops;
 
-static long privcmd_ioctl_mmap_batch(void __user *udata, int version)
+static long privcmd_ioctl_mmap_batch(void __user *udata, int version,
+				     domid_t restrict_domid)
 {
 	int ret;
 	struct privcmd_mmapbatch_v2 m;
@@ -397,6 +555,10 @@ static long privcmd_ioctl_mmap_batch(void __user *udata, int version)
 		return -EINVAL;
 	}
 
+	if (restrict_domid != UNRESTRICTED_DOMID &&
+	    restrict_domid != m.dom)
+		return -EACCES;
+
 	nr_pages = m.num;
 	if ((m.num <= 0) || (nr_pages > (LONG_MAX >> PAGE_SHIFT)))
 		return -EINVAL;
@@ -498,27 +660,53 @@ out_unlock:
 	goto out;
 }
 
+static inline domid_t privcmd_get_restrict_domid(const struct file *file)
+{
+	return (domid_t) (long) file->private_data;
+}
+
+static inline void privcmd_set_restrict_domid(struct file *file,
+					      domid_t domid)
+{
+	file->private_data = (void *) (long) domid;
+}
+
 static long privcmd_ioctl(struct file *file,
 			  unsigned int cmd, unsigned long data)
 {
 	int ret = -ENOSYS;
 	void __user *udata = (void __user *) data;
+	domid_t restrict_domid = privcmd_get_restrict_domid(file);
 
 	switch (cmd) {
 	case IOCTL_PRIVCMD_HYPERCALL:
-		ret = privcmd_ioctl_hypercall(udata);
+		ret = privcmd_ioctl_hypercall(udata, restrict_domid);
 		break;
 
 	case IOCTL_PRIVCMD_MMAP:
-		ret = privcmd_ioctl_mmap(udata);
+		ret = privcmd_ioctl_mmap(udata, restrict_domid);
 		break;
 
 	case IOCTL_PRIVCMD_MMAPBATCH:
-		ret = privcmd_ioctl_mmap_batch(udata, 1);
+		ret = privcmd_ioctl_mmap_batch(udata, 1, restrict_domid);
 		break;
 
 	case IOCTL_PRIVCMD_MMAPBATCH_V2:
-		ret = privcmd_ioctl_mmap_batch(udata, 2);
+		ret = privcmd_ioctl_mmap_batch(udata, 2, restrict_domid);
+		break;
+
+	case IOCTL_PRIVCMD_RESTRICT_DOMID: {
+		struct privcmd_restrict_domid prd;
+
+		if (restrict_domid != UNRESTRICTED_DOMID)
+			return -EACCES;
+		if (copy_from_user(&prd, udata, sizeof(prd)))
+			return -EFAULT;
+		if (prd.domid >= DOMID_FIRST_RESERVED)
+			return -EINVAL;
+		privcmd_set_restrict_domid(file, prd.domid);
+		ret = 0;
+		}
 		break;
 
 	default:
@@ -593,10 +781,17 @@ static int privcmd_vma_range_is_mapped(
 				   is_mapped_fn, NULL) != 0;
 }
 
+static int privcmd_open(struct inode *ino, struct file *filp)
+{
+	privcmd_set_restrict_domid(filp, UNRESTRICTED_DOMID);
+	return 0;
+}
+
 const struct file_operations xen_privcmd_fops = {
 	.owner = THIS_MODULE,
 	.unlocked_ioctl = privcmd_ioctl,
 	.mmap = privcmd_mmap,
+	.open = privcmd_open,
 };
 EXPORT_SYMBOL_GPL(xen_privcmd_fops);
 
diff --git a/include/uapi/xen/privcmd.h b/include/uapi/xen/privcmd.h
index a853168..461a999 100644
--- a/include/uapi/xen/privcmd.h
+++ b/include/uapi/xen/privcmd.h
@@ -73,6 +73,10 @@ struct privcmd_mmapbatch_v2 {
 	int __user *err;  /* array of error codes */
 };
 
+struct privcmd_restrict_domid {
+	domid_t domid;
+};
+
 /*
  * @cmd: IOCTL_PRIVCMD_HYPERCALL
  * @arg: &privcmd_hypercall_t
@@ -94,5 +98,7 @@ struct privcmd_mmapbatch_v2 {
 	_IOC(_IOC_NONE, 'P', 3, sizeof(struct privcmd_mmapbatch))
 #define IOCTL_PRIVCMD_MMAPBATCH_V2				\
 	_IOC(_IOC_NONE, 'P', 4, sizeof(struct privcmd_mmapbatch_v2))
+#define IOCTL_PRIVCMD_RESTRICT_DOMID				\
+	_IOC(_IOC_NONE, 'P', 5, sizeof(struct privcmd_restrict_domid))
 
 #endif /* __LINUX_PUBLIC_PRIVCMD_H__ */
diff --git a/include/xen/interface/domctl.h b/include/xen/interface/domctl.h
new file mode 100644
index 0000000..0668fed
--- /dev/null
+++ b/include/xen/interface/domctl.h
@@ -0,0 +1,1090 @@
+/******************************************************************************
+ * domctl.h
+ *
+ * Domain management operations. For use by node control stack.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ *
+ * Copyright (c) 2002-2003, B Dragovic
+ * Copyright (c) 2002-2006, K Fraser
+ */
+
+#ifndef __XEN_PUBLIC_DOMCTL_H__
+#define __XEN_PUBLIC_DOMCTL_H__
+
+#include "xen.h"
+#include "grant_table.h"
+
+#define XEN_DOMCTL_INTERFACE_VERSION 0x0000000a
+
+#if 1
+/*
+ * NB. xen_domctl.domain is an IN/OUT parameter for this operation.
+ * If it is specified as zero, an id is auto-allocated and returned.
+ */
+/* XEN_DOMCTL_createdomain */
+struct xen_domctl_createdomain {
+	/* IN parameters */
+	uint32_t ssidref;
+	xen_domain_handle_t handle;
+ /* Is this an HVM guest (as opposed to a PVH or PV guest)? */
+#define _XEN_DOMCTL_CDF_hvm_guest     0
+#define XEN_DOMCTL_CDF_hvm_guest      (1U<<_XEN_DOMCTL_CDF_hvm_guest)
+ /* Use hardware-assisted paging if available? */
+#define _XEN_DOMCTL_CDF_hap           1
+#define XEN_DOMCTL_CDF_hap            (1U<<_XEN_DOMCTL_CDF_hap)
+ /* Should domain memory integrity be verifed by tboot during Sx? */
+#define _XEN_DOMCTL_CDF_s3_integrity  2
+#define XEN_DOMCTL_CDF_s3_integrity   (1U<<_XEN_DOMCTL_CDF_s3_integrity)
+ /* Disable out-of-sync shadow page tables? */
+#define _XEN_DOMCTL_CDF_oos_off       3
+#define XEN_DOMCTL_CDF_oos_off        (1U<<_XEN_DOMCTL_CDF_oos_off)
+ /* Is this a PVH guest (as opposed to an HVM or PV guest)? */
+#define _XEN_DOMCTL_CDF_pvh_guest     4
+#define XEN_DOMCTL_CDF_pvh_guest      (1U<<_XEN_DOMCTL_CDF_pvh_guest)
+	uint32_t flags;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_createdomain);
+
+/* XEN_DOMCTL_getdomaininfo */
+struct xen_domctl_getdomaininfo {
+	/* OUT variables. */
+	domid_t  domain;              /* Also echoed in domctl.domain */
+ /* Domain is scheduled to die. */
+#define _XEN_DOMINF_dying     0
+#define XEN_DOMINF_dying      (1U<<_XEN_DOMINF_dying)
+ /* Domain is an HVM guest (as opposed to a PV guest). */
+#define _XEN_DOMINF_hvm_guest 1
+#define XEN_DOMINF_hvm_guest  (1U<<_XEN_DOMINF_hvm_guest)
+ /* The guest OS has shut down. */
+#define _XEN_DOMINF_shutdown  2
+#define XEN_DOMINF_shutdown   (1U<<_XEN_DOMINF_shutdown)
+ /* Currently paused by control software. */
+#define _XEN_DOMINF_paused    3
+#define XEN_DOMINF_paused     (1U<<_XEN_DOMINF_paused)
+ /* Currently blocked pending an event.     */
+#define _XEN_DOMINF_blocked   4
+#define XEN_DOMINF_blocked    (1U<<_XEN_DOMINF_blocked)
+ /* Domain is currently running.            */
+#define _XEN_DOMINF_running   5
+#define XEN_DOMINF_running    (1U<<_XEN_DOMINF_running)
+ /* Being debugged.  */
+#define _XEN_DOMINF_debugged  6
+#define XEN_DOMINF_debugged   (1U<<_XEN_DOMINF_debugged)
+/* domain is PVH */
+#define _XEN_DOMINF_pvh_guest 7
+#define XEN_DOMINF_pvh_guest  (1U<<_XEN_DOMINF_pvh_guest)
+ /* XEN_DOMINF_shutdown guest-supplied code.  */
+#define XEN_DOMINF_shutdownmask 255
+#define XEN_DOMINF_shutdownshift 16
+	uint32_t flags;              /* XEN_DOMINF_* */
+	aligned_u64 tot_pages;
+	aligned_u64 max_pages;
+	aligned_u64 outstanding_pages;
+	aligned_u64 shr_pages;
+	aligned_u64 paged_pages;
+	aligned_u64 shared_info_frame; /* GMFN of shared_info struct */
+	aligned_u64 cpu_time;
+	uint32_t nr_online_vcpus;    /* Number of VCPUs currently online. */
+	uint32_t max_vcpu_id;        /* Maximum VCPUID in use by this domain. */
+	uint32_t ssidref;
+	xen_domain_handle_t handle;
+	uint32_t cpupool;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_getdomaininfo);
+
+
+/* XEN_DOMCTL_getmemlist */
+struct xen_domctl_getmemlist {
+	/* IN variables. */
+	/* Max entries to write to output buffer. */
+	aligned_u64 max_pfns;
+	/* Start index in guest's page list. */
+	aligned_u64 start_pfn;
+	GUEST_HANDLE(uint64_t) buffer;
+	/* OUT variables. */
+	aligned_u64 num_pfns;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_getmemlist);
+
+
+/* XEN_DOMCTL_getpageframeinfo */
+
+#define XEN_DOMCTL_PFINFO_LTAB_SHIFT 28
+#define XEN_DOMCTL_PFINFO_NOTAB   (0x0U<<28)
+#define XEN_DOMCTL_PFINFO_L1TAB   (0x1U<<28)
+#define XEN_DOMCTL_PFINFO_L2TAB   (0x2U<<28)
+#define XEN_DOMCTL_PFINFO_L3TAB   (0x3U<<28)
+#define XEN_DOMCTL_PFINFO_L4TAB   (0x4U<<28)
+#define XEN_DOMCTL_PFINFO_LTABTYPE_MASK (0x7U<<28)
+#define XEN_DOMCTL_PFINFO_LPINTAB (0x1U<<31)
+#define XEN_DOMCTL_PFINFO_XTAB    (0xfU<<28) /* invalid page */
+#define XEN_DOMCTL_PFINFO_XALLOC  (0xeU<<28) /* allocate-only page */
+#define XEN_DOMCTL_PFINFO_BROKEN  (0xdU<<28) /* broken page */
+#define XEN_DOMCTL_PFINFO_LTAB_MASK (0xfU<<28)
+
+struct xen_domctl_getpageframeinfo {
+	/* IN variables. */
+	aligned_u64 gmfn; /* GMFN to query */
+	/* OUT variables. */
+	/* Is the page PINNED to a type? */
+	uint32_t type;         /* see above type defs */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_getpageframeinfo);
+
+
+/* XEN_DOMCTL_getpageframeinfo2 */
+struct xen_domctl_getpageframeinfo2 {
+	/* IN variables. */
+	aligned_u64 num;
+	/* IN/OUT variables. */
+	GUEST_HANDLE(uint32_t) array;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_getpageframeinfo2);
+
+/* XEN_DOMCTL_getpageframeinfo3 */
+struct xen_domctl_getpageframeinfo3 {
+	/* IN variables. */
+	aligned_u64 num;
+	/* IN/OUT variables. */
+	GUEST_HANDLE(xen_pfn_t) array;
+};
+
+
+/*
+ * Control shadow pagetables operation
+ */
+/* XEN_DOMCTL_shadow_op */
+
+/* Disable shadow mode. */
+#define XEN_DOMCTL_SHADOW_OP_OFF         0
+
+/* Enable shadow mode (mode contains ORed XEN_DOMCTL_SHADOW_ENABLE_* flags). */
+#define XEN_DOMCTL_SHADOW_OP_ENABLE      32
+
+/* Log-dirty bitmap operations. */
+ /* Return the bitmap and clean internal copy for next round. */
+#define XEN_DOMCTL_SHADOW_OP_CLEAN       11
+ /* Return the bitmap but do not modify internal copy. */
+#define XEN_DOMCTL_SHADOW_OP_PEEK        12
+
+/* Memory allocation accessors. */
+#define XEN_DOMCTL_SHADOW_OP_GET_ALLOCATION   30
+#define XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION   31
+
+/* Legacy enable operations. */
+ /* Equiv. to ENABLE with no mode flags. */
+#define XEN_DOMCTL_SHADOW_OP_ENABLE_TEST       1
+ /* Equiv. to ENABLE with mode flag ENABLE_LOG_DIRTY. */
+#define XEN_DOMCTL_SHADOW_OP_ENABLE_LOGDIRTY   2
+ /* Equiv. to ENABLE with mode flags ENABLE_REFCOUNT and ENABLE_TRANSLATE. */
+#define XEN_DOMCTL_SHADOW_OP_ENABLE_TRANSLATE  3
+
+/* Mode flags for XEN_DOMCTL_SHADOW_OP_ENABLE. */
+ /*
+  * Shadow pagetables are refcounted: guest does not use explicit mmu
+  * operations nor write-protect its pagetables.
+  */
+#define XEN_DOMCTL_SHADOW_ENABLE_REFCOUNT  (1 << 1)
+ /*
+  * Log pages in a bitmap as they are dirtied.
+  * Used for live relocation to determine which pages must be re-sent.
+  */
+#define XEN_DOMCTL_SHADOW_ENABLE_LOG_DIRTY (1 << 2)
+ /*
+  * Automatically translate GPFNs into MFNs.
+  */
+#define XEN_DOMCTL_SHADOW_ENABLE_TRANSLATE (1 << 3)
+ /*
+  * Xen does not steal virtual address space from the guest.
+  * Requires HVM support.
+  */
+#define XEN_DOMCTL_SHADOW_ENABLE_EXTERNAL  (1 << 4)
+
+struct xen_domctl_shadow_op_stats {
+	uint32_t fault_count;
+	uint32_t dirty_count;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_shadow_op_stats);
+
+struct xen_domctl_shadow_op {
+	/* IN variables. */
+	uint32_t       op;       /* XEN_DOMCTL_SHADOW_OP_* */
+
+	/* OP_ENABLE */
+	uint32_t       mode;     /* XEN_DOMCTL_SHADOW_ENABLE_* */
+
+	/* OP_GET_ALLOCATION / OP_SET_ALLOCATION */
+	uint32_t       mb;       /* Shadow memory allocation in MB */
+
+	/* OP_PEEK / OP_CLEAN */
+	GUEST_HANDLE(uchar) dirty_bitmap;
+	aligned_u64 pages; /* Size of buffer. Updated with actual size. */
+	struct xen_domctl_shadow_op_stats stats;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_shadow_op);
+
+
+/* XEN_DOMCTL_max_mem */
+struct xen_domctl_max_mem {
+	/* IN variables. */
+	aligned_u64 max_memkb;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_max_mem);
+
+
+/* XEN_DOMCTL_setvcpucontext */
+/* XEN_DOMCTL_getvcpucontext */
+struct xen_domctl_vcpucontext {
+	uint32_t              vcpu;                  /* IN */
+	GUEST_HANDLE(vcpu_guest_context) ctxt; /* IN/OUT */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_vcpucontext);
+
+
+/* XEN_DOMCTL_getvcpuinfo */
+struct xen_domctl_getvcpuinfo {
+	/* IN variables. */
+	uint32_t vcpu;
+	/* OUT variables. */
+	uint8_t  online;         /* currently online (not hotplugged)? */
+	uint8_t  blocked;        /* blocked waiting for an event? */
+	uint8_t  running;        /* currently scheduled on its CPU? */
+	aligned_u64 cpu_time;    /* total cpu time consumed (ns) */
+	uint32_t cpu;            /* current mapping   */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_getvcpuinfo);
+
+#if 0
+/* Get/set the NUMA node(s) with which the guest has affinity with. */
+/* XEN_DOMCTL_setnodeaffinity */
+/* XEN_DOMCTL_getnodeaffinity */
+struct xen_domctl_nodeaffinity {
+	struct xenctl_bitmap nodemap;/* IN */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_nodeaffinity);
+
+
+/* Get/set which physical cpus a vcpu can execute on. */
+/* XEN_DOMCTL_setvcpuaffinity */
+/* XEN_DOMCTL_getvcpuaffinity */
+struct xen_domctl_vcpuaffinity {
+	uint32_t  vcpu;              /* IN */
+	struct xenctl_bitmap cpumap; /* IN/OUT */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_vcpuaffinity);
+#endif
+
+
+/* XEN_DOMCTL_max_vcpus */
+struct xen_domctl_max_vcpus {
+	uint32_t max;           /* maximum number of vcpus */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_max_vcpus);
+
+
+/* XEN_DOMCTL_scheduler_op */
+/* Scheduler types. */
+#define XEN_SCHEDULER_SEDF     4
+#define XEN_SCHEDULER_CREDIT   5
+#define XEN_SCHEDULER_CREDIT2  6
+#define XEN_SCHEDULER_ARINC653 7
+/* Set or get info? */
+#define XEN_DOMCTL_SCHEDOP_putinfo 0
+#define XEN_DOMCTL_SCHEDOP_getinfo 1
+struct xen_domctl_scheduler_op {
+	uint32_t sched_id;  /* XEN_SCHEDULER_* */
+	uint32_t cmd;       /* XEN_DOMCTL_SCHEDOP_* */
+	union {
+		struct xen_domctl_sched_sedf {
+			aligned_u64 period;
+			aligned_u64 slice;
+			aligned_u64 latency;
+			uint32_t extratime;
+			uint32_t weight;
+		} sedf;
+		struct xen_domctl_sched_credit {
+			uint16_t weight;
+			uint16_t cap;
+		} credit;
+		struct xen_domctl_sched_credit2 {
+			uint16_t weight;
+		} credit2;
+	} u;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_scheduler_op);
+
+
+/* XEN_DOMCTL_setdomainhandle */
+struct xen_domctl_setdomainhandle {
+	xen_domain_handle_t handle;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_setdomainhandle);
+
+
+/* XEN_DOMCTL_setdebugging */
+struct xen_domctl_setdebugging {
+	uint8_t enable;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_setdebugging);
+
+
+/* XEN_DOMCTL_irq_permission */
+struct xen_domctl_irq_permission {
+	uint8_t pirq;
+	uint8_t allow_access; /* flag to specify enable/disable of IRQ access */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_irq_permission);
+
+
+/* XEN_DOMCTL_iomem_permission */
+struct xen_domctl_iomem_permission {
+	aligned_u64 first_mfn;/* first page (physical page number) in range */
+	aligned_u64 nr_mfns;  /* number of pages in range (>0) */
+	uint8_t  allow_access;     /* allow (!0) or deny (0) access to range? */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_iomem_permission);
+
+
+/* XEN_DOMCTL_ioport_permission */
+struct xen_domctl_ioport_permission {
+	uint32_t first_port;              /* first port int range */
+	uint32_t nr_ports;                /* size of port range */
+	uint8_t  allow_access;            /* allow or deny access to range? */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_ioport_permission);
+
+
+/* XEN_DOMCTL_hypercall_init */
+struct xen_domctl_hypercall_init {
+	aligned_u64  gmfn;           /* GMFN to be initialised */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_hypercall_init);
+
+
+/* XEN_DOMCTL_arch_setup */
+#define _XEN_DOMAINSETUP_hvm_guest 0
+#define XEN_DOMAINSETUP_hvm_guest  (1UL<<_XEN_DOMAINSETUP_hvm_guest)
+#define _XEN_DOMAINSETUP_query 1 /* Get parameters (for save)  */
+#define XEN_DOMAINSETUP_query  (1UL<<_XEN_DOMAINSETUP_query)
+#define _XEN_DOMAINSETUP_sioemu_guest 2
+#define XEN_DOMAINSETUP_sioemu_guest  (1UL<<_XEN_DOMAINSETUP_sioemu_guest)
+struct xen_domctl_arch_setup {
+	aligned_u64 flags;  /* XEN_DOMAINSETUP_* */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_arch_setup);
+
+
+/* XEN_DOMCTL_settimeoffset */
+struct xen_domctl_settimeoffset {
+	int32_t  time_offset_seconds; /* applied to domain wallclock time */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_settimeoffset);
+
+/* XEN_DOMCTL_gethvmcontext */
+/* XEN_DOMCTL_sethvmcontext */
+struct xen_domctl_hvmcontext {
+	uint32_t size; /* IN/OUT: size of buffer / bytes filled */
+	GUEST_HANDLE(uchar) buffer; /* IN/OUT: data, or call
+				     * gethvmcontext with NULL
+				     * buffer to get size req'd */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_hvmcontext);
+
+
+/* XEN_DOMCTL_set_address_size */
+/* XEN_DOMCTL_get_address_size */
+struct xen_domctl_address_size {
+	uint32_t size;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_address_size);
+
+
+/* XEN_DOMCTL_real_mode_area */
+struct xen_domctl_real_mode_area {
+	uint32_t log; /* log2 of Real Mode Area size */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_real_mode_area);
+
+
+/* XEN_DOMCTL_sendtrigger */
+#define XEN_DOMCTL_SENDTRIGGER_NMI    0
+#define XEN_DOMCTL_SENDTRIGGER_RESET  1
+#define XEN_DOMCTL_SENDTRIGGER_INIT   2
+#define XEN_DOMCTL_SENDTRIGGER_POWER  3
+#define XEN_DOMCTL_SENDTRIGGER_SLEEP  4
+struct xen_domctl_sendtrigger {
+	uint32_t  trigger;  /* IN */
+	uint32_t  vcpu;     /* IN */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_sendtrigger);
+
+
+/* Assign PCI device to HVM guest. Sets up IOMMU structures. */
+/* XEN_DOMCTL_assign_device */
+/* XEN_DOMCTL_test_assign_device */
+/* XEN_DOMCTL_deassign_device */
+struct xen_domctl_assign_device {
+	uint32_t  machine_sbdf;   /* machine PCI ID of assigned device */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_assign_device);
+
+/* Retrieve sibling devices infomation of machine_sbdf */
+/* XEN_DOMCTL_get_device_group */
+struct xen_domctl_get_device_group {
+	uint32_t  machine_sbdf;     /* IN */
+	uint32_t  max_sdevs;        /* IN */
+	uint32_t  num_sdevs;        /* OUT */
+	GUEST_HANDLE(uint32_t)  sdev_array;   /* OUT */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_get_device_group);
+
+/* Pass-through interrupts: bind real irq -> hvm devfn. */
+/* XEN_DOMCTL_bind_pt_irq */
+/* XEN_DOMCTL_unbind_pt_irq */
+enum pt_irq_type_e {
+	PT_IRQ_TYPE_PCI,
+	PT_IRQ_TYPE_ISA,
+	PT_IRQ_TYPE_MSI,
+	PT_IRQ_TYPE_MSI_TRANSLATE,
+};
+struct xen_domctl_bind_pt_irq {
+	uint32_t machine_irq;
+	enum pt_irq_type_e irq_type;
+	uint32_t hvm_domid;
+
+	union {
+		struct {
+			uint8_t isa_irq;
+		} isa;
+		struct {
+			uint8_t bus;
+			uint8_t device;
+			uint8_t intx;
+		} pci;
+		struct {
+			uint8_t gvec;
+			uint32_t gflags;
+			aligned_u64 gtable;
+		} msi;
+	} u;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_bind_pt_irq);
+
+
+/* Bind machine I/O address range -> HVM address range. */
+/* XEN_DOMCTL_memory_mapping */
+#define DPCI_ADD_MAPPING         1
+#define DPCI_REMOVE_MAPPING      0
+struct xen_domctl_memory_mapping {
+	aligned_u64 first_gfn; /* first page (hvm guest phys page) in range */
+	aligned_u64 first_mfn; /* first page (machine page) in range */
+	aligned_u64 nr_mfns;   /* number of pages in range (>0) */
+	uint32_t add_mapping;       /* add or remove mapping */
+	uint32_t padding;           /* padding for 64-bit aligned structure */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_memory_mapping);
+
+
+/* Bind machine I/O port range -> HVM I/O port range. */
+/* XEN_DOMCTL_ioport_mapping */
+struct xen_domctl_ioport_mapping {
+	uint32_t first_gport;     /* first guest IO port*/
+	uint32_t first_mport;     /* first machine IO port */
+	uint32_t nr_ports;        /* size of port range */
+	uint32_t add_mapping;     /* add or remove mapping */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_ioport_mapping);
+
+
+/*
+ * Pin caching type of RAM space for x86 HVM domU.
+ */
+/* XEN_DOMCTL_pin_mem_cacheattr */
+/* Caching types: these happen to be the same as x86 MTRR/PAT type codes. */
+#define XEN_DOMCTL_MEM_CACHEATTR_UC  0
+#define XEN_DOMCTL_MEM_CACHEATTR_WC  1
+#define XEN_DOMCTL_MEM_CACHEATTR_WT  4
+#define XEN_DOMCTL_MEM_CACHEATTR_WP  5
+#define XEN_DOMCTL_MEM_CACHEATTR_WB  6
+#define XEN_DOMCTL_MEM_CACHEATTR_UCM 7
+struct xen_domctl_pin_mem_cacheattr {
+	aligned_u64 start, end;
+	uint32_t type; /* XEN_DOMCTL_MEM_CACHEATTR_* */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_pin_mem_cacheattr);
+
+
+#if 0
+/* XEN_DOMCTL_set_ext_vcpucontext */
+/* XEN_DOMCTL_get_ext_vcpucontext */
+struct xen_domctl_ext_vcpucontext {
+	/* IN: VCPU that this call applies to. */
+	uint32_t         vcpu;
+	/*
+	 * SET: Size of struct (IN)
+	 * GET: Size of struct (OUT, up to 128 bytes)
+	 */
+	uint32_t         size;
+#if defined(__i386__) || defined(__x86_64__)
+	/* SYSCALL from 32-bit mode and SYSENTER callback information. */
+	/* NB. SYSCALL from 64-bit mode is contained in vcpu_guest_context_t */
+	aligned_u64 syscall32_callback_eip;
+	aligned_u64 sysenter_callback_eip;
+	uint16_t         syscall32_callback_cs;
+	uint16_t         sysenter_callback_cs;
+	uint8_t          syscall32_disables_events;
+	uint8_t          sysenter_disables_events;
+#if defined(__GNUC__)
+	union {
+		aligned_u64 mcg_cap;
+		struct hvm_vmce_vcpu vmce;
+	};
+#else
+	struct hvm_vmce_vcpu vmce;
+#endif
+#endif
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_ext_vcpucontext);
+#endif
+
+/*
+ * Set the target domain for a domain
+ */
+/* XEN_DOMCTL_set_target */
+struct xen_domctl_set_target {
+	domid_t target;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_set_target);
+
+#if defined(__i386__) || defined(__x86_64__)
+# define XEN_CPUID_INPUT_UNUSED  0xFFFFFFFF
+/* XEN_DOMCTL_set_cpuid */
+struct xen_domctl_cpuid {
+	uint32_t input[2];
+	uint32_t eax;
+	uint32_t ebx;
+	uint32_t ecx;
+	uint32_t edx;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_cpuid);
+#endif
+
+/*
+ * Arranges that if the domain suspends (specifically, if it shuts
+ * down with code SHUTDOWN_suspend), this event channel will be
+ * notified.
+ *
+ * This is _instead of_ the usual notification to the global
+ * VIRQ_DOM_EXC.  (In most systems that pirq is owned by xenstored.)
+ *
+ * Only one subscription per domain is possible.  Last subscriber
+ * wins; others are silently displaced.
+ *
+ * NB that contrary to the rather general name, it only applies to
+ * domain shutdown with code suspend.  Shutdown for other reasons
+ * (including crash), and domain death, are notified to VIRQ_DOM_EXC
+ * regardless.
+ */
+/* XEN_DOMCTL_subscribe */
+struct xen_domctl_subscribe {
+	uint32_t port; /* IN */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_subscribe);
+
+/*
+ * Define the maximum machine address size which should be allocated
+ * to a guest.
+ */
+/* XEN_DOMCTL_set_machine_address_size */
+/* XEN_DOMCTL_get_machine_address_size */
+
+/*
+ * Do not inject spurious page faults into this domain.
+ */
+/* XEN_DOMCTL_suppress_spurious_page_faults */
+
+/* XEN_DOMCTL_debug_op */
+#define XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_OFF         0
+#define XEN_DOMCTL_DEBUG_OP_SINGLE_STEP_ON          1
+struct xen_domctl_debug_op {
+	uint32_t op;   /* IN */
+	uint32_t vcpu; /* IN */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_debug_op);
+
+/*
+ * Request a particular record from the HVM context
+ */
+/* XEN_DOMCTL_gethvmcontext_partial */
+struct xen_domctl_hvmcontext_partial {
+	uint32_t type;                      /* IN: Type of record required */
+	uint32_t instance;                  /* IN: Instance of that type */
+	GUEST_HANDLE(uchar) buffer;  /* OUT: buffer to write record into */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_hvmcontext_partial);
+
+/* XEN_DOMCTL_disable_migrate */
+struct xen_domctl_disable_migrate {
+	uint32_t disable; /* IN: 1: disable migration and restore */
+};
+
+
+/* XEN_DOMCTL_gettscinfo */
+/* XEN_DOMCTL_settscinfo */
+struct xen_guest_tsc_info {
+	uint32_t tsc_mode;
+	uint32_t gtsc_khz;
+	uint32_t incarnation;
+	uint32_t pad;
+	aligned_u64 elapsed_nsec;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_guest_tsc_info);
+
+struct xen_domctl_tsc_info {
+	GUEST_HANDLE(xen_guest_tsc_info) out_info; /* OUT */
+	struct xen_guest_tsc_info info; /* IN */
+};
+
+/* XEN_DOMCTL_gdbsx_guestmemio      guest mem io */
+struct xen_domctl_gdbsx_memio {
+	/* IN */
+	aligned_u64 pgd3val;/* optional: init_mm.pgd[3] value */
+	aligned_u64 gva;    /* guest virtual address */
+	aligned_u64 uva;    /* user buffer virtual address */
+	uint32_t         len;    /* number of bytes to read/write */
+	uint8_t          gwr;    /* 0 = read from guest. 1 = write to guest */
+	/* OUT */
+	uint32_t         remain; /* bytes remaining to be copied */
+};
+
+/* XEN_DOMCTL_gdbsx_pausevcpu */
+/* XEN_DOMCTL_gdbsx_unpausevcpu */
+struct xen_domctl_gdbsx_pauseunp_vcpu { /* pause/unpause a vcpu */
+	uint32_t         vcpu;         /* which vcpu */
+};
+
+/* XEN_DOMCTL_gdbsx_domstatus */
+struct xen_domctl_gdbsx_domstatus {
+	/* OUT */
+	uint8_t          paused;     /* is the domain paused */
+	uint32_t         vcpu_id;    /* any vcpu in an event? */
+	uint32_t         vcpu_ev;    /* if yes, what event? */
+};
+
+/*
+ * Memory event operations
+ */
+
+/* XEN_DOMCTL_mem_event_op */
+
+/*
+ * Domain memory paging
+ * Page memory in and out.
+ * Domctl interface to set up and tear down the
+ * pager<->hypervisor interface. Use XENMEM_paging_op*
+ * to perform per-page operations.
+ *
+ * The XEN_DOMCTL_MEM_EVENT_OP_PAGING_ENABLE domctl returns several
+ * non-standard error codes to indicate why paging could not be enabled:
+ * ENODEV - host lacks HAP support (EPT/NPT) or HAP is disabled in guest
+ * EMLINK - guest has iommu passthrough enabled
+ * EXDEV  - guest has PoD enabled
+ * EBUSY  - guest has or had paging enabled, ring buffer still active
+ */
+#define XEN_DOMCTL_MEM_EVENT_OP_PAGING            1
+
+#define XEN_DOMCTL_MEM_EVENT_OP_PAGING_ENABLE     0
+#define XEN_DOMCTL_MEM_EVENT_OP_PAGING_DISABLE    1
+
+/*
+ * Access permissions.
+ *
+ * As with paging, use the domctl for teardown/setup of the
+ * helper<->hypervisor interface.
+ *
+ * There are HVM hypercalls to set the per-page access permissions of every
+ * page in a domain.  When one of these permissions--independent, read,
+ * write, and execute--is violated, the VCPU is paused and a memory event
+ * is sent with what happened.  (See public/mem_event.h) .
+ *
+ * The memory event handler can then resume the VCPU and redo the access
+ * with a XENMEM_access_op_resume hypercall.
+ *
+ * The XEN_DOMCTL_MEM_EVENT_OP_ACCESS_ENABLE domctl returns several
+ * non-standard error codes to indicate why access could not be enabled:
+ * ENODEV - host lacks HAP support (EPT/NPT) or HAP is disabled in guest
+ * EBUSY  - guest has or had access enabled, ring buffer still active
+ */
+#define XEN_DOMCTL_MEM_EVENT_OP_ACCESS            2
+
+#define XEN_DOMCTL_MEM_EVENT_OP_ACCESS_ENABLE     0
+#define XEN_DOMCTL_MEM_EVENT_OP_ACCESS_DISABLE    1
+
+/*
+ * Sharing ENOMEM helper.
+ *
+ * As with paging, use the domctl for teardown/setup of the
+ * helper<->hypervisor interface.
+ *
+ * If setup, this ring is used to communicate failed allocations
+ * in the unshare path. XENMEM_sharing_op_resume is used to wake up
+ * vcpus that could not unshare.
+ *
+ * Note that shring can be turned on (as per the domctl below)
+ * *without* this ring being setup.
+ */
+#define XEN_DOMCTL_MEM_EVENT_OP_SHARING           3
+
+#define XEN_DOMCTL_MEM_EVENT_OP_SHARING_ENABLE    0
+#define XEN_DOMCTL_MEM_EVENT_OP_SHARING_DISABLE   1
+
+/* Use for teardown/setup of helper<->hypervisor interface for paging,
+ * access and sharing.*/
+struct xen_domctl_mem_event_op {
+	uint32_t       op;           /* XEN_DOMCTL_MEM_EVENT_OP_*_* */
+	uint32_t       mode;         /* XEN_DOMCTL_MEM_EVENT_OP_* */
+
+	uint32_t port;              /* OUT: event channel for ring */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_mem_event_op);
+
+/*
+ * Memory sharing operations
+ */
+/* XEN_DOMCTL_mem_sharing_op.
+ * The CONTROL sub-domctl is used for bringup/teardown. */
+#define XEN_DOMCTL_MEM_SHARING_CONTROL          0
+
+struct xen_domctl_mem_sharing_op {
+	uint8_t op; /* XEN_DOMCTL_MEM_SHARING_* */
+
+	union {
+		uint8_t enable;                   /* CONTROL */
+	} u;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_mem_sharing_op);
+
+struct xen_domctl_audit_p2m {
+	/* OUT error counts */
+	uint64_t orphans;
+	uint64_t m2p_bad;
+	uint64_t p2m_bad;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_audit_p2m);
+
+struct xen_domctl_set_virq_handler {
+	uint32_t virq; /* IN */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_set_virq_handler);
+
+#if defined(__i386__) || defined(__x86_64__)
+/* XEN_DOMCTL_setvcpuextstate */
+/* XEN_DOMCTL_getvcpuextstate */
+struct xen_domctl_vcpuextstate {
+	/* IN: VCPU that this call applies to. */
+	uint32_t         vcpu;
+	/*
+	 * SET: Ignored.
+	 * GET: xfeature support mask of struct (IN/OUT)
+	 * xfeature mask is served as identifications of the saving format
+	 * so that compatible CPUs can have a check on format to decide
+	 * whether it can restore.
+	 */
+	aligned_u64         xfeature_mask;
+	/*
+	 * SET: Size of struct (IN)
+	 * GET: Size of struct (IN/OUT)
+	 */
+	aligned_u64         size;
+	GUEST_HANDLE(uint64_t) buffer;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_vcpuextstate);
+#endif
+
+/* XEN_DOMCTL_set_access_required: sets whether a memory event listener
+ * must be present to handle page access events: if false, the page
+ * access will revert to full permissions if no one is listening;
+ *  */
+struct xen_domctl_set_access_required {
+	uint8_t access_required;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_set_access_required);
+
+struct xen_domctl_set_broken_page_p2m {
+	aligned_u64 pfn;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_set_broken_page_p2m);
+
+/*
+ * XEN_DOMCTL_set_max_evtchn: sets the maximum event channel port
+ * number the guest may use.  Use this limit the amount of resources
+ * (global mapping space, xenheap) a guest may use for event channels.
+ */
+struct xen_domctl_set_max_evtchn {
+	uint32_t max_port;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_set_max_evtchn);
+
+/*
+ * ARM: Clean and invalidate caches associated with given region of
+ * guest memory.
+ */
+struct xen_domctl_cacheflush {
+	/* IN: page range to flush. */
+	xen_pfn_t start_pfn, nr_pfns;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_cacheflush);
+
+#if defined(__i386__) || defined(__x86_64__)
+struct xen_domctl_vcpu_msr {
+	uint32_t         index;
+	uint32_t         reserved;
+	aligned_u64 value;
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_vcpu_msr);
+
+/*
+ * XEN_DOMCTL_set_vcpu_msrs / XEN_DOMCTL_get_vcpu_msrs.
+ *
+ * Input:
+ * - A NULL 'msrs' guest handle is a request for the maximum 'msr_count'.
+ * - Otherwise, 'msr_count' is the number of entries in 'msrs'.
+ *
+ * Output for get:
+ * - If 'msr_count' is less than the number Xen needs to write, -ENOBUFS shall
+ *   be returned and 'msr_count' updated to reflect the intended number.
+ * - On success, 'msr_count' shall indicate the number of MSRs written, which
+ *   may be less than the maximum if some are not currently used by the vcpu.
+ *
+ * Output for set:
+ * - If Xen encounters an error with a specific MSR, -EINVAL shall be returned
+ *   and 'msr_count' shall be set to the offending index, to aid debugging.
+ */
+struct xen_domctl_vcpu_msrs {
+	uint32_t vcpu;                                   /* IN     */
+	uint32_t msr_count;                              /* IN/OUT */
+	GUEST_HANDLE(xen_domctl_vcpu_msr) msrs; /* IN/OUT */
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_vcpu_msrs);
+#endif
+
+/*
+ * Return information about the state and running time of a domain.
+ * The "domain runstate" is based on the runstates of all the vcpus of the
+ * domain (see below).
+ * @extra_arg == pointer to domain_runstate_info structure.
+ */
+struct xen_domctl_runstate_info {
+	/* VCPU's current state (RUNSTATE_*). */
+	uint32_t      state;
+	uint32_t missed_changes;
+	/* Number of times we missed an update due to contention */
+	/* When was current state entered (system time, ns)? */
+	uint64_t state_entry_time;
+	/*
+	 * Time spent in each RUNSTATE_* (ns). The sum of these times is
+	 * NOT guaranteed not to drift from system time.
+	 */
+	uint64_t time[6];
+};
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_runstate_info);
+
+/* All vcpus are running */
+#define DOMAIN_RUNSTATE_full_run           0
+
+/* All vcpus are runnable (i.e., waiting for cpu) */
+#define DOMAIN_RUNSTATE_full_contention    1
+
+/* Some vcpus are running, some are runnable */
+#define DOMAIN_RUNSTATE_concurrency_hazard 2
+
+/* All vcpus are blocked / offline */
+#define DOMAIN_RUNSTATE_blocked            3
+
+/* Some vpcus are running, some are blocked */
+#define DOMAIN_RUNSTATE_partial_run        4
+
+/* Some vcpus are runnable, some are blocked */
+#define DOMAIN_RUNSTATE_partial_contention 5
+
+struct xen_domctl_corespersocket {
+	uint32_t cores_per_socket;
+};
+
+DEFINE_GUEST_HANDLE_STRUCT(xen_domctl_corespersocket);
+#endif
+
+struct xen_domctl {
+	uint32_t cmd;
+#define XEN_DOMCTL_createdomain                   1
+#define XEN_DOMCTL_destroydomain                  2
+#define XEN_DOMCTL_pausedomain                    3
+#define XEN_DOMCTL_unpausedomain                  4
+#define XEN_DOMCTL_getdomaininfo                  5
+#define XEN_DOMCTL_getmemlist                     6
+#define XEN_DOMCTL_getpageframeinfo               7
+#define XEN_DOMCTL_getpageframeinfo2              8
+#define XEN_DOMCTL_setvcpuaffinity                9
+#define XEN_DOMCTL_shadow_op                     10
+#define XEN_DOMCTL_max_mem                       11
+#define XEN_DOMCTL_setvcpucontext                12
+#define XEN_DOMCTL_getvcpucontext                13
+#define XEN_DOMCTL_getvcpuinfo                   14
+#define XEN_DOMCTL_max_vcpus                     15
+#define XEN_DOMCTL_scheduler_op                  16
+#define XEN_DOMCTL_setdomainhandle               17
+#define XEN_DOMCTL_setdebugging                  18
+#define XEN_DOMCTL_irq_permission                19
+#define XEN_DOMCTL_iomem_permission              20
+#define XEN_DOMCTL_ioport_permission             21
+#define XEN_DOMCTL_hypercall_init                22
+#define XEN_DOMCTL_arch_setup                    23
+#define XEN_DOMCTL_settimeoffset                 24
+#define XEN_DOMCTL_getvcpuaffinity               25
+#define XEN_DOMCTL_real_mode_area                26
+#define XEN_DOMCTL_resumedomain                  27
+#define XEN_DOMCTL_sendtrigger                   28
+#define XEN_DOMCTL_subscribe                     29
+#define XEN_DOMCTL_gethvmcontext                 33
+#define XEN_DOMCTL_sethvmcontext                 34
+#define XEN_DOMCTL_set_address_size              35
+#define XEN_DOMCTL_get_address_size              36
+#define XEN_DOMCTL_assign_device                 37
+#define XEN_DOMCTL_bind_pt_irq                   38
+#define XEN_DOMCTL_memory_mapping                39
+#define XEN_DOMCTL_ioport_mapping                40
+#define XEN_DOMCTL_pin_mem_cacheattr             41
+#define XEN_DOMCTL_set_ext_vcpucontext           42
+#define XEN_DOMCTL_get_ext_vcpucontext           43
+#define XEN_DOMCTL_set_opt_feature               44 /* Obsolete IA64 only */
+#define XEN_DOMCTL_test_assign_device            45
+#define XEN_DOMCTL_set_target                    46
+#define XEN_DOMCTL_deassign_device               47
+#define XEN_DOMCTL_unbind_pt_irq                 48
+#define XEN_DOMCTL_set_cpuid                     49
+#define XEN_DOMCTL_get_device_group              50
+#define XEN_DOMCTL_set_machine_address_size      51
+#define XEN_DOMCTL_get_machine_address_size      52
+#define XEN_DOMCTL_suppress_spurious_page_faults 53
+#define XEN_DOMCTL_debug_op                      54
+#define XEN_DOMCTL_gethvmcontext_partial         55
+#define XEN_DOMCTL_mem_event_op                  56
+#define XEN_DOMCTL_mem_sharing_op                57
+#define XEN_DOMCTL_disable_migrate               58
+#define XEN_DOMCTL_gettscinfo                    59
+#define XEN_DOMCTL_settscinfo                    60
+#define XEN_DOMCTL_getpageframeinfo3             61
+#define XEN_DOMCTL_setvcpuextstate               62
+#define XEN_DOMCTL_getvcpuextstate               63
+#define XEN_DOMCTL_set_access_required           64
+#define XEN_DOMCTL_audit_p2m                     65
+#define XEN_DOMCTL_set_virq_handler              66
+#define XEN_DOMCTL_set_broken_page_p2m           67
+#define XEN_DOMCTL_setnodeaffinity               68
+#define XEN_DOMCTL_getnodeaffinity               69
+#define XEN_DOMCTL_set_max_evtchn                70
+#define XEN_DOMCTL_cacheflush                    71
+#define XEN_DOMCTL_get_vcpu_msrs                 72
+#define XEN_DOMCTL_set_vcpu_msrs                 73
+#define XEN_DOMCTL_get_runstate_info             98
+#define XEN_DOMCTL_gdbsx_guestmemio            1000
+#define XEN_DOMCTL_gdbsx_pausevcpu             1001
+#define XEN_DOMCTL_gdbsx_unpausevcpu           1002
+#define XEN_DOMCTL_gdbsx_domstatus             1003
+#define XEN_DOMCTL_setcorespersocket           4001
+	uint32_t interface_version; /* XEN_DOMCTL_INTERFACE_VERSION */
+	domid_t  domain;
+	union {
+		struct xen_domctl_createdomain      createdomain;
+		struct xen_domctl_getdomaininfo     getdomaininfo;
+		struct xen_domctl_getmemlist        getmemlist;
+		struct xen_domctl_getpageframeinfo  getpageframeinfo;
+		struct xen_domctl_getpageframeinfo2 getpageframeinfo2;
+		struct xen_domctl_getpageframeinfo3 getpageframeinfo3;
+#if 0
+		struct xen_domctl_nodeaffinity      nodeaffinity;
+		struct xen_domctl_vcpuaffinity      vcpuaffinity;
+#endif
+		struct xen_domctl_shadow_op         shadow_op;
+		struct xen_domctl_max_mem           max_mem;
+		struct xen_domctl_vcpucontext       vcpucontext;
+		struct xen_domctl_getvcpuinfo       getvcpuinfo;
+		struct xen_domctl_max_vcpus         max_vcpus;
+		struct xen_domctl_scheduler_op      scheduler_op;
+		struct xen_domctl_setdomainhandle   setdomainhandle;
+		struct xen_domctl_setdebugging      setdebugging;
+		struct xen_domctl_irq_permission    irq_permission;
+		struct xen_domctl_iomem_permission  iomem_permission;
+		struct xen_domctl_ioport_permission ioport_permission;
+		struct xen_domctl_hypercall_init    hypercall_init;
+		struct xen_domctl_arch_setup        arch_setup;
+		struct xen_domctl_settimeoffset     settimeoffset;
+		struct xen_domctl_disable_migrate   disable_migrate;
+		struct xen_domctl_tsc_info          tsc_info;
+		struct xen_domctl_real_mode_area    real_mode_area;
+		struct xen_domctl_hvmcontext        hvmcontext;
+		struct xen_domctl_hvmcontext_partial hvmcontext_partial;
+		struct xen_domctl_address_size      address_size;
+		struct xen_domctl_sendtrigger       sendtrigger;
+		struct xen_domctl_get_device_group  get_device_group;
+		struct xen_domctl_assign_device     assign_device;
+		struct xen_domctl_bind_pt_irq       bind_pt_irq;
+		struct xen_domctl_memory_mapping    memory_mapping;
+		struct xen_domctl_ioport_mapping    ioport_mapping;
+		struct xen_domctl_pin_mem_cacheattr pin_mem_cacheattr;
+#if 0
+		struct xen_domctl_ext_vcpucontext   ext_vcpucontext;
+#endif
+		struct xen_domctl_set_target        set_target;
+		struct xen_domctl_subscribe         subscribe;
+		struct xen_domctl_debug_op          debug_op;
+		struct xen_domctl_mem_event_op      mem_event_op;
+		struct xen_domctl_mem_sharing_op    mem_sharing_op;
+#if defined(__i386__) || defined(__x86_64__)
+		struct xen_domctl_cpuid             cpuid;
+		struct xen_domctl_vcpuextstate      vcpuextstate;
+		struct xen_domctl_vcpu_msrs         vcpu_msrs;
+#endif
+		struct xen_domctl_set_access_required access_required;
+		struct xen_domctl_audit_p2m         audit_p2m;
+		struct xen_domctl_set_virq_handler  set_virq_handler;
+		struct xen_domctl_set_max_evtchn    set_max_evtchn;
+		struct xen_domctl_runstate_info     domain_runstate;
+		struct xen_domctl_corespersocket    corespersocket;
+		struct xen_domctl_gdbsx_memio       gdbsx_guest_memio;
+		struct xen_domctl_set_broken_page_p2m set_broken_page_p2m;
+		struct xen_domctl_cacheflush        cacheflush;
+		struct xen_domctl_gdbsx_pauseunp_vcpu gdbsx_pauseunp_vcpu;
+		struct xen_domctl_gdbsx_domstatus   gdbsx_domstatus;
+		uint8_t                             pad[128];
+	} u __aligned(8);
+};
+
+#endif /* __XEN_PUBLIC_DOMCTL_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/include/xen/interface/hvm/hvm_op.h b/include/xen/interface/hvm/hvm_op.h
index 956a046..5fb5260 100644
--- a/include/xen/interface/hvm/hvm_op.h
+++ b/include/xen/interface/hvm/hvm_op.h
@@ -32,6 +32,72 @@ struct xen_hvm_param {
 };
 DEFINE_GUEST_HANDLE_STRUCT(xen_hvm_param);
 
+#define HVMOP_set_pci_intx_level  2
+struct xen_hvm_set_pci_intx_level {
+	/* Domain to be updated. */
+	domid_t  domid;
+	/* PCI INTx identification in PCI topology (domain:bus:device:intx). */
+	uint8_t  domain, bus, device, intx;
+	/* Assertion level (0 = unasserted, 1 = asserted). */
+	uint8_t  level;
+};
+
+#define HVMOP_set_isa_irq_level   3
+struct xen_hvm_set_isa_irq_level {
+	/* Domain to be updated. */
+	domid_t  domid;
+	/* ISA device identification, by ISA IRQ (0-15). */
+	uint8_t  isa_irq;
+	/* Assertion level (0 = unasserted, 1 = asserted). */
+	uint8_t  level;
+};
+
+#define HVMOP_set_pci_link_route  4
+struct xen_hvm_set_pci_link_route {
+	/* Domain to be updated. */
+	domid_t  domid;
+	/* PCI link identifier (0-3). */
+	uint8_t  link;
+	/* ISA IRQ (1-15), or 0 (disable link). */
+	uint8_t  isa_irq;
+};
+
+#define HVMOP_track_dirty_vram    6
+struct xen_hvm_track_dirty_vram {
+	/* Domain to be tracked. */
+	domid_t  domid;
+	/* First pfn to track. */
+	aligned_u64 first_pfn;
+	/* Number of pages to track. */
+	aligned_u64 nr;
+	/* OUT variable. */
+	/* Dirty bitmap buffer. */
+	aligned_u64 dirty_bitmap;
+};
+
+#define HVMOP_modified_memory    7
+struct xen_hvm_modified_memory {
+	/* Domain to be updated. */
+	domid_t  domid;
+	/* First pfn. */
+	aligned_u64 first_pfn;
+	/* Number of pages. */
+	aligned_u64 nr;
+};
+
+#define HVMOP_set_mem_type    8
+/* Notify that a region of memory is to be treated in a specific way. */
+struct xen_hvm_set_mem_type {
+	/* Domain to be updated. */
+	domid_t domid;
+	/* Memory type */
+	uint16_t hvmmem_type;
+	/* Number of pages. */
+	uint32_t nr;
+	/* First pfn. */
+	aligned_u64 first_pfn;
+};
+
 /* Hint from PV drivers for pagetable destruction. */
 #define HVMOP_pagetable_dying       9
 struct xen_hvm_pagetable_dying {
diff --git a/include/xen/interface/memory.h b/include/xen/interface/memory.h
index 2ecfe4f..a5fd2e6 100644
--- a/include/xen/interface/memory.h
+++ b/include/xen/interface/memory.h
@@ -248,6 +248,14 @@ DEFINE_GUEST_HANDLE_STRUCT(xen_memory_map);
  */
 extern spinlock_t xen_reservation_lock;
 
+#define XENMEM_set_memory_map       13
+struct xen_foreign_memory_map {
+	domid_t domid;
+	struct xen_memory_map map;
+};
+
+#define XENMEM_maximum_gpfn         14
+
 /*
  * Unmaps the page appearing at a particular GPFN from the specified guest's
  * pseudophysical address space.
diff --git a/include/xen/interface/xen.h b/include/xen/interface/xen.h
index de08213..075cb6f 100644
--- a/include/xen/interface/xen.h
+++ b/include/xen/interface/xen.h
@@ -57,6 +57,7 @@
 #define __HYPERVISOR_event_channel_op     32
 #define __HYPERVISOR_physdev_op           33
 #define __HYPERVISOR_hvm_op               34
+#define __HYPERVISOR_domctl               36
 #define __HYPERVISOR_tmem_op              38
 
 /* Architecture-specific hypercall definitions. */
-- 
1.9.1

^ permalink raw reply related

* Re: [PATCH -v4] random: introduce getrandom(2) system call
From: Bernd Petrovitsch @ 2014-07-31  8:06 UTC (permalink / raw)
  To: Pavel Machek
  Cc: Bob Beck, Theodore Ts'o, linux-kernel, linux-api,
	linux-crypto, Theo de Raadt
In-Reply-To: <20140730221819.GB18189@amd.pavel.ucw.cz>

On Don, 2014-07-31 at 00:18 +0200, Pavel Machek wrote:
> On Wed 2014-07-30 16:40:52, Bernd Petrovitsch wrote:
> > On Mit, 2014-07-30 at 07:56 -0600, Bob Beck wrote:
> > > Pavel. I have bit 'ol enterprise daemon running with established file
> > > descriptors serving thousands of connections
> > > which periodically require entropy.  Now I run out of descriptors. I
> > > can't establish new connections. but I should
> > > now halt all the other ones that require entropy?  I should raise
> > > SIGKILL on my process serving these thousands
> > > of connetions?  I don't think so.
> > 
> > If that long-running daemon periodically needs something from a device,
> > one would better keep the fd for that open the whole time. Saves some
> > CPU cycles and latency too BTW.
> 
> Agreed.
> 
> On the other hand, keeping a fd open is quite tricky for a
> library. But better solution might be to make that easier.

Yes, in a (full-fledged, standalone) library seems at least tricky (also
referring to some off-list mails here: think about fork() - which could
be inside system() or popen() or similar). 

But as part of the *application* (where one has control over fork()
etc.), this should be somewhat less risky. Yes, that doesn't really help
libssl;-)

Hehe, we (Unix!) have (had) gettimeofday(), time() and similar sys-calls
since ages and no one proposed to make devices for them and get rid of
the system-calls.

> open( , O_IM_A_LIBRARY_GIVE_ME_ONE_OF_THREE_RESERVED_FDS) might be one
> solution. Actually, one reserved fd should be enough.

Well, this can also be DoSed and the proposal aims to make that
impossible (and where does this reserved count against? process-limits,
kernel-wide limit?).

	Bernd
-- 
"I dislike type abstraction if it has no real reason. And saving
on typing is not a good reason - if your typing speed is the main
issue when you're coding, you're doing something seriously wrong."
    - Linus Torvalds

^ permalink raw reply

* Re: [RFC PATCH 1/2] fs: Add dirreadahead syscall and VFS hooks
From: Dave Chinner @ 2014-07-31  3:31 UTC (permalink / raw)
  To: Michael Kerrisk
  Cc: Abhi Das, Linux Kernel, Linux-Fsdevel, cluster-devel, Linux API
In-Reply-To: <CAHO5Pa2fW6mZRTao3uEx2p_X9GvO1btrbb9Bg2ns94+p4biKAQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>

On Tue, Jul 29, 2014 at 10:21:50AM +0200, Michael Kerrisk wrote:
> [CC+=linux-api]
> 
> On Fri, Jul 25, 2014 at 7:37 PM, Abhi Das <adas-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> > Also adds a void *opaque field to struct dir_context that can be
> > used by filesystems to temporarily store any context as this
> > struct gets passed around in the fs.

So the prototype is:

int dir_readahead(int fd, off64_t offset, unsigned int count);

Why do we need a new syscall for this?

$ man 2 readahead
....
ssize_t readahead(int fd, off64_t offset, size_t count);
....
	EINVAL fd does not refer to a file type to which readahead() can be applied.


Cheers,

Dave.
-- 
Dave Chinner
david-FqsqvQoI3Ljby3iVrkZq2A@public.gmane.org

^ permalink raw reply

* Re: General flags to turn things off (getrandom, pid lookup, etc)
From: Eric W. Biederman @ 2014-07-31  2:37 UTC (permalink / raw)
  To: One Thousand Gnomes
  Cc: Andy Lutomirski, Paolo Bonzini, linux-crypto,
	Henrique de Moraes Holschuh, linux-kernel@vger.kernel.org,
	James Morris, LSM List, Al Viro, Linux API, Julien Tinnes,
	Theodore Ts'o, Greg Kroah-Hartman, Paul Moore, David Drysdale,
	Kees Cook, Meredydd Luff, Christoph Hellwig
In-Reply-To: <20140730222903.4c83a652@alan.etchedpixels.co.uk>

One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> writes:

> On Wed, 30 Jul 2014 11:41:41 -0700
> ebiederm@xmission.com (Eric W. Biederman) wrote:
>
>> One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> writes:
>> 
>> >> Andy you seem to be arguing here for two system calls.
>> >> get_urandom() and get_random().
>> >> 
>> >> Where get_urandom only blocks if there is not enough starting entropy,
>> >> and get_random(GRND_RANDOM) blocks if there is currently not enough
>> >> entropy.
>> >> 
>> >> That would allow -ENOSYS to be the right return value and it would
>> >> simply things for everyone.
>> >
>> > So you replace the "no file handle" special case with the "unsupported or
>> > disabled syscall" special case, which is even harder to test.
>> >
>> > Interfaces have failure modes. People who can't deal with that shouldn't
>> > be writing code that does anything important in languages which don't
>> > handle it for them.
>> 
>> Perhaps I misread the earlier conversation but it what I have read of
>> this discussion people want to disable some of get_random() modes with
>> seccomp.  Today get_random does not have any failure codes define except
>> -ENOSYS.
>> 
>> get_random(0) succeeding and get_random(GRND_RANDOM) returning -ENOSYS
>> has every chance of causing applications to legitimately assume the
>> get_random system call is not available in any mode.
>
> Or more likely it'll be used like this
>
> 	get_random(foo);		/* always works */
>
>
> Now the existing failure mode is is
>
> 	open(...)
> 	/* forget the check */
> 	read()
> 	/* forget the check */
>
> and triggered by evil local attacks on file handles. The "improved"
> behaviour is unchecked -ENOSYS returns which are likely to occur
> systemically when users run stuff on old kernels, in vm's with it off etc.
>
> So you've swapped the odd evil user attack on a single target for the
> likelyhood of mass generation of flawed keys with no error reporting.
>
> In fact you could do a better job of the whole mess in libc rather than
> the kernel, because in libc you'd write it like this
>
>          if (open(.. ) < 0)
> 		kill(getpid(), 9);
> 	 if (read(...) < expected)
> 		kill(getpid(), 9);
> 	 close(fd);
>
> and 
> a) on an older library you'd get a good failure (unable to execute the
> binary)
> b) on a newer system you'd get "do or die" behaviour and can improve its
> robustness as desired

I have said enough about the silliness of disabling this syscall with
seccomp or related infrastructure.

The aspect I like about get_random() is that it will silence the
requests from people to enable binary sysctl support in the kernel.
Just so they can get random numbers when /dev/random and /dev/urandom
are absent in their chroots.

sysctl(2) is finally legitmately going fading away.

Eric


^ permalink raw reply

* Re: [PATCH -v4] random: introduce getrandom(2) system call
From: Pavel Machek @ 2014-07-30 22:18 UTC (permalink / raw)
  To: Bernd Petrovitsch
  Cc: Bob Beck, Theodore Ts'o, linux-kernel, linux-api,
	linux-crypto, Theo de Raadt
In-Reply-To: <1406731254.26034.4.camel@thorin>

On Wed 2014-07-30 16:40:52, Bernd Petrovitsch wrote:
> On Mit, 2014-07-30 at 07:56 -0600, Bob Beck wrote:
> > Pavel. I have bit 'ol enterprise daemon running with established file
> > descriptors serving thousands of connections
> > which periodically require entropy.  Now I run out of descriptors. I
> > can't establish new connections. but I should
> > now halt all the other ones that require entropy?  I should raise
> > SIGKILL on my process serving these thousands
> > of connetions?  I don't think so.
> 
> If that long-running daemon periodically needs something from a device,
> one would better keep the fd for that open the whole time. Saves some
> CPU cycles and latency too BTW.

Agreed.

On the other hand, keeping a fd open is quite tricky for a
library. But better solution might be to make that easier.

open( , O_IM_A_LIBRARY_GIVE_ME_ONE_OF_THREE_RESERVED_FDS) might be one
solution. Actually, one reserved fd should be enough.

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply

* Re: General flags to turn things off (getrandom, pid lookup, etc)
From: One Thousand Gnomes @ 2014-07-30 21:29 UTC (permalink / raw)
  To: Eric W. Biederman
  Cc: Andy Lutomirski, Paolo Bonzini, linux-crypto,
	Henrique de Moraes Holschuh, linux-kernel@vger.kernel.org,
	James Morris, LSM List, Al Viro, Linux API, Julien Tinnes,
	Theodore Ts'o, Greg Kroah-Hartman, Paul Moore, David Drysdale,
	Kees Cook, Meredydd Luff, Christoph Hellwig
In-Reply-To: <87r412g04a.fsf@x220.int.ebiederm.org>

On Wed, 30 Jul 2014 11:41:41 -0700
ebiederm@xmission.com (Eric W. Biederman) wrote:

> One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> writes:
> 
> >> Andy you seem to be arguing here for two system calls.
> >> get_urandom() and get_random().
> >> 
> >> Where get_urandom only blocks if there is not enough starting entropy,
> >> and get_random(GRND_RANDOM) blocks if there is currently not enough
> >> entropy.
> >> 
> >> That would allow -ENOSYS to be the right return value and it would
> >> simply things for everyone.
> >
> > So you replace the "no file handle" special case with the "unsupported or
> > disabled syscall" special case, which is even harder to test.
> >
> > Interfaces have failure modes. People who can't deal with that shouldn't
> > be writing code that does anything important in languages which don't
> > handle it for them.
> 
> Perhaps I misread the earlier conversation but it what I have read of
> this discussion people want to disable some of get_random() modes with
> seccomp.  Today get_random does not have any failure codes define except
> -ENOSYS.
> 
> get_random(0) succeeding and get_random(GRND_RANDOM) returning -ENOSYS
> has every chance of causing applications to legitimately assume the
> get_random system call is not available in any mode.

Or more likely it'll be used like this

	get_random(foo);		/* always works */


Now the existing failure mode is is

	open(...)
	/* forget the check */
	read()
	/* forget the check */

and triggered by evil local attacks on file handles. The "improved"
behaviour is unchecked -ENOSYS returns which are likely to occur
systemically when users run stuff on old kernels, in vm's with it off etc.

So you've swapped the odd evil user attack on a single target for the
likelyhood of mass generation of flawed keys with no error reporting.

In fact you could do a better job of the whole mess in libc rather than
the kernel, because in libc you'd write it like this

         if (open(.. ) < 0)
		kill(getpid(), 9);
	 if (read(...) < expected)
		kill(getpid(), 9);
	 close(fd);

and 
a) on an older library you'd get a good failure (unable to execute the
binary)
b) on a newer system you'd get "do or die" behaviour and can improve its
robustness as desired

Alan

^ permalink raw reply

* Re: [PATCH RFC v3 net-next 3/3] samples: bpf: eBPF dropmon example in C
From: Alexei Starovoitov @ 2014-07-30 18:53 UTC (permalink / raw)
  To: Frank Ch. Eigler
  Cc: David S. Miller, Ingo Molnar, Linus Torvalds, Andy Lutomirski,
	Steven Rostedt, Daniel Borkmann, Chema Gonzalez, Eric Dumazet,
	Peter Zijlstra, Arnaldo Carvalho de Melo, Jiri Olsa,
	Thomas Gleixner, H. Peter Anvin, Andrew Morton, Kees Cook,
	Linux API, Network Development, LKML
In-Reply-To: <20140730173638.GB8745-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>

On Wed, Jul 30, 2014 at 10:36 AM, Frank Ch. Eigler <fche-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>> > What kind of locking/serialization is provided by the ebpf runtime
>> > over shared variables such as my_map?
>>
>> it's traditional rcu scheme.
>
> OK, that protects the table structure, but:
>
>> [...] In such case concurrent write access to map value can be done
>> with bpf_xadd instruction, though using normal read/write is also
>> allowed. In some cases the speed of racy var++ is preferred over
>> 'lock xadd'.
>
> ... so concurrency control over shared values is left up to the
> programmer.

yes. It has to be flexible and fast.
One of our main use cases is network analytics where a lot of
packets are going through ebpf programs, so every cycle counts.
Mandatory locks in critical path are not acceptable. If we add
locks they will be optional.

>> There are no lock/unlock function helpers available to ebpf
>> programs, since program may terminate early with div by zero
>> for example, so in-kernel lock helper implementation would
>> be complicated and slow. It's possible to do, but for the use
>> cases so far there is no need.
>
> OK, I hope that works out.  I've been told that dtrace does something
> similiar (!)  by eschewing protection on global variables such as
> strings.  In their case it's less bad than it sounds because they are
> used to offloading computation to userspace or to store only
> thread-local state, and accept the corollary limitations on control.

interesting.
btw, things like global variables, per-cpu storage are potential ebpf
features. So far they're 'nice to have' instead of 'mandatory'.
The maps are powerful enough to do the same:
Global storage is map of one element.
Per-cpu storage is map of num_cpu elements.

^ permalink raw reply

* Re: General flags to turn things off (getrandom, pid lookup, etc)
From: Eric W. Biederman @ 2014-07-30 18:41 UTC (permalink / raw)
  To: One Thousand Gnomes
  Cc: Andy Lutomirski, Paolo Bonzini, linux-crypto,
	Henrique de Moraes Holschuh, linux-kernel@vger.kernel.org,
	James Morris, LSM List, Al Viro, Linux API, Julien Tinnes,
	Theodore Ts'o, Greg Kroah-Hartman, Paul Moore, David Drysdale,
	Kees Cook, Meredydd Luff, Christoph Hellwig
In-Reply-To: <20140730153713.736881f0@alan.etchedpixels.co.uk>

One Thousand Gnomes <gnomes@lxorguk.ukuu.org.uk> writes:

>> Andy you seem to be arguing here for two system calls.
>> get_urandom() and get_random().
>> 
>> Where get_urandom only blocks if there is not enough starting entropy,
>> and get_random(GRND_RANDOM) blocks if there is currently not enough
>> entropy.
>> 
>> That would allow -ENOSYS to be the right return value and it would
>> simply things for everyone.
>
> So you replace the "no file handle" special case with the "unsupported or
> disabled syscall" special case, which is even harder to test.
>
> Interfaces have failure modes. People who can't deal with that shouldn't
> be writing code that does anything important in languages which don't
> handle it for them.

Perhaps I misread the earlier conversation but it what I have read of
this discussion people want to disable some of get_random() modes with
seccomp.  Today get_random does not have any failure codes define except
-ENOSYS.

get_random(0) succeeding and get_random(GRND_RANDOM) returning -ENOSYS
has every chance of causing applications to legitimately assume the
get_random system call is not available in any mode.

So the code either needs a defined error code for bad flags (-EINVAL) or
we need to split the syscall in two.  Now that I think about it having
the seccomp filter return -EINVAL if it doesn't like the parameter is
better that splitting a syscall.  Presumably that is what
get_random(UNSUPPORTED_FLAG) returns.

Eric

^ permalink raw reply

* Re: [PATCH RFC v3 net-next 3/3] samples: bpf: eBPF dropmon example in C
From: Frank Ch. Eigler @ 2014-07-30 17:36 UTC (permalink / raw)
  To: Alexei Starovoitov
  Cc: David S. Miller, Ingo Molnar, Linus Torvalds, Andy Lutomirski,
	Steven Rostedt, Daniel Borkmann, Chema Gonzalez, Eric Dumazet,
	Peter Zijlstra, Arnaldo Carvalho de Melo, Jiri Olsa,
	Thomas Gleixner, H. Peter Anvin, Andrew Morton, Kees Cook,
	Linux API, Network Development, LKML
In-Reply-To: <CAMEtUuxNs=HBhtFwhBX3HOU6+QqtYZ+v7sJ74+cg-o+bi5GdoA@mail.gmail.com>

Hi, Alexei -

> My understanding of systemtap is that the whole .stp script is converted
> to C, compiled as .ko and loaded, so all map walking and prints are
> happening in the kernel. Similarly for ktap which has special functions
> in kernel to print histograms.

That is correct.

> I thought dtrace printf are also happening from the kernel. What is
> the trick they use to know which pieces of dtrace script should be
> run in user space?

It appears as though the bytecode language running in the kernel sends
some action commands back out to userspace, not just plain data.


> In ebpf examples there are two C files: one for kernel with ebpf isa
> and one for userspace as native. I thought about combining them,
> but couldn't figure out a clean way of doing it.

(#if ?)


> > What kind of locking/serialization is provided by the ebpf runtime
> > over shared variables such as my_map?
> 
> it's traditional rcu scheme.

OK, that protects the table structure, but:

> [...] In such case concurrent write access to map value can be done
> with bpf_xadd instruction, though using normal read/write is also
> allowed. In some cases the speed of racy var++ is preferred over
> 'lock xadd'.

... so concurrency control over shared values is left up to the
programmer.

> There are no lock/unlock function helpers available to ebpf
> programs, since program may terminate early with div by zero
> for example, so in-kernel lock helper implementation would
> be complicated and slow. It's possible to do, but for the use
> cases so far there is no need.

OK, I hope that works out.  I've been told that dtrace does something
similiar (!)  by eschewing protection on global variables such as
strings.  In their case it's less bad than it sounds because they are
used to offloading computation to userspace or to store only
thread-local state, and accept the corollary limitations on control.

(Systemtap does fully & automatically protect shared variables, even
in the face of run-time script errors.)


- FChE

^ permalink raw reply

* Re: [RFC PATCH 1/1] ethtool: adding support for multiple slave port configuration
From: Mugunthan V N @ 2014-07-30 17:34 UTC (permalink / raw)
  To: John Fastabend, Ben Hutchings
  Cc: netdev-u79uwXL29TY76Z2rM5mHXA, davem-fT/PcQaiUtIeIZ0/mPfg9Q,
	linux-api-u79uwXL29TY76Z2rM5mHXA,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <53D52452.2020300-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>

On Sunday 27 July 2014 09:39 PM, John Fastabend wrote:
> On 07/26/2014 07:47 PM, Ben Hutchings wrote:
>> On Fri, 2014-07-25 at 17:58 +0530, Mugunthan V N wrote:
>>> Some Ethernet Swtich controllers like CPSW in AM335x, TI814x, DRA7x and
>>> AM43xx SoCs, Network Coprocessor in AM5K2E0x, Realtek Switch
>>> controllers
>>> etc has to capability of conneting multiple networks using L2 switching
>>> and has multiple phys. With the existing code, ethtool can communicate
>>> only to one phy.
>>>
>>> To enable user to communicate multiple phy connected to single Ethernet
>>> Switch controller, intoducing a optional new parameter in Ethtool
>>> interface
>>> to pass which slave to set/get the phy configuration.
>>
>> There was some discussion about configuration APIs for hardware/firmware
>> bridges earlier this year and I thought there was a consensus for
>> assigning a network device to each port.  This would remove the need to
>> identify ports within a device.  But I may have misremembered.
>>
>
> I like the approach of creating a network device for each port over
> having to use ethtool to program/discover them. I am currently looking
> at writing management applications for this and IMO it is much easier
> to discover and listen for events on network devices vs polling ethtool
> and iterating through slave indexs. Also you miss a lot of functionality
> that may be useful MTU for example that is not available configured via
> ethtool.
>
> One of the sticking points in earlier discussions was how to handle
> devices that have limited support for slave devices. When we create a
> netdev we expect the stack can bind to it and TX/RX packets which as
> I understand is not always possible? (I missed why we couldn't recv the
> packets over a switch port though with some skb->dev manipulation). In
> this case a feature flag could be used to resolve the feature
> dependencies.
>
>

John

I am also interested in participating in the above management api
development.

Regards
Mugunthan V N

^ permalink raw reply

* Re: [PATCH RFC v3 net-next 3/3] samples: bpf: eBPF dropmon example in C
From: Alexei Starovoitov @ 2014-07-30 17:17 UTC (permalink / raw)
  To: Frank Ch. Eigler
  Cc: David S. Miller, Ingo Molnar, Linus Torvalds, Andy Lutomirski,
	Steven Rostedt, Daniel Borkmann, Chema Gonzalez, Eric Dumazet,
	Peter Zijlstra, Arnaldo Carvalho de Melo, Jiri Olsa,
	Thomas Gleixner, H. Peter Anvin, Andrew Morton, Kees Cook,
	Linux API, Network Development, LKML
In-Reply-To: <y0m1tt226lj.fsf@fche.csb>

On Wed, Jul 30, 2014 at 8:45 AM, Frank Ch. Eigler <fche@redhat.com> wrote:
> For the record, this is not entirely accurate as to dtrace.  dtrace
> delegates aggregation and most reporting to userspace.  Also,
> systemtap is "short and deterministic" even for aggregations & nice
> graphs, but since it limits its storage & cpu consumption, its
> arrays/reports cannot get super large.

My understanding of systemtap is that the whole .stp script is converted
to C, compiled as .ko and loaded, so all map walking and prints are
happening in the kernel. Similarly for ktap which has special functions
in kernel to print histograms.
I thought dtrace printf are also happening from the kernel. What is the
trick they use to know which pieces of dtrace script should be run in
user space?
In ebpf examples there are two C files: one for kernel with ebpf isa
and one for userspace as native. I thought about combining them,
but couldn't figure out a clean way of doing it.

>> [...]
>> +SEC("events/skb/kfree_skb")
>> +int bpf_prog2(struct bpf_context *ctx)
>> +{
>> +[...]
>> +     value = bpf_map_lookup_elem(&my_map, &loc);
>> +     if (value)
>> +             (*(long *) value) += 1;
>> +     else
>> +             bpf_map_update_elem(&my_map, &loc, &init_val);
>> +     return 0;
>> +}
>
> What kind of locking/serialization is provided by the ebpf runtime
> over shared variables such as my_map?

it's traditional rcu scheme.
Programs are running under rcu_read_lock(), so that
bpf_map_lookup_elem() can return pointer to map value which
won't disappear while program is running.
In-kernel map implementation needs to use rcu style to match
ebpf program assumptions. map implementation is enforcing
the limit to the number of elements.
I didn't post 'array' type of map yet. bpf_map_lookup in this
implementation will just return 'base + index' pointer.
Regardless of type of map the same ebpf program running on
different cpus may lookup the same 'key' and receive the same
map value pointer. In such case concurrent write access to
map value can be done with bpf_xadd instruction, though
using normal read/write is also allowed. In some cases
the speed of racy var++ is preferred over 'lock xadd'.
There are no lock/unlock function helpers available to ebpf
programs, since program may terminate early with div by zero
for example, so in-kernel lock helper implementation would
be complicated and slow. It's possible to do, but for the use
cases so far there is no need.

^ permalink raw reply

* Re: [PATCH RFC v3 net-next 3/3] samples: bpf: eBPF dropmon example in C
From: Frank Ch. Eigler @ 2014-07-30 15:45 UTC (permalink / raw)
  To: Alexei Starovoitov
  Cc: David S. Miller, Ingo Molnar, Linus Torvalds, Andy Lutomirski,
	Steven Rostedt, Daniel Borkmann, Chema Gonzalez, Eric Dumazet,
	Peter Zijlstra, Arnaldo Carvalho de Melo, Jiri Olsa,
	Thomas Gleixner, H. Peter Anvin, Andrew Morton, Kees Cook,
	linux-api-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <1406000723-4872-4-git-send-email-ast-uqk4Ao+rVK5Wk0Htik3J/w@public.gmane.org>


ast wrote earlier:

> [...]
> dtrace/systemtap/ktap approach is to use one script file that should provide
> all desired functionality. That architectural decision overcomplicated their
> implementations.
>
> eBPF follows split model: everything that needs to process millions of events
> per second needs to run in kernel and needs to be short and deterministic,
> all other things like aggregation and nice graphs should run in user space.
> [...]

For the record, this is not entirely accurate as to dtrace.  dtrace
delegates aggregation and most reporting to userspace.  Also,
systemtap is "short and deterministic" even for aggregations & nice
graphs, but since it limits its storage & cpu consumption, its
arrays/reports cannot get super large.


> [...]
> +SEC("events/skb/kfree_skb")
> +int bpf_prog2(struct bpf_context *ctx)
> +{
> +[...]
> +	value = bpf_map_lookup_elem(&my_map, &loc);
> +	if (value)
> +		(*(long *) value) += 1;
> +	else
> +		bpf_map_update_elem(&my_map, &loc, &init_val);
> +	return 0;
> +}

What kind of locking/serialization is provided by the ebpf runtime
over shared variables such as my_map?


- FChE

^ permalink raw reply

* Re: [PATCH 11/11] seccomp: Add tgid and tid into seccomp_data
From: Andy Lutomirski @ 2014-07-30 14:52 UTC (permalink / raw)
  To: Eric W. Biederman
  Cc: Paolo Bonzini, Greg KH,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	James Morris, Paul Moore, LSM List, Al Viro, David Drysdale,
	Linux API, Kees Cook, Meredydd Luff, Julien Tinnes,
	Christoph Hellwig
In-Reply-To: <8761ifie81.fsf-JOvCrm2gF+uungPnsOpG7nhyD016LWXt@public.gmane.org>

On Jul 29, 2014 10:57 PM, "Eric W. Biederman" <ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org> wrote:
>
> Andy Lutomirski <luto-kltTT9wpgjJwATOyAt5JVQ@public.gmane.org> writes:
>
> > On Tue, Jul 29, 2014 at 9:08 PM, Eric W. Biederman
> > <ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org> wrote:
> >> Andy Lutomirski <luto-kltTT9wpgjJwATOyAt5JVQ@public.gmane.org> writes:
> >>
> >>> On Mon, Jul 28, 2014 at 2:18 PM, Eric W. Biederman
> >>> <ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org> wrote:
> >>>> Andy Lutomirski <luto-kltTT9wpgjJwATOyAt5JVQ@public.gmane.org> writes:
> >>>>
> >>>>> [cc: Eric Biederman]
> >>>>>
> >>>>
> >>>>> Can we do one better and add a flag to prevent any non-self pid
> >>>>> lookups?  This might actually be easy on top of the pid namespace work
> >>>>> (e.g. we could change the way that find_task_by_vpid works).
> >>>>>
> >>>>> It's far from just being signals.  There's access_process_vm, ptrace,
> >>>>> all the signal functions, clock_gettime (see CPUCLOCK_PID -- yes, this
> >>>>> is ridiculous), and probably some others that I've forgotten about or
> >>>>> never noticed in the first place.
> >>>>
> >>>> So here is the practical question.
> >>>>
> >>>> Are these processes that only can send signals to their thread group
> >>>> allowed to call fork()?
> >>>>
> >>>>
> >>>> If fork is allowed and all pid lookups are restricted to their own
> >>>> thread group that wait, waitpid, and all of the rest of the wait family
> >>>> will never return the pids of their children, and zombies will
> >>>> accumulate.  Aka the semantics are fundamentally broken.
> >>>
> >>> Good point.
> >>>
> >>> I can imagine at least three ways that fork() could continue working, though:
> >>>
> >>> 1. Allow lookups of immediate children, too.  (I don't love this one.)
> >>> 2. Allow non-self pids to be translated in but not out.  This way
> >>> P_ALL will continue working.
> >>> 3. Have the kernel treat any PID-restricted process as though it were NOCLDWAIT.
> >>>
> >>> I think I like #3.  Thoughts?
> >>>
> >>>>
> >>>> If fork is not allowed pid namespaces already solve this problem.
> >>>
> >>> PID namespaces are fairly heavyweight.  Julien pointed out that using
> >>> PID namespaces requires a bunch of dummy PID 1 processes.
> >>
> >> Only if you can't tolerate init exiting.  The reasoning with respect to
> >> signals and signals being ignored was wrong.  And if you only have one
> >> process you care about and no children to worry about neither the
> >> difference in signal handling nor the world dies whe init exits applies.
> >
> > Can you elaborate?  It seems entirely plausible to me that there are
> > programs that won't work right as PID 1 without considerable
> > adaptation.
>
> The only funny things about pid 1 of a pid namespace are:
> - children can't send signals to pid 1 unless a signal handler has
>   been established.
> - All children die when the parent dies.
> - Grand children become zombies of the parent when the children die.
> - The pid is 1.
>
> That is almost everything is the same and it takes almost no adaptation
> (really) to run as the initial pid in a pid namespace.
>
> Not being able to receive signals (which is the argument I read against
> them) is bogus.  You just have to set your signal handler to something
> besides SIG_DFL.
>
> So I have my question:  What is the use case people are trying to solve
> by filtering signals and pid lookups.  If children are not part of the
> goal a pid namespace will work just fine.
>
> >> Therefore given what I have read described pid namespaces are a trivial
> >> solution to this problem space.
> >
> > pid namespaces also won't work in the context of Capsicum unless you
> > want every single Capsicum process to be its own pid namespace.
>
> For a tightly bound process I don't see why each process could not be
> it's own pid namespace.

Two main reasons: You can't put yourself in a pid namespace, so you
need to fork into your sandbox, and you can't prevent yourself from
seeing your children (although, as noted, my approach has issues here,
too, but I think this is more easily solved outside the context of
namespaces).

>
> > Also,
> > pid namespaces don't offer any way to protect children from parents.
>
> And my presumption was that there were not any children because the
> semantics suggested so far do not properly support children.
>

I'd like to try to fix that.

Another approach: let waiting for zombies that are immediate children
be an exception.

--Andy

> Eric

^ permalink raw reply

* Re: [RFC PATCHv2 00/11] Adding FreeBSD's Capsicum security framework
From: Andy Lutomirski @ 2014-07-30 14:51 UTC (permalink / raw)
  To: Eric W. Biederman
  Cc: Al Viro, Paolo Bonzini, Greg Kroah-Hartman,
	linux-kernel@vger.kernel.org, Paul Moore, James Morris, LSM List,
	David Drysdale, Linux API, Kees Cook, Meredydd Luff,
	Christoph Hellwig
In-Reply-To: <87iomfgyd2.fsf@x220.int.ebiederm.org>

On Jul 29, 2014 11:25 PM, "Eric W. Biederman" <ebiederm@xmission.com> wrote:
>
>
> I have cut this down to just focus on O_BENEATH openat case.
>
> David Drysdale <drysdale@google.com> writes:
>
> > On Mon, Jul 28, 2014 at 10:13 PM, Eric W. Biederman
> > <ebiederm@xmission.com> wrote:
>
> >> Nope.  What you can implement today if you want fine grained limitations
> >> like this is to create a mount namespace with exactly the subdirectory
> >> tree you want to allow access to and to return a file descriptor that
> >> points into that mount namespace.  (When complete the only user of that
> >> mount namespace would be your file descriptor).
> >
> > How does that solve the particular example I mentioned?  The DFD
> > within the mount namespace will still allow any operation on any file
> > that's already in the subdirectory -- or am I misunderstanding
> > something?
>
> The goal was to bound the DFD to the directory and all of it's
> subdirectories such that openat(dfd, "../../..") would open
> the dfd, and that further opens of other directories would also not
> allow you to escape.
>
> Since the mount namespace only contains the choosen directory and it's
> subdirectories that works easily and trivially.
>
> So while you can indeed perform any file operation on that dfd who
> cares because none of those operations can get you anywhere you aren't
> supposed to be.
>
> My point was that you can as granular as you would like by binding a dfd
> to a mount namespace instead of binding a process to a mount namespace,
> and the code already exists and is being maintained.
>
> So while things are not packaged in the form that has been requested it
> looks to me as if the functionality for directories already exists
> within the Linux kernel.

I think this would be amazingly expensive -- every constrained fd
would need to carry an entire mount namespace with it.  That namespace
might need to have shared recursive mounts under it.  And dfds created
for subdirectories would need yet another mount namespace.  And all
these mount namespaces would probably need user namespaces to go along
with them.

It would also have odd semantics.  If you have a dfd pointing to /foo,
and /foo/link is a symlink to "../bar", then looking up "link"
relative to /foo should fail; it should not try to resolve /foo/bar.

IOW I think this is impractical.

>
> Eric

^ permalink raw reply

* Re: [PATCH -v4] random: introduce getrandom(2) system call
From: Bernd Petrovitsch @ 2014-07-30 14:40 UTC (permalink / raw)
  To: Bob Beck
  Cc: Pavel Machek, Theodore Ts'o, linux-kernel, linux-api,
	linux-crypto, Theo de Raadt
In-Reply-To: <CAComcpMnAdOk=Hs8Dtc7VZeJDHnpKQtzZ_=_67tqCeYU3pdJ_Q@mail.gmail.com>

On Mit, 2014-07-30 at 07:56 -0600, Bob Beck wrote:
> Pavel. I have bit 'ol enterprise daemon running with established file
> descriptors serving thousands of connections
> which periodically require entropy.  Now I run out of descriptors. I
> can't establish new connections. but I should
> now halt all the other ones that require entropy?  I should raise
> SIGKILL on my process serving these thousands
> of connetions?  I don't think so.

If that long-running daemon periodically needs something from a device,
one would better keep the fd for that open the whole time. Saves some
CPU cycles and latency too BTW.

	Bernd
-- 
"I dislike type abstraction if it has no real reason. And saving
on typing is not a good reason - if your typing speed is the main
issue when you're coding, you're doing something seriously wrong."
    - Linus Torvalds

^ permalink raw reply

* Re: General flags to turn things off (getrandom, pid lookup, etc)
From: One Thousand Gnomes @ 2014-07-30 14:37 UTC (permalink / raw)
  To: Andy Lutomirski
  Cc: Eric W. Biederman, Julien Tinnes, David Drysdale, Al Viro,
	Paolo Bonzini, LSM List, Greg Kroah-Hartman, Paul Moore,
	James Morris, Linux API, Meredydd Luff, Christoph Hellwig,
	linux-kernel@vger.kernel.org, Kees Cook, Theodore Ts'o,
	Henrique de Moraes Holschuh, linux-crypto
In-Reply-To: <CALCETrXHNFYaQjksBJyN3O0HJpKw_D_tU7b4O23=nyJt9CPSLg@mail.gmail.com>

> > We sort of have one. It's called capable(). Just needs extending to cover
> > anything else you care about, and probably all the numeric constants
> > replacing with textual names.
> >
> 
> Except that it's all backwards: these are things that default to *on*,
> and people might want them to turn off.  capable() is totally fscked
> if you want otherwise unprivileged users to carry capabilities around

The userspace API is, but capable() as a userspace API and capable() as
an in kernel check are only connected by history.

For the in kernel part you can either teach everyone another disjoint API
or we can have a single API in kernel for saying "is XYZ allowed".

^ permalink raw reply

* Re: General flags to turn things off (getrandom, pid lookup, etc)
From: One Thousand Gnomes @ 2014-07-30 14:37 UTC (permalink / raw)
  To: Eric W. Biederman
  Cc: Andy Lutomirski, Paolo Bonzini, linux-crypto,
	Henrique de Moraes Holschuh, linux-kernel@vger.kernel.org,
	James Morris, LSM List, Al Viro, Linux API, Julien Tinnes,
	Theodore Ts'o, Greg Kroah-Hartman, Paul Moore, David Drysdale,
	Kees Cook, Meredydd Luff, Christoph Hellwig
In-Reply-To: <87oawa740c.fsf@x220.int.ebiederm.org>

> Andy you seem to be arguing here for two system calls.
> get_urandom() and get_random().
> 
> Where get_urandom only blocks if there is not enough starting entropy,
> and get_random(GRND_RANDOM) blocks if there is currently not enough
> entropy.
> 
> That would allow -ENOSYS to be the right return value and it would
> simply things for everyone.

So you replace the "no file handle" special case with the "unsupported or
disabled syscall" special case, which is even harder to test.

Interfaces have failure modes. People who can't deal with that shouldn't
be writing code that does anything important in languages which don't
handle it for them.

Alan

^ permalink raw reply

* Re: [PATCH -v5] random: introduce getrandom(2) system call
From: Rolf Eike Beer @ 2014-07-30 14:34 UTC (permalink / raw)
  To: Theodore Ts'o
  Cc: Linux Kernel Developers List, linux-api-u79uwXL29TY76Z2rM5mHXA,
	linux-crypto-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <1406212287-9855-1-git-send-email-tytso-3s7WtUTddSA@public.gmane.org>

Theodore Ts'o wrote:

It's me again, finding only one issue per cycle :/

> 	EAGAIN		The requested entropy was not available, and
> 			getentropy(2) would have blocked if GRND_BLOCK flag
> 			was set.

"if GRND_NONBLOCK flag was not set"

Eike

^ permalink raw reply

* Re: [PATCH -v4] random: introduce getrandom(2) system call
From: Bob Beck @ 2014-07-30 13:56 UTC (permalink / raw)
  To: Pavel Machek
  Cc: Theodore Ts'o, linux-kernel, linux-api, linux-crypto,
	Theo de Raadt
In-Reply-To: <20140730122620.GC13965@amd.pavel.ucw.cz>

Pavel. I have bit 'ol enterprise daemon running with established file
descriptors serving thousands of connections
which periodically require entropy.  Now I run out of descriptors. I
can't establish new connections. but I should
now halt all the other ones that require entropy?  I should raise
SIGKILL on my process serving these thousands
of connetions?  I don't think so.



On Wed, Jul 30, 2014 at 6:26 AM, Pavel Machek <pavel@ucw.cz> wrote:
> Hi!
>
>> The rationale of this system call is to provide resiliance against
>> file descriptor exhaustion attacks, where the attacker consumes all
>> available file descriptors, forcing the use of the fallback code where
>> /dev/[u]random is not available.  Since the fallback code is often not
>> well-tested, it is better to eliminate this potential failure mode
>> entirely.
>
> I'm not sure I understand the rationale; if someone can eat all your
> file descriptors, he can make you stop working. So you can just stop
> working when you can't open /dev/urandom, no?
>
> Fallback code is probably very bad idea to use...
>
>> The other feature provided by this new system call is the ability to
>> request randomness from the /dev/urandom entropy pool, but to block
>> until at least 128 bits of entropy has been accumulated in the
>> /dev/urandom entropy pool.  Historically, the emphasis in the
>> /dev/urandom development has been to ensure that urandom pool is
>> initialized as quickly as possible after system boot, and preferably
>> before the init scripts start execution.
>
> Sounds like ioctl() for /dev/urandom for this behaviour would be nice?
>
>                                                                         Pavel
> --
> (english) http://www.livejournal.com/~pavelmachek
> (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply

* Re: [PATCH -v4] random: introduce getrandom(2) system call
From: Pavel Machek @ 2014-07-30 12:26 UTC (permalink / raw)
  To: Theodore Ts'o; +Cc: linux-kernel, linux-api, linux-crypto, beck, deraadt
In-Reply-To: <1405718127-30042-1-git-send-email-tytso@mit.edu>

Hi!

> The rationale of this system call is to provide resiliance against
> file descriptor exhaustion attacks, where the attacker consumes all
> available file descriptors, forcing the use of the fallback code where
> /dev/[u]random is not available.  Since the fallback code is often not
> well-tested, it is better to eliminate this potential failure mode
> entirely.

I'm not sure I understand the rationale; if someone can eat all your
file descriptors, he can make you stop working. So you can just stop
working when you can't open /dev/urandom, no?

Fallback code is probably very bad idea to use...

> The other feature provided by this new system call is the ability to
> request randomness from the /dev/urandom entropy pool, but to block
> until at least 128 bits of entropy has been accumulated in the
> /dev/urandom entropy pool.  Historically, the emphasis in the
> /dev/urandom development has been to ensure that urandom pool is
> initialized as quickly as possible after system boot, and preferably
> before the init scripts start execution.

Sounds like ioctl() for /dev/urandom for this behaviour would be nice?

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply

* Re: [RFC PATCHv2 00/11] Adding FreeBSD's Capsicum security framework
From: Eric W. Biederman @ 2014-07-30  6:22 UTC (permalink / raw)
  To: David Drysdale
  Cc: LSM List, linux-kernel@vger.kernel.org, Greg Kroah-Hartman,
	Alexander Viro, Meredydd Luff, Kees Cook, James Morris,
	Andy Lutomirski, Paolo Bonzini, Paul Moore, Christoph Hellwig,
	Linux API
In-Reply-To: <CAHse=S_w4+AMuc=-XbAK_PiaD56_ks13R53RENMHif5KRN_Kiw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>


I have cut this down to just focus on O_BENEATH openat case.

David Drysdale <drysdale-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> writes:

> On Mon, Jul 28, 2014 at 10:13 PM, Eric W. Biederman
> <ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org> wrote:

>> Nope.  What you can implement today if you want fine grained limitations
>> like this is to create a mount namespace with exactly the subdirectory
>> tree you want to allow access to and to return a file descriptor that
>> points into that mount namespace.  (When complete the only user of that
>> mount namespace would be your file descriptor).
>
> How does that solve the particular example I mentioned?  The DFD
> within the mount namespace will still allow any operation on any file
> that's already in the subdirectory -- or am I misunderstanding
> something?

The goal was to bound the DFD to the directory and all of it's
subdirectories such that openat(dfd, "../../..") would open
the dfd, and that further opens of other directories would also not
allow you to escape.

Since the mount namespace only contains the choosen directory and it's
subdirectories that works easily and trivially.

So while you can indeed perform any file operation on that dfd who
cares because none of those operations can get you anywhere you aren't
supposed to be.

My point was that you can as granular as you would like by binding a dfd
to a mount namespace instead of binding a process to a mount namespace,
and the code already exists and is being maintained.

So while things are not packaged in the form that has been requested it
looks to me as if the functionality for directories already exists
within the Linux kernel.

Eric

^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox