* Re: [patch 2/3] xen/privcmd: return -EFAULT on error
From: Andres Lagar-Cavilla @ 2012-09-09 19:49 UTC (permalink / raw)
To: Dan Carpenter
Cc: Jeremy Fitzhardinge, xen-devel, Konrad Rzeszutek Wilk,
Andres Lagar-Cavilla, kernel-janitors, virtualization
In-Reply-To: <20120908095735.GB608@elgon.mountain>
On Sep 8, 2012, at 5:57 AM, Dan Carpenter wrote:
> __copy_to_user() returns the number of bytes remaining to be copied but
> we want to return a negative error code here.
>
> Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Acked-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>
>
> diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
> index fdff8f9..0ce006a 100644
> --- a/drivers/xen/privcmd.c
> +++ b/drivers/xen/privcmd.c
> @@ -393,8 +393,11 @@ static long privcmd_ioctl_mmap_batch(void __user *udata, int version)
> state.err = err_array;
> ret = traverse_pages(m.num, sizeof(xen_pfn_t),
> &pagelist, mmap_return_errors_v1, &state);
> - } else if (version == 2)
> + } else if (version == 2) {
> ret = __copy_to_user(m.err, err_array, m.num * sizeof(int));
> + if (ret)
> + ret = -EFAULT;
> + }
>
> /* If we have not had any EFAULT-like global errors then set the global
> * error to -ENOENT if necessary. */
^ permalink raw reply
* Re: [patch 1/3] xen/privcmd: check for integer overflow in ioctl
From: Andres Lagar-Cavilla @ 2012-09-09 19:49 UTC (permalink / raw)
To: Dan Carpenter
Cc: Jeremy Fitzhardinge, xen-devel, Konrad Rzeszutek Wilk,
Andres Lagar-Cavilla, kernel-janitors, virtualization
In-Reply-To: <20120908095208.GA608@elgon.mountain>
On Sep 8, 2012, at 5:52 AM, Dan Carpenter wrote:
> If m.num is too large then the "m.num * sizeof(*m.arr)" multiplication
> could overflow and the access_ok() check wouldn't test the right size.
>
> Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
Acked-by: Andres Lagar-Cavilla <andres@lagarcavilla.org>
> ---
> Only needed in linux-next.
>
> diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
> index 215a3c0..fdff8f9 100644
> --- a/drivers/xen/privcmd.c
> +++ b/drivers/xen/privcmd.c
> @@ -325,6 +325,8 @@ static long privcmd_ioctl_mmap_batch(void __user *udata, int version)
> return -EFAULT;
> /* Returns per-frame error in m.arr. */
> m.err = NULL;
> + if (m.num > SIZE_MAX / sizeof(*m.arr))
> + return -EINVAL;
> if (!access_ok(VERIFY_WRITE, m.arr, m.num * sizeof(*m.arr)))
> return -EFAULT;
> break;
> @@ -332,6 +334,8 @@ static long privcmd_ioctl_mmap_batch(void __user *udata, int version)
> if (copy_from_user(&m, udata, sizeof(struct privcmd_mmapbatch_v2)))
> return -EFAULT;
> /* Returns per-frame error code in m.err. */
> + if (m.num > SIZE_MAX / sizeof(*m.err))
> + return -EINVAL;
> if (!access_ok(VERIFY_WRITE, m.err, m.num * (sizeof(*m.err))))
> return -EFAULT;
> break;
^ permalink raw reply
* [PATCHv4] virtio-spec: virtio network device multiqueue support
From: Michael S. Tsirkin @ 2012-09-09 13:03 UTC (permalink / raw)
To: kvm, virtualization, netdev; +Cc: rick.jones2, pbonzini, levinsasha928
Add multiqueue support to virtio network device. Add a new feature flag
VIRTIO_NET_F_MULTIQUEUE for this feature, a +new configuration field
max_virtqueue_pairs to detect supported number +of virtqueues as well as
a new command VIRTIO_NET_CTRL_STEERING to +program packet steering.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Changes from v3:
Address Sasha's comments
- drop debug fields - less fields less to debug :)
- clarify max_virtqueue_pairs field and steering param field
- misc typos
Address Paolo's comments
- Fixed old rule name left over from v2
Address Rick's comment
- Tweaked wording
Changes from v2:
Address Jason's comments on v2:
- Changed STEERING_HOST to STEERING_RX_FOLLOWS_TX:
this is both clearer and easier to support.
It does not look like we need a separate steering command
since host can just watch tx packets as they go.
- Moved RX and TX steering sections near each other.
- Add motivation for other changes in v2
Changes from Jason's rfc:
- reserved vq 3: this makes all rx vqs even and tx vqs odd, which
looks nicer to me.
- documented packet steering, added a generalized steering programming
command. Current modes are single queue and host driven multiqueue,
but I envision support for guest driven multiqueue in the future.
- make default vqs unused when in mq mode - this wastes some memory
but makes it more efficient to switch between modes as
we can avoid this causing packet reordering.
If this looks OK to everyone, we can proceed with finalizing the
implementation. This patch is against
eb9fc84d0d3c46438aaab190e2401a9e5409a052 in virtio-spec git tree.
---
virtio-spec.lyx | 453 +++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 446 insertions(+), 7 deletions(-)
diff --git a/virtio-spec.lyx b/virtio-spec.lyx
index fb6a4e3..2c2490e 100644
--- a/virtio-spec.lyx
+++ b/virtio-spec.lyx
@@ -58,6 +58,7 @@
\html_be_strict false
\author -608949062 "Rusty Russell,,,"
\author 1531152142 "Paolo Bonzini,,,"
+\author 1986246365 "Michael S. Tsirkin"
\end_header
\begin_body
@@ -3896,6 +3897,61 @@ Only if VIRTIO_NET_F_CTRL_VQ set
\end_inset
+\change_inserted 1986246365 1346663522
+ 3: reserved
+\end_layout
+
+\begin_layout Description
+
+\change_inserted 1986246365 1346663550
+4: receiveq1.
+ 5: transmitq1.
+ 6: receiveq2.
+ 7.
+ transmitq2.
+ ...
+ 2
+\emph on
+N
+\emph default
++2:receivq
+\emph on
+N
+\emph default
+, 2
+\emph on
+N
+\emph default
++3:transmitq
+\emph on
+N
+\emph default
+
+\begin_inset Foot
+status open
+
+\begin_layout Plain Layout
+
+\change_inserted 1986246365 1346663558
+Only if VIRTIO_NET_F_CTRL_VQ set.
+
+\emph on
+N
+\emph default
+ is indicated by
+\emph on
+max_virtqueue_pairs
+\emph default
+ field.
+\change_unchanged
+
+\end_layout
+
+\end_inset
+
+
+\change_unchanged
+
\end_layout
\begin_layout Description
@@ -4056,6 +4112,17 @@ VIRTIO_NET_F_CTRL_VLAN
\begin_layout Description
VIRTIO_NET_F_GUEST_ANNOUNCE(21) Guest can send gratuitous packets.
+\change_inserted 1986246365 1346617842
+
+\end_layout
+
+\begin_layout Description
+
+\change_inserted 1986246365 1346618103
+VIRTIO_NET_F_MULTIQUEUE(22) Device has multiple receive and transmission
+ queues.
+\change_unchanged
+
\end_layout
\end_deeper
@@ -4068,11 +4135,45 @@ configuration
\begin_inset space ~
\end_inset
-layout Two configuration fields are currently defined.
+layout
+\change_deleted 1986246365 1346671560
+Two
+\change_inserted 1986246365 1346671647
+Six
+\change_unchanged
+ configuration fields are currently defined.
The mac address field always exists (though is only valid if VIRTIO_NET_F_MAC
is set), and the status field only exists if VIRTIO_NET_F_STATUS is set.
Two read-only bits are currently defined for the status field: VIRTIO_NET_S_LIN
K_UP and VIRTIO_NET_S_ANNOUNCE.
+
+\change_inserted 1986246365 1347194909
+ The following read-only field,
+\emph on
+max_virtqueue_pairs
+\emph default
+ only exists if VIRTIO_NET_F_MULTIQUEUE is set.
+ This field specifies the maximum number of each of transmit and receive
+ virtqueues (receiveq1..receiveq
+\emph on
+N
+\emph default
+ and transmitq1..transmitq
+\emph on
+N
+\emph default
+ respectively;
+\emph on
+N
+\emph default
+=
+\emph on
+max_virtqueue_pairs
+\emph default
+) that can be used for multiqueue operation, excluding the default receiveq(0)
+ and transmitq(1) virtqueues.
+
+\change_unchanged
\begin_inset listings
inline false
@@ -4105,6 +4206,15 @@ struct virtio_net_config {
\begin_layout Plain Layout
u16 status;
+\change_inserted 1986246365 1346671221
+
+\end_layout
+
+\begin_layout Plain Layout
+
+\change_inserted 1986246365 1346671532
+
+ u16 max_virtqueue_pairs;
\end_layout
\begin_layout Plain Layout
@@ -4151,6 +4261,18 @@ physical
\begin_layout Enumerate
If the VIRTIO_NET_F_CTRL_VQ feature bit is negotiated, identify the control
virtqueue.
+\change_inserted 1986246365 1346618052
+
+\end_layout
+
+\begin_layout Enumerate
+
+\change_inserted 1986246365 1346618175
+If VIRTIO_NET_F_MULTIQUEUE feature bit is negotiated, identify the receive
+ and transmission queues that are going to be used in multiqueue mode.
+ Only queues that are going to be used need to be initialized.
+\change_unchanged
+
\end_layout
\begin_layout Enumerate
@@ -4168,7 +4290,11 @@ status
\end_layout
\begin_layout Enumerate
-The receive virtqueue should be filled with receive buffers.
+The receive virtqueue
+\change_inserted 1986246365 1346618180
+(s)
+\change_unchanged
+ should be filled with receive buffers.
This is described in detail below in
\begin_inset Quotes eld
\end_inset
@@ -4513,6 +4639,8 @@ Note that the header will be two bytes longer for the VIRTIO_NET_F_MRG_RXBUF
\end_inset
+\change_deleted 1986246365 1346932640
+
\end_layout
\begin_layout Subsection*
@@ -4988,8 +5116,24 @@ status open
The Guest needs to check VIRTIO_NET_S_ANNOUNCE bit in status field when
it notices the changes of device configuration.
The command VIRTIO_NET_CTRL_ANNOUNCE_ACK is used to indicate that driver
- has recevied the notification and device would clear the VIRTIO_NET_S_ANNOUNCE
- bit in the status filed after it received this command.
+ has rece
+\change_inserted 1986246365 1346663932
+i
+\change_unchanged
+v
+\change_deleted 1986246365 1346663934
+i
+\change_unchanged
+ed the notification and device would clear the VIRTIO_NET_S_ANNOUNCE bit
+ in the status fi
+\change_inserted 1986246365 1346663942
+e
+\change_unchanged
+l
+\change_deleted 1986246365 1346663943
+e
+\change_unchanged
+d after it received this command.
\end_layout
\begin_layout Standard
@@ -5004,10 +5148,306 @@ Sending the gratuitous packets or marking there are pending gratuitous packets
\begin_layout Enumerate
Sending VIRTIO_NET_CTRL_ANNOUNCE_ACK command through control vq.
+\change_deleted 1986246365 1346662247
+
\end_layout
-\begin_layout Enumerate
+\begin_layout Subsection*
+
+\change_inserted 1986246365 1346932658
+\begin_inset CommandInset label
+LatexCommand label
+name "sub:Transmit-Packet-Steering"
+
+\end_inset
+
+Transmit Packet Steering
+\end_layout
+
+\begin_layout Standard
+
+\change_inserted 1986246365 1346932658
+When VIRTIO_NET_F_MULTIQUEUE feature bit is negotiated, guest can use any
+ of multiple configured transmit queues to transmit a given packet.
+ To avoid packet reordering by device (which generally leads to performance
+ degradation) driver should attempt to utilize the same transmit virtqueue
+ for all packets of a given transmit flow.
+ For bi-directional protocols (in practice, TCP), a given network connection
+ can utilize both transmit and receive queues.
+ For best performance, packets from a single connection should utilize the
+ paired transmit and receive queues from the same virtqueue pair; for example
+ both transmitqN and receiveqN.
+ This rule makes it possible to optimize processing on the device side,
+ but this is not a hard requirement: devices should function correctly even
+ when this rule is not followed.
+\end_layout
+
+\begin_layout Standard
+
+\change_inserted 1986246365 1346932658
+Driver selects an active steering rule using VIRTIO_NET_CTRL_STEERING command
+ (this controls both which virtqueue is selected for a given packet for
+ receive and notifies the device which virtqueues are about to be used for
+ transmit).
+\end_layout
+
+\begin_layout Standard
+
+\change_inserted 1986246365 1346932658
+This command accepts a single out argument in the following format:
+\end_layout
+
+\begin_layout Standard
+
+\change_inserted 1986246365 1346932658
+\begin_inset listings
+inline false
+status open
+
+\begin_layout Plain Layout
+
+\change_inserted 1986246365 1347192845
+
+#define VIRTIO_NET_CTRL_STEERING 4
+\end_layout
+
+\begin_layout Plain Layout
+
+\change_inserted 1986246365 1346932658
+
+\end_layout
+
+\begin_layout Plain Layout
+
+\change_inserted 1986246365 1346932658
+
+struct virtio_net_ctrl_steering {
+\end_layout
+
+\begin_layout Plain Layout
+
+\change_inserted 1986246365 1346932658
+
+ u8 current_steering_rule;
+\end_layout
+
+\begin_layout Plain Layout
+
+\change_inserted 1986246365 1346932658
+
+ u8 reserved;
+\end_layout
+
+\begin_layout Plain Layout
+
+\change_inserted 1986246365 1346932658
+
+ u16 current_steering_param;
+\end_layout
+
+\begin_layout Plain Layout
+
+\change_inserted 1986246365 1346932658
+
+};
+\end_layout
+
+\begin_layout Plain Layout
+
+\change_inserted 1986246365 1347192841
+
+#define VIRTIO_NET_CTRL_STEERING_SINGLE 0
+\end_layout
+
+\begin_layout Plain Layout
+
+\change_inserted 1986246365 1347192840
+
+#define VIRTIO_NET_CTRL_STEERING_RX_FOLLOWS_TX 1
+\end_layout
+
+\end_inset
+
+
+\end_layout
+
+\begin_layout Standard
+
+\change_inserted 1986246365 1347193028
+The field
+\emph on
+rule
+\emph default
+ specifies the function used to select transmit virtqueue for a given packet;
+ the field
+\emph on
+param
+\emph default
+ makes it possible to pass an extra parameter if appropriate.
+ When
+\emph on
+rule
+\emph default
+ is set to VIRTIO_NET_CTRL_STEERING_SINGLE (this is the default) all packets
+ are steered to the default virtqueue transmitq (1); param is unused; this
+ is the default.
+ With any other rule, When
+\emph on
+rule
+\emph default
+ is set to VIRTIO_NET_CTRL_STEERING_RX_FOLLOWS_TX packets are steered by
+ driver to the first
+\emph on
+N
+\emph default
+=(
+\emph on
+param
+\emph default
++1) multiqueue virtqueues transmitq1...transmitqN; the default transmitq is
+ unused.
+ Driver must have configured all these (
+\emph on
+param
+\emph default
++1) virtqueues beforehand.
+\end_layout
+
+\begin_layout Standard
+
+\change_inserted 1986246365 1347193114
+Supported steering rules can be added and removed in the future.
+ Driver should check that the request to change the steering rule was successful
+ by checking ack values of the command.
+ As selecting a specific steering is an optimization feature, drivers should
+ avoid hard failure and fall back on using a supported steering rule if
+ this command fails.
+ The default steering rule is VIRTIO_NET_CTRL_STEERING_SINGLE.
+ It will not be removed.
+\end_layout
+
+\begin_layout Standard
+
+\change_inserted 1986246365 1346932658
+When the steering rule is modified, some packets can still be outstanding
+ in one or more of the transmit virtqueues.
+ Since drivers might choose to modify the current steering rule at a high
+ rate (e.g.
+ adaptively in response to changes in the workload) to avoid reordering
+ packets, device is recommended to complete processing of the transmit queue(s)
+ utilized by the original steering before processing any packets delivered
+ by the modified steering rule.
+\end_layout
+
+\begin_layout Standard
+
+\change_inserted 1986246365 1346932658
+For debugging, the current steering rule can also be read from the configuration
+ space.
+\end_layout
+
+\begin_layout Subsection*
+
+\change_inserted 1986246365 1346670357
+\begin_inset CommandInset label
+LatexCommand label
+name "sub:Receive-Packet-Steering"
+
+\end_inset
+
+Receive Packet Steering
+\end_layout
+
+\begin_layout Standard
+
+\change_inserted 1986246365 1346671046
+When VIRTIO_NET_F_MULTIQUEUE feature bit is negotiated, device can use any
+ of multiple configured receive queues to pass a given packet to driver.
+ Driver controls which virtqueue is selected in practice by configuring
+ packet steering rule using VIRTIO_NET_CTRL_STEERING command, as described
+ above
+\begin_inset CommandInset ref
+LatexCommand ref
+reference "sub:Transmit-Packet-Steering"
+
+\end_inset
+
.
+\end_layout
+
+\begin_layout Standard
+
+\change_inserted 1986246365 1347193175
+The field
+\emph on
+rule
+\emph default
+ specifies the function used to select receive virtqueue for a given packet;
+ the field
+\emph on
+param
+\emph default
+ makes it possible to pass an extra parameter if appropriate.
+ When
+\emph on
+rule
+\emph default
+ is set to VIRTIO_NET_CTRL_STEERING_SINGLE all packets are steered to the
+ default virtqueue receiveq (0); param is unused; this is the default.
+ When
+\emph on
+rule
+\emph default
+ is set to VIRTIO_NET_CTRL_STEERING_RX_FOLLOWS_TX packets are steered by
+ host to the first
+\emph on
+N
+\emph default
+=(
+\emph on
+param
+\emph default
++1) multiqueue virtqueues receiveq1...receiveqN; the default receiveq is unused.
+ Driver must have configured all these (
+\emph on
+param
+\emph default
++1) virtqueues beforehand.
+ For best performance for bi-directional flows (such as TCP) device should
+ detect the flow to virtqueue pair mapping on transmit and select the receive
+ virtqueue from the same virtqueue pair.
+ For uni-directional flows, or when this mapping information is missing,
+ a device-specific steering function is used.
+\change_unchanged
+
+\end_layout
+
+\begin_layout Standard
+
+\change_inserted 1986246365 1346669564
+Supported steering rules can be added and removed in the future.
+ Driver should probe for supported rules by checking ack values of the command.
+\end_layout
+
+\begin_layout Standard
+
+\change_inserted 1986246365 1346932135
+When the steering rule is modified, some packets can still be outstanding
+ in one or more of the virtqueues.
+ Device is not required to wait for these packets to be consumed before
+ delivering packets using the new streering rule.
+ Drivers modifying the steering rule at a high rate (e.g.
+ adaptively in response to changes in the workload) are recommended to complete
+ processing of the receive queue(s) utilized by the original steering before
+ processing any packets delivered by the modified steering rule.
+\end_layout
+
+\begin_layout Standard
+
+\change_deleted 1986246365 1346664095
+.
+
+\change_unchanged
\end_layout
@@ -5973,8 +6413,7 @@ If the VIRTIO_CONSOLE_F_MULTIPORT feature is negotiated, the driver can
spawn multiple ports, not all of which may be attached to a console.
Some could be generic ports.
In this case, the control virtqueues are enabled and according to the max_nr_po
-rts configuration-space value, an appropriate number of virtqueues are
- created.
+rts configuration-space value, an appropriate number of virtqueues are created.
A control message indicating the driver is ready is sent to the host.
The host can then send control messages for adding new ports to the device.
After creating and initializing each port, a VIRTIO_CONSOLE_PORT_READY
--
MST
^ permalink raw reply related
* Re: [PATCHv3] virtio-spec: virtio network device multiqueue support
From: Michael S. Tsirkin @ 2012-09-09 12:40 UTC (permalink / raw)
To: Sasha Levin; +Cc: netdev, kvm, virtualization
In-Reply-To: <50493D04.1090408@gmail.com>
On Fri, Sep 07, 2012 at 02:17:08AM +0200, Sasha Levin wrote:
> Hi Michael,
>
> On 09/06/2012 02:08 PM, Michael S. Tsirkin wrote:
> > Add multiqueue support to virtio network device.
> > Add a new feature flag VIRTIO_NET_F_MULTIQUEUE for this feature, a new
> > configuration field max_virtqueue_pairs to detect supported number of
> > virtqueues as well as a new command VIRTIO_NET_CTRL_STEERING to program
> > packet steering.
> >
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>
> Some comments about the change:
>
> - "The following four read-only fields only exists if VIRTIO_NET_F_MULTIQUEUE
> is set." => Should be "exist" (I think).
>
> - "When rule is set to VIRTIO_NET_CTRL_STEERING_RX_FOLLOWS_TX packets are
> steered by driver to the first (param+1) multiqueue virtqueues
> transmitq1...transmitqN;" - Why param+1? I thought we ignore the default
> transmit/receive in this case.
>
> - "As selecting a specific steering ais n optimization feature" - "is an".
>
> - It's mentioned several times that the ability to read the steering rule from
> the virtio-net config is there for debug reasons. Is it really necessary? I
> think it's the first time I see debug features go in as part of the spec.
Yes, let features -> less stuff to debug. I'll drop it.
> - I'm slightly confused, why are there both receive and transmit steering? I
> can't find a difference in the way to configure the rule for transmit and
> receive.
This paragraph is there to address this:
Driver selects an active steering rule using VIRTIO_NET_CTRL_STEERING
command (this controls both which virtqueue is selected for a given
packet for receive and notifies the device which virtqueues are about to
be used for transmit).
How can I clarify this better?
> Is it a plan for the future to allow different rules for tx and rx? If
> so, shouldn't we use different ctrl commands (
> VIRTIO_NET_CTRL_TX_STEERING/VIRTIO_NET_CTRL_RX_STEERING)?
I don't see separate steering as very useful:
it does not work for RX follows TX or for TX follows
RX, and separate commands imediately create lots of
options with behaviour which hard to define.
For example if you configure SINGLE on TX but RX_FOLLOWS_TX
on RX what does it mean?
> - "When rule is set to VIRTIO_NET_CTRL_STEERING_SINGLE all packets are steered
> to the default virtqueue receveq (0);" - "receiveq (0)"
>
>
>
> Thanks,
> Sasha
^ permalink raw reply
* [PATCH] virtio-spec: serial: english tweak
From: Michael S. Tsirkin @ 2012-09-09 12:11 UTC (permalink / raw)
To: virtualization, Amit Shah
"A number of virtqueues are created" seems clearer
than "the number of virtqueues": it's
virtqueues that are created not the number.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
I'm not a native english speaker but the below
seems correct to me. Rusty?
virtio-spec.lyx | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/virtio-spec.lyx b/virtio-spec.lyx
index d9626a9..a29315c 100644
--- a/virtio-spec.lyx
+++ b/virtio-spec.lyx
@@ -6406,7 +6406,7 @@ If the VIRTIO_CONSOLE_F_MULTIPORT feature is negotiated, the driver can
spawn multiple ports, not all of which may be attached to a console.
Some could be generic ports.
In this case, the control virtqueues are enabled and according to the max_nr_po
-rts configuration-space value, the appropriate number of virtqueues are
+rts configuration-space value, an appropriate number of virtqueues are
created.
A control message indicating the driver is ready is sent to the host.
The host can then send control messages for adding new ports to the device.
--
MST
^ permalink raw reply related
* [PATCH] virtio-spec: balloon: MUST_TELL_HOST is optional
From: Michael S. Tsirkin @ 2012-09-09 11:36 UTC (permalink / raw)
To: virtualization, pbonzini
Drivers treat MUST_TELL_HOST as optional: windows drivers do not ack it
and expect this means they can tell host *after* deflate. This was not
the intent but the documentation was not very clear on this point.
Luckily hyprevisors did not implement this feature yet so to provide
guidance for future devices make spec match drivers expectations, and
clarify that this feature only has effect if negotiated.
This is needed as recent developments in kvm (memory
region rework and the addition of vfio) finally make
it practical to implement this in devices.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
OK how about this minimal patch for starters?
Other changes are possible on top but I think
both me and Paolo agree on this one.
Paolo?
virtio-spec.lyx | 20 +++++++++++++++-----
1 file changed, 15 insertions(+), 5 deletions(-)
diff --git a/virtio-spec.lyx b/virtio-spec.lyx
index d801aa6..d9626a9 100644
--- a/virtio-spec.lyx
+++ b/virtio-spec.lyx
@@ -6331,8 +6331,13 @@ VIRTIO_CONSOLE_F_SIZE
\begin_layout Description
VIRTIO_CONSOLE_F_MULTIPORT(1) Device has support for multiple ports; configurati
-on fields nr_ports and max_nr_ports are valid and control virtqueues will
- be used.
+on fields nr_ports and max_nr_ports are valid
+\change_inserted 1986246365 1347188404
+; if this bit is negotiated,
+\change_deleted 1986246365 1347188406
+ and
+\change_unchanged
+ control virtqueues will be used.
\end_layout
\end_deeper
@@ -6840,9 +6845,14 @@ The driver constructs an array of addresses of memory pages it has previously
\end_layout
\begin_layout Enumerate
-If the VIRTIO_BALLOON_F_MUST_TELL_HOST feature is set, the guest may not
- use these requested pages until that descriptor in the deflateq has been
- used by the device.
+If the VIRTIO_BALLOON_F_MUST_TELL_HOST feature is
+\change_inserted 1986246365 1347188540
+negotiated
+\change_deleted 1986246365 1347188542
+set
+\change_unchanged
+, the guest may not use these requested pages until that descriptor in the
+ deflateq has been used by the device.
\end_layout
\begin_layout Enumerate
--
MST
^ permalink raw reply related
* [PATCH] virtio-spec: fix thinko in multiport documentation
From: Michael S. Tsirkin @ 2012-09-09 11:08 UTC (permalink / raw)
To: Amit Shah, virtualization
spec numbers ports port0, port 1 etc, then goes on to say
"ports 2 onwards only if MULTIPORT is set".
An obvious thinko, should be port 1 onwards.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
virtio-spec.lyx | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/virtio-spec.lyx b/virtio-spec.lyx
index a713807..d801aa6 100644
--- a/virtio-spec.lyx
+++ b/virtio-spec.lyx
@@ -6298,7 +6298,13 @@ Virtqueues 0:receiveq(port0).
status open
\begin_layout Plain Layout
-Ports 2 onwards only if VIRTIO_CONSOLE_F_MULTIPORT is set
+Ports
+\change_inserted 1986246365 1347188327
+1
+\change_deleted 1986246365 1347188327
+2
+\change_unchanged
+ onwards only if VIRTIO_CONSOLE_F_MULTIPORT is set
\end_layout
\end_inset
--
MST
^ permalink raw reply related
* Re: [PATCH 4/5] virtio-scsi: Add start/stop functionality for vhost-scsi
From: Michael S. Tsirkin @ 2012-09-08 22:40 UTC (permalink / raw)
To: Paolo Bonzini
Cc: Stefan Hajnoczi, kvm-devel, Jan Kiszka, qemu-devel, lf-virt,
Anthony Liguori, target-devel, Zhi Yong Wu, Christoph Hellwig
In-Reply-To: <504A1A32.5050705@redhat.com>
On Fri, Sep 07, 2012 at 06:00:50PM +0200, Paolo Bonzini wrote:
> Il 07/09/2012 08:48, Nicholas A. Bellinger ha scritto:
> > Cc: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> > Cc: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
> > Cc: Michael S. Tsirkin <mst@redhat.com>
> > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
> > ---
> > hw/virtio-pci.c | 2 ++
> > hw/virtio-scsi.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
> > hw/virtio-scsi.h | 1 +
> > 3 files changed, 52 insertions(+), 0 deletions(-)
>
> Please create a completely separate device vhost-scsi-pci instead (or
> virtio-scsi-tcm-pci, or something like that). It is used completely
> differently from virtio-scsi-pci, it does not make sense to conflate the
> two.
>
> Paolo
Ideally the name would say how it is different, not what backend it
uses. Any good suggestions?
--
MST
^ permalink raw reply
* Re: [PATCH] virtio-balloon spec: provide a version of the "silent deflate" feature that works
From: Michael S. Tsirkin @ 2012-09-08 22:37 UTC (permalink / raw)
To: Rusty Russell
Cc: fes, aarcange, riel, kvm, yvugenfi, linux-kernel, mikew, yinghan,
Paolo Bonzini, virtualization
In-Reply-To: <87sjatf6iv.fsf@rustcorp.com.au>
On Sat, Sep 08, 2012 at 02:36:00PM +0930, Rusty Russell wrote:
> "Michael S. Tsirkin" <mst@redhat.com> writes:
> > On Fri, Sep 07, 2012 at 04:09:50PM +0930, Rusty Russell wrote:
> >> > So it looks like a bug: we should teach driver to tell host first on leak?
> >> > Yan, Vadim, can you comment please?
> >> >
> >> > Also if true, looks like this bit will be useful to detect a fixed driver on
> >> > the hypervisor side - to avoid unmapping such pages? Rusty what do you
> >> > think?
> >>
> >> So, feature is unimplemented in qemu, and broken in drivers. I starting
> >> to share Paolo's dislike of it.
> >
> > What is broken in drivers?
>
> Because supporting the feature is *not* optional for a driver.
>
> If the device said MUST_TELL_HOST, it meant that the driver *had* to
> tell the host before it touched the page, otherwise Bad Things might
> happen. It was in the original spec precisely to allow devices to
> actually *remove* pages.
>
> Noone ever noticed the windows driver didn't support it, because qemu
> never requires MUST_TELL_HOST.
>
> So in practice, it's now an optional feature. Since no device used it
> anyway, we're better off discarding it than trying to fix it.
I trust you this was not the intent. But it seems to be
the intent in spec, because almost all features are optional.
And so windows driver authors interpreted it
this way. And it is *useful* like this. See below.
> If someone wants an *optional* "tell me first" feature later, that's
> easy to add, but I don't see why they'd want to.
Suggested use is for device assignment which needs all guest memory
locked. hypervisor can unlock pages in balloon but guest must wait for
hypervisor to lock them back before use.
when a hypervisor implements this,
this will work with linux guests but not windows
guests and the existing bit fits exactly the purpose.
> > Do we really know there are no hypervisors implementing it?
>
> As much as can be known. Qemu doesn't, lkvm doesn't.
But we can add it in qemu and it will be a useful feature.
> > As I said above drivers do have support.
>
> Not the windows drivers. So it's optional, thus removing it will likely
> harm noone.
>
> Cheers,
> Rusty.
I think the issue is that kvm always locked all guest memory
for assignment. This restriction is removed
with vfio which has separate page tables.
Now that vfio is upstream and work on qemu integration
is being worked on, we might finally see people using this bit
to allow memory overcommit with device assignment.
So let's not remove yet, there is no rush.
Let's document it that this feature is optional,
maybe renaming as Paolo suggests.
--
MST
^ permalink raw reply
* Re: [PATCH] virtio-balloon spec: provide a version of the "silent deflate" feature that works
From: Michael S. Tsirkin @ 2012-09-08 22:22 UTC (permalink / raw)
To: Paolo Bonzini
Cc: fes, aarcange, riel, yvugenfi, kvm, linux-kernel, mikew, yinghan,
virtualization
In-Reply-To: <504A0858.4080508@redhat.com>
On Fri, Sep 07, 2012 at 04:44:40PM +0200, Paolo Bonzini wrote:
> Il 07/09/2012 16:25, Michael S. Tsirkin ha scritto:
> >> > Silent deflate is deflating just by using the page, without using the
> >> > deflateq at all. So it can be done even from GFP_ATOMIC context.
> > BTW I don't see a need to avoid deflateq.
> > Without MUST_TELL_HOST you can just deflate and then
> > bounce telling the host out to a thread.
>
> Yes, that's fine too.
OK so it's preferable: will work on existing hypervisors.
> >> > But knowing that the guest will _not_ deflate silently also benefits the
> >> > host. This is the cause of unpinning ballooned pages and pinning them
> >> > again upon deflation. This allows cooperative memory overcommit even if
> >> > the guests' memory is pinned, similar to what Xen did before it
> >> > supported overcommit. This is the second feature bit.
> > This is the MUST_TELL_HOST.
>
> Almost. One is "the guest, if really needed, can tell the host of
> pages". If not negotiated, and the host does not support it, the host
> must break the guest (e.g. fail to offer any virtqueues).
There is no way in spec to break the guest.
You can not fail to offer virtqueues.
Besides, there is no guarantee that virtqueue setup
happens after feature negotiation.
> The other is "the guest, though, would prefer not to do so". It is
> different because the guest can proceed in a fallback mode even if the
> host doesn't offer it.
I think I get what your proposed SILENT means what I do not get
is the motivation. It looks like a premature optimization to me.
> You're interpreting features as something that dictates behavior, but
> they're really just advisory.
The spec is pretty clear that if guest acks feature it
is a contract that dictates behaviour.
If it doesn't it is either ignored or just informative
depending on feature.
> You could negotiate VIRTIO_BLK_F_TOPOLOGY
> and end up never reading the fields; you could negotiate
> VIRTIO_NET_F_GUEST_ANNOUNCE and never send a guest announcement.
Block example is just informative. It does not need to be
negotiated even to be used.
But last example is wrong.
If you ack GUEST_ANNOUNCE hypervisor assumes guest will
announce self, if guest does not do it this break migration.
> >> > * guest will always do silent deflation: current Windows driver.
> > Not exactly. It is not silent. It tells host
> > just after use.
>
> Yeah, but no difference from the POV of the driver.
>
> Delaying or avoiding is the same in the end. The spec says it well: "In
> this case, deflation advice is merely a courtesy".
So it looks like we don't need a new bit to leak in atomic ctx.
Just do not ack MUST_TELL_HOST and delay telling host to a wq.
IMO we should not add random stuff to spec like this just because it
seems like a good idea.
> >> > Negotiates nothing, or if it cares it can negotiate
> >> > VIRTIO_BALLOON_F_SILENT_DEFLATE. Host mustn't do munlock/mlock dance.
> >> >
> >> > * guest may do silent deflation if available: combo of Linux driver +
> >> > Frank's driver. Negotiates both features, looks at
> >> > VIRTIO_BALLOON_F_SILENT_DEFLATE host feature to decide how to behave:
> >> >
> >> > ** If PCI passthrough, the host will not negotiate
> >> > VIRTIO_BALLOON_F_SILENT_DEFLATE. The driver will behave as the
> >> > current Linux driver, the host can do the munlock/mlock dance.
> > So this case is just existing interface. OK.
> >
> >> > ** If no PCI passthrough, the host will negotiate
> >> > VIRTIO_BALLOON_F_SILENT_DEFLATE, and the driver can balloon more
> >> > aggressively and not use the deflateq.
> >> >
> > This last trickery confuses me. It just does not make sense to set both
> > SILENT and TELL: either you are silent or you tell.
>
> "I can be chatty if you ask me, but I'd rather be silent if you let me".
>
> TELL is a request of the host to the guest; the guest can agree or not.
>
> SILENT is a request of the guest to the host; the host can agree or not.
OK so TELL says *when* to notify host, SILENT if set allows guest
to skip leak notifications? In this case TELL should just be ignored
when SILENT is set. Otherwise it's a reasonable extension.
But I am still not sure *why* do we need SILENT - any data showing
that avoiding these notifications is a win?
Let's not add new features until we see an actual use.
> > It seems cleaner for the host to know the state.
>
> Yeah, if you want to do it you can.
>
> Paolo
IMHO, renaming is fine since there is confusion.
But WILL_TELL is also not all that clear imho.
I think the confusion is that TELL_HOST seems to
imply we can avoid telling host at all.
How about being explicit?
VIRTIO_BALLOON_F_HOST_ACK_BEFORE_DEFLATE
Hmm?
"MUST" "WILL" are not really informative.
Also some confusion I think is from spec text saying "feature is set"
in many cases where it really almost always means "feature is negotiated".
Let's address?
--
MSt
^ permalink raw reply
* Re: [PATCH] virtio-balloon spec: provide a version of the "silent deflate" feature that works
From: Paolo Bonzini @ 2012-09-08 10:32 UTC (permalink / raw)
To: Rusty Russell
Cc: fes, aarcange, riel, kvm, Michael S. Tsirkin, yvugenfi,
linux-kernel, mikew, yinghan, virtualization
In-Reply-To: <87sjatf6iv.fsf@rustcorp.com.au>
Il 08/09/2012 07:06, Rusty Russell ha scritto:
>> > Do we really know there are no hypervisors implementing it?
> As much as can be known. Qemu doesn't, lkvm doesn't.
>
However, there are cases in which it would be nice to have it.
Repurposing the bit to how it has been used so far (as a guest->host
communication bit, not host->guest) is a better choice.
Paolo
^ permalink raw reply
* [patch 3/3] xen/privcmd: remove const modifier from declaration
From: Dan Carpenter @ 2012-09-08 9:58 UTC (permalink / raw)
To: Andres Lagar-Cavilla
Cc: kernel-janitors, Jeremy Fitzhardinge, xen-devel, virtualization,
Konrad Rzeszutek Wilk
When we use this pointer, we cast away the const modifier and modify the
data. I think it was an accident to declare it as const.
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
diff --git a/include/xen/privcmd.h b/include/xen/privcmd.h
index a853168..58ed953 100644
--- a/include/xen/privcmd.h
+++ b/include/xen/privcmd.h
@@ -69,7 +69,7 @@ struct privcmd_mmapbatch_v2 {
unsigned int num; /* number of pages to populate */
domid_t dom; /* target domain */
__u64 addr; /* virtual address */
- const xen_pfn_t __user *arr; /* array of mfns */
+ xen_pfn_t __user *arr; /* array of mfns */
int __user *err; /* array of error codes */
};
diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
index 0ce006a..fceb83e 100644
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -389,7 +389,7 @@ static long privcmd_ioctl_mmap_batch(void __user *udata, int version)
if (state.global_error && (version == 1)) {
/* Write back errors in second pass. */
- state.user_mfn = (xen_pfn_t *)m.arr;
+ state.user_mfn = m.arr;
state.err = err_array;
ret = traverse_pages(m.num, sizeof(xen_pfn_t),
&pagelist, mmap_return_errors_v1, &state);
^ permalink raw reply related
* [patch 2/3] xen/privcmd: return -EFAULT on error
From: Dan Carpenter @ 2012-09-08 9:57 UTC (permalink / raw)
To: Andres Lagar-Cavilla
Cc: kernel-janitors, Jeremy Fitzhardinge, xen-devel, virtualization,
Konrad Rzeszutek Wilk
__copy_to_user() returns the number of bytes remaining to be copied but
we want to return a negative error code here.
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
index fdff8f9..0ce006a 100644
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -393,8 +393,11 @@ static long privcmd_ioctl_mmap_batch(void __user *udata, int version)
state.err = err_array;
ret = traverse_pages(m.num, sizeof(xen_pfn_t),
&pagelist, mmap_return_errors_v1, &state);
- } else if (version == 2)
+ } else if (version == 2) {
ret = __copy_to_user(m.err, err_array, m.num * sizeof(int));
+ if (ret)
+ ret = -EFAULT;
+ }
/* If we have not had any EFAULT-like global errors then set the global
* error to -ENOENT if necessary. */
^ permalink raw reply related
* [patch 1/3] xen/privcmd: check for integer overflow in ioctl
From: Dan Carpenter @ 2012-09-08 9:52 UTC (permalink / raw)
To: Andres Lagar-Cavilla
Cc: kernel-janitors, Jeremy Fitzhardinge, xen-devel, virtualization,
Konrad Rzeszutek Wilk
If m.num is too large then the "m.num * sizeof(*m.arr)" multiplication
could overflow and the access_ok() check wouldn't test the right size.
Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
---
Only needed in linux-next.
diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c
index 215a3c0..fdff8f9 100644
--- a/drivers/xen/privcmd.c
+++ b/drivers/xen/privcmd.c
@@ -325,6 +325,8 @@ static long privcmd_ioctl_mmap_batch(void __user *udata, int version)
return -EFAULT;
/* Returns per-frame error in m.arr. */
m.err = NULL;
+ if (m.num > SIZE_MAX / sizeof(*m.arr))
+ return -EINVAL;
if (!access_ok(VERIFY_WRITE, m.arr, m.num * sizeof(*m.arr)))
return -EFAULT;
break;
@@ -332,6 +334,8 @@ static long privcmd_ioctl_mmap_batch(void __user *udata, int version)
if (copy_from_user(&m, udata, sizeof(struct privcmd_mmapbatch_v2)))
return -EFAULT;
/* Returns per-frame error code in m.err. */
+ if (m.num > SIZE_MAX / sizeof(*m.err))
+ return -EINVAL;
if (!access_ok(VERIFY_WRITE, m.err, m.num * (sizeof(*m.err))))
return -EFAULT;
break;
^ permalink raw reply related
* Re: [PATCH] virtio-balloon spec: provide a version of the "silent deflate" feature that works
From: Rusty Russell @ 2012-09-08 5:06 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: fes, aarcange, riel, kvm, yvugenfi, linux-kernel, mikew, yinghan,
Paolo Bonzini, virtualization
In-Reply-To: <20120907104306.GA17211@redhat.com>
"Michael S. Tsirkin" <mst@redhat.com> writes:
> On Fri, Sep 07, 2012 at 04:09:50PM +0930, Rusty Russell wrote:
>> > So it looks like a bug: we should teach driver to tell host first on leak?
>> > Yan, Vadim, can you comment please?
>> >
>> > Also if true, looks like this bit will be useful to detect a fixed driver on
>> > the hypervisor side - to avoid unmapping such pages? Rusty what do you
>> > think?
>>
>> So, feature is unimplemented in qemu, and broken in drivers. I starting
>> to share Paolo's dislike of it.
>
> What is broken in drivers?
Because supporting the feature is *not* optional for a driver.
If the device said MUST_TELL_HOST, it meant that the driver *had* to
tell the host before it touched the page, otherwise Bad Things might
happen. It was in the original spec precisely to allow devices to
actually *remove* pages.
Noone ever noticed the windows driver didn't support it, because qemu
never requires MUST_TELL_HOST.
So in practice, it's now an optional feature. Since no device used it
anyway, we're better off discarding it than trying to fix it.
If someone wants an *optional* "tell me first" feature later, that's
easy to add, but I don't see why they'd want to.
> Do we really know there are no hypervisors implementing it?
As much as can be known. Qemu doesn't, lkvm doesn't.
> As I said above drivers do have support.
Not the windows drivers. So it's optional, thus removing it will likely
harm noone.
Cheers,
Rusty.
^ permalink raw reply
* Call for Participation: The 9th ICAC 2012 Conference - September 17-21 in San Jose CA
From: Ioan Raicu @ 2012-09-08 0:10 UTC (permalink / raw)
To: virtualization, micro_publicity
**********************************************************************
CALL FOR PARTICIPATION
======================
The 9th ACM International Conference on Autonomic Computing (ICAC 2012)
San Jose, California, USA
September 17-21, 2012
http://icac2012.cs.fiu.edu
Sponsored by ACM
**********************************************************************
Online registration is open at
http://icac2012.cs.fiu.edu/registration.shtm
Reduced fees are available for those registering by September 9, 2012
(extended).
This year's technical program features:
- 3 distinguished keynote speakers:
* Dr. Amin Vahdat, Google/UCSD
* Dr. Subutai Ahmad (VP Engineering), Numenta
* Dr. Eitan Frachtenberg, Facebook
- 24 outstanding technical papers (15 full + 9 short):
* covering core and emerging topics such as clouds, virtualization,
control, monitoring and diagnosis, and energy
* Half of the papers involve authors from industry or government labs
- 4 co-located workshops covering hot topics in:
* Feedback Computing
* Self-Aware Internet of Things
* Management of Big Data Systems
* Federated Clouds
The Conference will be held at the Fairmont Hotel in downtown San Jose,
CA, USA.
**********************************************************************
IMPORTANT DATES
===============
Early registration deadline: September 9, 2012 (extended)
Hotel special rate deadline: September 7, 2012
**********************************************************************
CORPORATE SPONSORS
==================
Gold Level Partner: IBM
Conference partners: VMware, HP, Neustar
PhD Student Sponsor: Google
Other Sponsor: NEC Labs, Microsoft
**********************************************************************
PROGRAM
=======
======================================================================
MONDAY, SEPTEMBER 17, 2012 - WORKSHOPS
Feedback Computing 2012
Self-Aware Internet of Things 2012
======================================================================
TUESDAY, SEPTEMBER 18, 2012 - MAIN CONFERENCE
8:00AM – 8:45AM
Registration
8:45AM – 9:00AM
Welcome Remarks
9:00AM – 10:00AM
Keynote Talk I
Symbiosis in Scale Out Networking and Data Management
Amin Vahdat, Google and University of California, San Diego
10:00AM – 10.30AM
Break
10:30AM – 12:00PM
Session : Virtualization
Net-Cohort: Detecting and Managing VM Ensembles in Virtualized Data Centers
Liting Hu, Karsten Schwan (Georgia Institute of Technology);
Ajay Gulati (VMware); Junjie Zhang, Chengwei Wang (Georgia Institute
of Technology)
Application-aware Cross-layer Virtual Machine Resource Management
Lixi Wang, Jing Xu, Ming Zhao (Florida International University)
Shifting GEARS to Enable Guest-context Virtual Services
Kyle Hale, Lei Xia, Peter Dinda (Northwestern University)
12:00PM-1:30PM
Lunch
----------------------------------------------------------------------
1:30PM - 3:30PM
Session: Performance and Resource Management
When Average is Not Average: Large Response Time Fluctuations in N-tier
Systems
Qingyang Wang (Georgia Institute of Technology); Yasuhiko Kanemasa,
Motoyuki Kawaba (Fujitsu Laboratories Ltd.); Calton Pu (Georgia Institute
of Technology)
Provisioning Multi-tier Cloud Applications Using Statistical Bounds on
Sojourn Time
Upendra Sharma, Prashant Shenoy, Don Towsley (University of
Massachusetts Amherst)
Automated Profiling and Resource Management of Pig Programs for Meeting
Service Level Objectives
Zhuoyao Zhang (University of Pennsylvania); Ludmila Cherkasova
(Hewlett-Packard Labs);
Abhishek Verma (University of Illinois at Urbana-Champaign);
Boon Thau Loo (University of Pennsylvania)
AROMA: Automated Resource Allocation and Configuration of MapReduce
Environment
in the Cloud
Palden Lama, Xiaobo Zhou (University of Colorado at Colorado Springs)
3:30PM-4:00PM
Break
4:00PM – 5:15PM
Short Papers I
Locomotion@Location: When the Rubber hits the Road
Gerold Hoelzl, Marc Kurz, Alois Ferscha (Johannes Kepler University
Linz, Austria)
An Autonomic Resource Provisioning Framework for Mobile Computing Grids
Hariharasudhan Viswanathan, Eun Kyung Lee, Ivan Rodero, Dario Pompili
(Rutgers University)
A Self-Tuning Self-Optimizing Approach for Automated Network Anomaly
Detection Systems
Dennis Ippoliti, Xiaobo Zhou (University of Colorado at Colorado Springs)
Offline and On-Demand Event Correlation for Operations Management of
Large Scale IT Systems
Chetan Gupta (Hewlett-Packard Labs)
PowerTracer: Tracing Requests in Multi-tier Services to Diagnose Energy
Inefficiency
Gang Lu, Jianfeng Zhan (Institute of Computing Technology, Chinese
Academy of Sciences);
Haining Wang (College of William and Mary); Lin Yuan (Institute of
Computing Technology,
Chinese Academy of Sciences); Chuliang Weng (Shanghai Jiao Tong
University, China)
6:00PM - 9:00PM
Conference Dinner
======================================================================
WEDNESDAY, SEPTEMBER 19 - MAIN CONFERENCE
8:00AM – 9:00AM
Registration
9:00AM – 10:00AM
Keynote Talk II
Automated Machine Learning For Autonomic Computing
Subutai Ahmad, VP Engineering, Numenta
10:00AM - 10:30AM
Break
10:30AM – 12:00PM
Session: Control-Based Approaches
Budget-based Control for Interactive Services with Adaptive Execution
Yuxiong He, Zihao Ye, Qiang Fu, Sameh Elnikety (Microsoft Research)
On the Design of Decentralized Control Architectures for Workload
Consolidation in
Large-Scale Server Clusters
Rui Wang, Nagarajan Kandasamy (Drexel University)
Transactional Auto Scaler: Elastic Scaling of In-Memory Transactional
Data Grids
Diego Didona, Paolo Romano (Instituto Superior Técnico/INESC-ID);
Sebastiano Peluso,
Francesco Quaglia (Sapienza, Università di Roma)
12:00PM - 1:30PM
Lunch
----------------------------------------------------------------------
1:30PM - 2:30PM
Session: Energy
Adaptive Green Hosting
Nan Deng, Christopher Stewart, Jaimie Kelley (The Ohio State
University); Daniel Gmach,
Martin Arlitt (Hewlett Packard Labs)
Dynamic Energy-Aware Capacity Provisioning for Cloud Computing Environments
Qi Zhang, Mohamed Faten Zhani (University of Waterloo); Shuo Zhang
(National University
of Defense Technology); Quanyan Zhu (University of Illinois at
Urbana-Champaign);
Raouf Boutaba (University of Waterloo); Joseph L. Hellerstein (Google,
Inc.)
2:30PM - 3:30PM
Short Papers II
VESPA: Multi-Layered Self-Protection for Cloud Resources
Aurélien Wailly, Marc Lacoste (Orange Labs); Hervé Debar (Télécom SudParis)
Usage Patterns in Multi-tenant Data Centers: a Temporal Perspective
Robert Birke, Lydia Y. Chen (IBM Research Zurich Lab); Evgenia Smirni
(College of
William and Mary)
Toward Fast Eventual Consistency with Performance Guarantees
Feng Yan (College of William and Mary); Alma Riska (EMC Corporation);
Evgenia Smirni (College
of William and Mary)
Optimal Autoscaling in the IaaS Cloud
Hamoun Ghanbari, Bradley Simmons, Marin Litoiu, Cornel Barna (York
University);
Gabriel Iszlai (IBM Toronto)
3:30PM – 4:00PM
Break
4:00PM - 6:00PM
Poster and Demo Session
6:00PM-9:00PM
Conference Outing (tentative)
======================================================================
THURSDAY, SEPTEMBER 20, 2012 - MAIN CONFERENCE
8:00AM – 9:00AM
Registration
9:00AM – 10:00AM
Keynote Talk III
High Efficiency at Web Scale
Eitan Frachtenberg, Facebook
10:00AM-10:30AM
Break
10:30AM - 12:00PM
Session: Diagnosis and Monitoring
Chair: TBD
3-Dimensional Root Cause Diagnosis via Co-analysi
Ziming Zheng, Li Yu, Zhiling Lan (Illinois Institute of Technology);
Terry Jones
(Oak Ridge National Laboratory)
UBL: Unsupervised Behavior Learning for Predicting Performance Anomalies
in Virtualized Cloud Systems
Daniel J. Dean, Hiep Nguyen, Xiaohui Gu (North Carolina State University)
Evaluating Compressive Sampling Strategies for Performance Monitoring of
Data Centers
Tingshan Huang, Nagarajan Kandasamy, Harish Sethu (Drexel University)
12:00PM
Adjourn
======================================================================
FRIDAY, SEPTEMBER 21, 2012 - WORKSHOPS
Management of Big Data Systems 2012
Federated Clouds 2012
**********************************************************************
ORGANIZERS
==========
GENERAL CHAIR
Dejan Milojicic, HP Labs
PROGRAM CHAIRS
Dongyan Xu, Purdue University
Vanish Talwar, HP Labs
INDUSTRY CHAIR
Xiaoyun Zhu, VMware
WORKSHOPS CHAIR
Fred Douglis, EMC
POSTERS/DEMO/EXHIBITS CHAIR
Eno Thereska, Microsoft Research
FINANCE CHAIR
Michael Kozuch, Intel
LOCAL ARRANGEMENT CHAIR
Jessica Blaine
PUBLICITY CHAIRS
Daniel Batista, University of São Paulo
Vartan Padaryan, ISP/Russian Academy of Sci.
Ioan Raicu, Illinois Inst. of Technology
Jianfeng Zhan, ICT/Chinese Academy of Sci.
Ming Zhao, Florida Intl. University
PROGRAM COMMITTEE
Tarek Abdelzaher, UIUC
Umesh Bellur, IIT, Bombay
Ken Birman, Cornell University
Rajkumar Buyya, Univ. of Melbourne
Rocky Chang, Hong Kong Polytechnic University
Yuan Chen, HP Labs
Alva Couch, Tufts University
Peter Dinda, Northwestern University
Fred Douglis, EMC
Renato Figueiredo, University of Florida
Mohamed Hefeeda, QCRI
Joe Hellerstein, Google
Geoff Jiang, NEC Labs
Jeff Kephart, IBM Research
Emre Kiciman, Microsoft Research
Fabio Kon, University of São Paulo
Mike Kozuch, Intel Labs
Dejan Milojicic, HP Labs
Klara Nahrstedt, UIUC
Priya Narasimhan, CMU
Manish Parashar, Rutgers University
Ioan Raicu, Illinois Inst. of Technology
Omer Rana, Cardiff University
Masoud Sadjadi, Florida Intl. University
Richard Schlichting, AT&T Labs
Hartmut Schmeck, KIT
Karsten Schwan, Georgia Tech
Onn Shehory, IBM Research
Eno Thereska, Microsoft Research
Xiaoyun Zhu, VMware
**********************************************************************
--
=================================================================
Ioan Raicu, Ph.D.
Assistant Professor, Illinois Institute of Technology (IIT)
Guest Research Faculty, Argonne National Laboratory (ANL)
=================================================================
Data-Intensive Distributed Systems Laboratory, CS/IIT
Distributed Systems Laboratory, MCS/ANL
=================================================================
Cel: 1-847-722-0876
Office: 1-312-567-5704
Email: iraicu@cs.iit.edu
Web: http://www.cs.iit.edu/~iraicu/
Web: http://datasys.cs.iit.edu/
=================================================================
=================================================================
^ permalink raw reply
* eScience 2012 Program now available
From: Ioan Raicu @ 2012-09-07 22:13 UTC (permalink / raw)
To: virtualization, micro_publicity
[-- Attachment #1.1: Type: text/plain, Size: 2176 bytes --]
The Conference Program for the 8th IEEE International Conference on
eScience (eScience 2012) is now available at
http://www.ci.uchicago.edu/escience2012/.
All eScience conference and workshops will be held at the Hyatt Regency
Chicago. We have reserved a block of rooms at the hotel with a room
rate of $205 per night + tax. Please make your reservations before
September 16 at https://resweb.passkey.com/go/ESCIENCE2012 or by phone
at +1-800-233-1234 from the US or +1-312-565-1234 from outside the US.
Reference "e-Science" when making reservations by phone.
The hotel is completely sold out on Sunday night (10/7) due to major
events being held in Chicago over the weekend. We have secured a block
of rooms at the Hyatt Regency O'Hare for $190 + tax and will provide
transportation to the conference hotel on Monday morning. You will be
able to make your reservation at the Hyatt O'Hare upon registration for
the conference or via email to kristih@ci.uchicago.edu
<mailto:kristih@ci.uchicago.edu>.
Advance registration is available until September 28 - see
http://www.ci.uchicago.edu/escience2012/registration.php.
ACKNOWLEDGEMENTS--
We gratefully acknowledge the support of the following organizations--
Gold Level:
Microsoft Research
Silver Level:
CSIRO Australia
Bronze Level:
Argonne National Laboratory
EMC
Indiana University
Media:
HPCwire
Also:
The Computation Institute
Cray
Conference Sponsors:
IEEE
The IEEE Computer Society
--
=================================================================
Ioan Raicu, Ph.D.
Assistant Professor, Illinois Institute of Technology (IIT)
Guest Research Faculty, Argonne National Laboratory (ANL)
=================================================================
Data-Intensive Distributed Systems Laboratory, CS/IIT
Distributed Systems Laboratory, MCS/ANL
=================================================================
Cel: 1-847-722-0876
Office: 1-312-567-5704
Email: iraicu@cs.iit.edu
Web: http://www.cs.iit.edu/~iraicu/
Web: http://datasys.cs.iit.edu/
=================================================================
=================================================================
[-- Attachment #1.2: Type: text/html, Size: 3273 bytes --]
[-- Attachment #2: Type: text/plain, Size: 183 bytes --]
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [PATCH 4/5] virtio-scsi: Add start/stop functionality for vhost-scsi
From: Paolo Bonzini @ 2012-09-07 20:30 UTC (permalink / raw)
To: Nicholas A. Bellinger
Cc: Stefan Hajnoczi, kvm-devel, Michael S. Tsirkin, Jan Kiszka,
qemu-devel, Zhi Yong Wu, Anthony Liguori, target-devel, lf-virt,
Christoph Hellwig
In-Reply-To: <1347045783.4162.651.camel@haakon2.linux-iscsi.org>
Il 07/09/2012 21:23, Nicholas A. Bellinger ha scritto:
>> > Please create a completely separate device vhost-scsi-pci instead (or
>> > virtio-scsi-tcm-pci, or something like that). It is used completely
>> > differently from virtio-scsi-pci, it does not make sense to conflate the
>> > two.
>> >
> Ok, I need to figure out what this will involve over the next days, and
> will likely have some more questions for you to get a standlone
> vhost-scsi-pci up and running.
>
> Also just curious (question for Anthony + QEMU folks), how long can we
> expect the QEMU 1.3 merge window to be open..?
wiki.qemu.org/Planning/1.3 - no hurry, until November 15th.
Paolo
^ permalink raw reply
* Extended deadline: 5th Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS) 2012 -- at IEEE/ACM Supercomputing 2012
From: ZHAO ZHANG @ 2012-09-07 20:25 UTC (permalink / raw)
To: virtualization
[-- Attachment #1.1: Type: text/plain, Size: 9390 bytes --]
Call for Papers
---------------------------------------------------------------------------------------
The 5th Workshop on Many-Task Computing on Grids and Supercomputers (MTAGS) 2012
http://datasys.cs.iit.edu/events/MTAGS12/
---------------------------------------------------------------------------------------
November 12th, 2012
Salt Lake City, Utah, USA
Co-located with with IEEE/ACM International Conference for
High Performance Computing, Networking, Storage and Analysis (SC12)
=======================================================================================
The 5th workshop on Many-Task Computing on Grids and Supercomputers (MTAGS) will
provide the scientific community a dedicated forum for presenting new research,
development, and deployment efforts of large-scale many-task computing (MTC)
applications on large scale clusters, Grids, Supercomputers, and Cloud Computing
infrastructure. MTC, the theme of the workshop encompasses loosely coupled applications,
which are generally composed of many tasks (both independent and dependent tasks) to
achieve some larger application goal. This workshop will cover challenges that can
hamper efficiency and utilization in running applications on large-scale systems, such
as local resource manager scalability and granularity, efficient utilization of raw
hardware, parallel file system contention and scalability, data management, I/O
management, reliability at scale, and application scalability. We welcome paper
submissions on all theoretical, simulations, and systems topics related to MTC, but we
give special consideration to papers addressing petascale to exascale challenges.
Papers will be peer-reviewed, and accepted papers will be published in the workshop
proceedings as part of the IEEE digital library (pending approval). The workshop will
be co-located with the IEEE/ACM Supercomputing 2012 Conference in Salt Lake City Utah
on November 12th, 2012.
For more information, please see http://datasys.cs.iit.edu/events/MTAGS12/. For more
information on past workshops, please see MTAGS11, MTAGS10, MTAGS09, and MTAGS08. We also
ran a Special Issue on Many-Task Computing in the IEEE Transactions on Parallel and
Distributed Systems (TPDS) which has appeared in June 2011; the proceedings can be found
online at http://www.computer.org/portal/web/csdl/abs/trans/td/2011/06/ttd201106toc.htm.
We, the workshop organizers, also published a highly relevant paper that defines
Many-Task Computing which was published in MTAGS08, titled "Many-Task Computing for
Grids and Supercomputers"
(http://www.cs.iit.edu/~iraicu/research/publications/2008_MTAGS08_MTC.pdf); we encourage
potential authors to read this paper.
Topics
---------------------------------------------------------------------------------------
We invite the submission of original work that is related to the topics below. The
papers can be either short (4 pages) position papers, or long (8 pages) research
papers. Topics of interest include (in the context of Many-Task Computing):
* Compute Resource Management
* Scheduling
* Job execution frameworks
* Local resource manager extensions
* Performance evaluation of resource managers in use on large scale systems
* Dynamic resource provisioning
* Techniques to manage many-core resources and/or GPUs
* Challenges and opportunities in running many-task workloads on HPC systems
* Challenges and opportunities in running many-task workloads on Cloud Computing
infrastructure
* Storage architectures and implementations
* Distributed file systems
* Parallel file systems
* Distributed meta-data management
* Content distribution systems for large data
* Data caching frameworks and techniques
* Data management within and across data centers
* Data-aware scheduling
* Data-intensive computing applications
* Eventual-consistency storage usage and management
* Programming models and tools
* Map-reduce and its generalizations
* Many-task computing middleware and applications
* Parallel programming frameworks
* Ensemble MPI techniques and frameworks
* Service-oriented science applications
* Large-Scale Workflow Systems
* Workflow system performance and scalability analysis
* Scalability of workflow systems
* Workflow infrastructure and e-Science middleware
* Programming Paradigms and Models
* Large-Scale Many-Task Applications
* High-throughput computing (HTC) applications
* Data-intensive applications
* Quasi-supercomputing applications, deployments, and experiences
* Performance Evaluation
* Performance evaluation
* Real systems
* Simulations
* Reliability of large systems
* How MTC Addresses Challenges of Petascale and Exascale Computing
* Concurency & Programmability
* I/O & Memory
* Energy
* Resilience
* Heterogeneity
Paper Submission and Publication
---------------------------------------------------------------------------------------
Authors are invited to submit papers with unpublished, original work of not more than 8
pages of double column text using single spaced 10 point size on 8.5 x 11 inch pages,
as per IEEE 8.5 x 11 manuscript guidelines; document templates can be found at
http://www.ieee.org/conferences_events/conferences/publishing/templates.html. We are
also seeking position papers of no more than 4 pages in length. The final 4/8 page
papers in PDF format must be submitted online at
https://cmt.research.microsoft.com/MTAGS2012/ before the deadline. Papers will be
peer-reviewed, and accepted papers will be published in the workshop proceedings as part
of the IEEE digital library (pending approval). Notifications of the paper decisions
will be sent out by October 12th, 2012. Selected excellent work may be eligible for
additional post-conference publication as journal articles or book chapters, such as the
previous Special Issue on Many-Task Computing in the IEEE Transactions on Parallel and
Distributed Systems (TPDS) which has appeared in June 2011. Submission implies the
willingness of at least one of the authors to register and present the paper. For more
information, please http://datasys.cs.iit.edu/events/MTAGS12/, or send email to
mtags12-chairs@datasys.cs.iit.edu.
Important Dates
---------------------------------------------------------------------------------------
* Abstract submission: September 10th, 2012 (11:59PM PST)
* Paper submission: September 24th, 2012 (11:59PM PST)
* Acceptance notification: October 12th, 2012
* Final papers due: November 7th, 2012
Committee Members
---------------------------------------------------------------------------------------
Workshop Chairs (mtags12-chairs@datasys.cs.iit.edu)
* Ioan Raicu, Illinois Institute of Technology & Argonne National Laboratory
* Ian Foster, University of Chicago & Argonne National Laboratory
* Yong Zhao, University of Electronic Science and Technology of China
Steering Committee
* David Abramson, Monash University, Australia
* Jack Dongara, University of Tennessee, USA
* Geoffrey Fox, Indiana University, USA
* Manish Parashar, Rutgers University, USA
* Marc Snir, Argonne National Laboratory & University of Illinois at Urbana Champaign, USA
* Xian-He Sun, Illinois Institute of Technology, USA
* Weimin Zheng, Tsinghua University, China
Publicity Chair (mtags12-publicity@datasys.cs.iit.edu)
* Zhao Zhang, University of Chicago, USA
Program Committee Chair (mtags12-pc-chair@datasys.cs.iit.edu)
* Justin Wozniak, Argonne National Laboratory, USA
Technical Committee
* Roger Barga, Microsoft Research, USA
* Mihai Budiu, Microsoft Research, USA
* Kyle Chard, University of Chicago, USA
* Yong Chen, Texas Tech University, USA
* Evangelinos Constantinos, Massachusetts Institute of Technology, USA
* John Dennis, National Center for Atmospheric Research, USA
* Catalin Dumitrescu, Fermi National Labs, USA
* Dennis Gannon, Microsoft Research, USA
* Indranil Gupta, University of Illinois at Urbana Champaign, USA
* Florin Isaila, Universidad Carlos III de Madrid, Spain
* Kamil Iskra, Argonne National Laboratory, USA
* Alexandru Iosup, Delft University of Technology, Netherlands
* Hui Jin, Oracle Corporation, USA
* Daniel S. Katz, University of Chicago & Argonne National Laboratory, USA
* Carl Kesselman, University of Southern California, USA
* Zhiling Lan, Illinois Institute of Technology, USA
* Mike Lang, Los Alamos National Laboratory, USA
* Gregor von Laszewski, Indiana University, USA
* Reagan Moore, University of North Carolina, Chappel Hill, USA
* Jose Moreira, IBM Research, USA
* Chris Moretti, Princeton University, USA
* David O'Hallaron, Carnegie Mellon University, Intel Labs, USA
* Marlon Pierce, Indiana University, USA
* Judy Qiu, Indiana University, USA
* Lavanya Ramakrishnan, Lawrence Berkeley National Laboratory, USA
* Kui Ren, SUNY Buffalo, USA
* Matei Ripeanu, University of British Columbia, Canada
* Karen Schuchardt, Pacific Northwest National Laboratory, USA
* Wei Tang, Argonne National Laboratory, USA
* Valerie Taylor, Texas A&M, USA
* Douglas Thain University of Notre Dame, USA
* Edward Walker, Whitworth University, USA
* Matthew Woitaszek, Occipital, Inc., USA
* Ken Yocum, University of California, San Diego, USA
* Zhifeng Yun, Louisiana State University, USA
* Zhao Zhang, University of Chicago, USA
* Ziming Zheng, Illinois Institute of Technology, USA
[-- Attachment #1.2: Type: text/html, Size: 11105 bytes --]
[-- Attachment #2: Type: text/plain, Size: 183 bytes --]
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
^ permalink raw reply
* Re: [PATCH 4/5] virtio-scsi: Add start/stop functionality for vhost-scsi
From: Nicholas A. Bellinger @ 2012-09-07 19:23 UTC (permalink / raw)
To: Paolo Bonzini
Cc: Stefan Hajnoczi, kvm-devel, Michael S. Tsirkin, Jan Kiszka,
qemu-devel, Zhi Yong Wu, Anthony Liguori, target-devel, lf-virt,
Christoph Hellwig
In-Reply-To: <504A1A32.5050705@redhat.com>
On Fri, 2012-09-07 at 18:00 +0200, Paolo Bonzini wrote:
> Il 07/09/2012 08:48, Nicholas A. Bellinger ha scritto:
> > Cc: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> > Cc: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
> > Cc: Michael S. Tsirkin <mst@redhat.com>
> > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
> > ---
> > hw/virtio-pci.c | 2 ++
> > hw/virtio-scsi.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
> > hw/virtio-scsi.h | 1 +
> > 3 files changed, 52 insertions(+), 0 deletions(-)
>
> Please create a completely separate device vhost-scsi-pci instead (or
> virtio-scsi-tcm-pci, or something like that). It is used completely
> differently from virtio-scsi-pci, it does not make sense to conflate the
> two.
>
Ok, I need to figure out what this will involve over the next days, and
will likely have some more questions for you to get a standlone
vhost-scsi-pci up and running.
Also just curious (question for Anthony + QEMU folks), how long can we
expect the QEMU 1.3 merge window to be open..?
Thanks Paolo!
--nab
^ permalink raw reply
* Re: [PATCH 4/5] virtio-scsi: Add start/stop functionality for vhost-scsi
From: Paolo Bonzini @ 2012-09-07 16:00 UTC (permalink / raw)
To: Nicholas A. Bellinger
Cc: Stefan Hajnoczi, kvm-devel, Michael S. Tsirkin, Jan Kiszka,
qemu-devel, Zhi Yong Wu, Anthony Liguori, target-devel, lf-virt,
Christoph Hellwig
In-Reply-To: <1347000499-28701-5-git-send-email-nab@linux-iscsi.org>
Il 07/09/2012 08:48, Nicholas A. Bellinger ha scritto:
> Cc: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> Cc: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
> Cc: Michael S. Tsirkin <mst@redhat.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
> ---
> hw/virtio-pci.c | 2 ++
> hw/virtio-scsi.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
> hw/virtio-scsi.h | 1 +
> 3 files changed, 52 insertions(+), 0 deletions(-)
Please create a completely separate device vhost-scsi-pci instead (or
virtio-scsi-tcm-pci, or something like that). It is used completely
differently from virtio-scsi-pci, it does not make sense to conflate the
two.
Paolo
^ permalink raw reply
* Re: [PATCH] virtio-balloon spec: provide a version of the "silent deflate" feature that works
From: Paolo Bonzini @ 2012-09-07 14:44 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: fes, aarcange, riel, yvugenfi, kvm, linux-kernel, mikew, yinghan,
virtualization
In-Reply-To: <20120907142545.GC17397@redhat.com>
Il 07/09/2012 16:25, Michael S. Tsirkin ha scritto:
>> > Silent deflate is deflating just by using the page, without using the
>> > deflateq at all. So it can be done even from GFP_ATOMIC context.
> BTW I don't see a need to avoid deflateq.
> Without MUST_TELL_HOST you can just deflate and then
> bounce telling the host out to a thread.
Yes, that's fine too.
>> > But knowing that the guest will _not_ deflate silently also benefits the
>> > host. This is the cause of unpinning ballooned pages and pinning them
>> > again upon deflation. This allows cooperative memory overcommit even if
>> > the guests' memory is pinned, similar to what Xen did before it
>> > supported overcommit. This is the second feature bit.
> This is the MUST_TELL_HOST.
Almost. One is "the guest, if really needed, can tell the host of
pages". If not negotiated, and the host does not support it, the host
must break the guest (e.g. fail to offer any virtqueues).
The other is "the guest, though, would prefer not to do so". It is
different because the guest can proceed in a fallback mode even if the
host doesn't offer it.
You're interpreting features as something that dictates behavior, but
they're really just advisory. You could negotiate VIRTIO_BLK_F_TOPOLOGY
and end up never reading the fields; you could negotiate
VIRTIO_NET_F_GUEST_ANNOUNCE and never send a guest announcement.
>> > * guest will always do silent deflation: current Windows driver.
> Not exactly. It is not silent. It tells host
> just after use.
Yeah, but no difference from the POV of the driver.
Delaying or avoiding is the same in the end. The spec says it well: "In
this case, deflation advice is merely a courtesy".
>> > Negotiates nothing, or if it cares it can negotiate
>> > VIRTIO_BALLOON_F_SILENT_DEFLATE. Host mustn't do munlock/mlock dance.
>> >
>> > * guest may do silent deflation if available: combo of Linux driver +
>> > Frank's driver. Negotiates both features, looks at
>> > VIRTIO_BALLOON_F_SILENT_DEFLATE host feature to decide how to behave:
>> >
>> > ** If PCI passthrough, the host will not negotiate
>> > VIRTIO_BALLOON_F_SILENT_DEFLATE. The driver will behave as the
>> > current Linux driver, the host can do the munlock/mlock dance.
> So this case is just existing interface. OK.
>
>> > ** If no PCI passthrough, the host will negotiate
>> > VIRTIO_BALLOON_F_SILENT_DEFLATE, and the driver can balloon more
>> > aggressively and not use the deflateq.
>> >
> This last trickery confuses me. It just does not make sense to set both
> SILENT and TELL: either you are silent or you tell.
"I can be chatty if you ask me, but I'd rather be silent if you let me".
TELL is a request of the host to the guest; the guest can agree or not.
SILENT is a request of the guest to the host; the host can agree or not.
> What is the benefit of avoiding host notification?
> It seems cleaner for the host to know the state.
Yeah, if you want to do it you can.
Paolo
^ permalink raw reply
* Re: [PATCH] virtio-balloon spec: provide a version of the "silent deflate" feature that works
From: Michael S. Tsirkin @ 2012-09-07 14:25 UTC (permalink / raw)
To: Paolo Bonzini
Cc: fes, aarcange, riel, yvugenfi, kvm, linux-kernel, mikew, yinghan,
virtualization
In-Reply-To: <5049FEDD.40303@redhat.com>
On Fri, Sep 07, 2012 at 04:04:13PM +0200, Paolo Bonzini wrote:
> Il 07/09/2012 14:44, Michael S. Tsirkin ha scritto:
> >> Well, according to your reading of the spec.
> >>
> >> In the spec I'm reading "Host must be told before pages from the balloon
> >> are used". Doesn't say anything about the guest.
> >
> > No? How is host told then? By divine force?
>
> For a simple madvise-based implementation such as the one in QEMU, the
> host doesn't need to be told about anything (except periodic updating of
> the "actual" field, or the statsq).
>
> The guest doesn't need at all to use the deflateq in fact.
>
> >> Now, indeed a very free interpretation is "Guest will tell host before
> >> pages from the balloon are used". It turns out that it's exactly what
> >> guests have been doing, hence that's exactly what I'm proposing now:
> >> rename the feature to VIRTIO_BALLOON_F_WILL_TELL_HOST.
> >
> > Rename is fine.
>
> Cool.
>
> >> Yes, that part we agree on I think. We disagree that (after the rename)
> >> QEMU should start always proposing VIRTIO_BALLOON_F_WILL_TELL_HOST.
> >
> > Not always. It must be off if compatibility with old qemu is disabled.
>
> Yes, of course.
>
> >> _Plus_ adding the new feature bit, which is needed to actually tell the
> >
> > This part I do not get. What is silent deflate, why is it useful
> > and what it has to do with what we are discussing here?
>
> Silent deflate is deflating just by using the page, without using the
> deflateq at all. So it can be done even from GFP_ATOMIC context.
BTW I don't see a need to avoid deflateq.
Without MUST_TELL_HOST you can just deflate and then
bounce telling the host out to a thread.
> But knowing that the guest will _not_ deflate silently also benefits the
> host. This is the cause of unpinning ballooned pages and pinning them
> again upon deflation. This allows cooperative memory overcommit even if
> the guests' memory is pinned, similar to what Xen did before it
> supported overcommit. This is the second feature bit.
This is the MUST_TELL_HOST.
> There are three cases:
>
> * guest will never do silent deflation: current Linux driver. Host may
> do munlock/mlock dance. Negotiates VIRTIO_BALLOON_F_WILL_TELL_HOST
> only, features = VIRTIO_BALLOON_F_WILL_TELL_HOST.
>
> * guest will always do silent deflation: current Windows driver.
Not exactly. It is not silent. It tells host
just after use.
> Negotiates nothing, or if it cares it can negotiate
> VIRTIO_BALLOON_F_SILENT_DEFLATE. Host mustn't do munlock/mlock dance.
>
> * guest may do silent deflation if available: combo of Linux driver +
> Frank's driver. Negotiates both features, looks at
> VIRTIO_BALLOON_F_SILENT_DEFLATE host feature to decide how to behave:
>
> ** If PCI passthrough, the host will not negotiate
> VIRTIO_BALLOON_F_SILENT_DEFLATE. The driver will behave as the
> current Linux driver, the host can do the munlock/mlock dance.
So this case is just existing interface. OK.
> ** If no PCI passthrough, the host will negotiate
> VIRTIO_BALLOON_F_SILENT_DEFLATE, and the driver can balloon more
> aggressively and not use the deflateq.
>
> Paolo
This last trickery confuses me. It just does not make sense to set both
SILENT and TELL: either you are silent or you tell.
What is the benefit of avoiding host notification?
It seems cleaner for the host to know the state.
--
MST
^ permalink raw reply
* Re: [PATCH] virtio-balloon spec: provide a version of the "silent deflate" feature that works
From: Paolo Bonzini @ 2012-09-07 14:04 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: fes, aarcange, riel, yvugenfi, kvm, linux-kernel, mikew, yinghan,
virtualization
In-Reply-To: <20120907124432.GB17397@redhat.com>
Il 07/09/2012 14:44, Michael S. Tsirkin ha scritto:
>> Well, according to your reading of the spec.
>>
>> In the spec I'm reading "Host must be told before pages from the balloon
>> are used". Doesn't say anything about the guest.
>
> No? How is host told then? By divine force?
For a simple madvise-based implementation such as the one in QEMU, the
host doesn't need to be told about anything (except periodic updating of
the "actual" field, or the statsq).
The guest doesn't need at all to use the deflateq in fact.
>> Now, indeed a very free interpretation is "Guest will tell host before
>> pages from the balloon are used". It turns out that it's exactly what
>> guests have been doing, hence that's exactly what I'm proposing now:
>> rename the feature to VIRTIO_BALLOON_F_WILL_TELL_HOST.
>
> Rename is fine.
Cool.
>> Yes, that part we agree on I think. We disagree that (after the rename)
>> QEMU should start always proposing VIRTIO_BALLOON_F_WILL_TELL_HOST.
>
> Not always. It must be off if compatibility with old qemu is disabled.
Yes, of course.
>> _Plus_ adding the new feature bit, which is needed to actually tell the
>
> This part I do not get. What is silent deflate, why is it useful
> and what it has to do with what we are discussing here?
Silent deflate is deflating just by using the page, without using the
deflateq at all. So it can be done even from GFP_ATOMIC context.
But knowing that the guest will _not_ deflate silently also benefits the
host. This is the cause of unpinning ballooned pages and pinning them
again upon deflation. This allows cooperative memory overcommit even if
the guests' memory is pinned, similar to what Xen did before it
supported overcommit. This is the second feature bit.
There are three cases:
* guest will never do silent deflation: current Linux driver. Host may
do munlock/mlock dance. Negotiates VIRTIO_BALLOON_F_WILL_TELL_HOST
only, features = VIRTIO_BALLOON_F_WILL_TELL_HOST.
* guest will always do silent deflation: current Windows driver.
Negotiates nothing, or if it cares it can negotiate
VIRTIO_BALLOON_F_SILENT_DEFLATE. Host mustn't do munlock/mlock dance.
* guest may do silent deflation if available: combo of Linux driver +
Frank's driver. Negotiates both features, looks at
VIRTIO_BALLOON_F_SILENT_DEFLATE host feature to decide how to behave:
** If PCI passthrough, the host will not negotiate
VIRTIO_BALLOON_F_SILENT_DEFLATE. The driver will behave as the
current Linux driver, the host can do the munlock/mlock dance.
** If no PCI passthrough, the host will negotiate
VIRTIO_BALLOON_F_SILENT_DEFLATE, and the driver can balloon more
aggressively and not use the deflateq.
Paolo
^ permalink raw reply
* Re: [PATCH] virtio-balloon spec: provide a version of the "silent deflate" feature that works
From: Michael S. Tsirkin @ 2012-09-07 12:44 UTC (permalink / raw)
To: Paolo Bonzini
Cc: fes, aarcange, riel, yvugenfi, kvm, linux-kernel, mikew, yinghan,
virtualization
In-Reply-To: <5049E717.8080307@redhat.com>
On Fri, Sep 07, 2012 at 02:22:47PM +0200, Paolo Bonzini wrote:
> Il 07/09/2012 14:17, Michael S. Tsirkin ha scritto:
> >>> Next, consider the interface proposed here. You defacto declare
> >>> all existing drivers buggy.
> >>
> >> No, only Windows (and it is buggy, it calls tell_host last).
> >
> > It is not buggy. It does not ack MUST_TELL_HOST. So it is free to tell
> > host at any point, it behaves exactly
> > to spec. You can not retroactively declare drivers buggy like that.
>
> Well, according to your reading of the spec.
>
> In the spec I'm reading "Host must be told before pages from the balloon
> are used". Doesn't say anything about the guest.
No? How is host told then? By divine force?
> Now, indeed a very free interpretation is "Guest will tell host before
> pages from the balloon are used". It turns out that it's exactly what
> guests have been doing, hence that's exactly what I'm proposing now:
> rename the feature to VIRTIO_BALLOON_F_WILL_TELL_HOST.
Rename is fine.
> >> True, but the choice is:
> >>
> >> 1) add a once-only hack to QEMU that fixes migration of
> >> VIRTIO_BALLOON_F_MUST_TELL_HOST;
> >>
> >> 2) always advertise VIRTIO_BALLOON_F_MUST_TELL_HOST. If you do this,
> >> guests cannot use anymore silent deflate, which is a regression.
> >>
> >> 3) use two bits. One tells the device that the driver supports chatty
> >> deflate; one tells the driver that the device supports silent deflate.
> >
> > The right thing to do is either
> > 4. realize we can not address all user errors, so no real issue
> > 5. address this class of user errors by migrating host features
> >
> >> So in the end you do use two feature bits for two different things.
> >> Plus, both feature bits are "positive" and I'm happy.
> >
> > I am not happy.
> > We lose compatibility with all existing drivers
>
> How so?
>
> > so it will take years until the feature is actually
> > useful.
>
> No, we don't! Windows guests will just not be able to use munlock/mlock
> until the driver is fixed. Which will probably happen before someone
> writes the munlock/mlock code.
If the only change is rename then ofcourse things keep working.
I don't care about the name it is up to Rusty.
> > Is this just a matter of naming? Same functionality:
> > driver that acks this bit will tell host first,
> > driver that does not will not?
> >
> > If yes that is fine.
>
> Yes, that part we agree on I think. We disagree that (after the rename)
> QEMU should start always proposing VIRTIO_BALLOON_F_WILL_TELL_HOST.
Not always. It must be off if compatibility with old qemu is disabled.
> _Plus_ adding the new feature bit, which is needed to actually tell the
This part I do not get. What is silent deflate, why is it useful
and what it has to do with what we are discussing here?
> driver that the host supports the silent deflate.
> Spec patch on the way.
>
> Paolo
Hang on.
Can we please talk about motivation? These patches which come
without motivation are very hard to review.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox