* [PATCH] drivers: hv: use Linux version in guest ID
From: Bjørn Mork @ 2012-07-20 8:03 UTC (permalink / raw)
To: KY Srinivasan
Cc: Greg KH, Paolo Bonzini, devel, linux-kernel, virtualization,
Bjørn Mork
In-Reply-To: <87fw8mlv7e.fsf@nemi.mork.no>
Use OS version number in the guest ID as recommended by Microsoft
Signed-off-by: Bjørn Mork <bjorn@mork.no>
---
So how about something like this? Yes, I know including
linux/version.h often is unwanted, but the spec does
recommend using the actual OS version in the guest ID
Yes, this will be offensive again when Linux is at
version 181.0, but we might have found a better solution by
then :-)
drivers/hv/hyperv_vmbus.h | 20 ++++++++++++++++----
1 file changed, 16 insertions(+), 4 deletions(-)
diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
index b9426a6..cc0c3f0 100644
--- a/drivers/hv/hyperv_vmbus.h
+++ b/drivers/hv/hyperv_vmbus.h
@@ -29,6 +29,7 @@
#include <asm/sync_bitops.h>
#include <linux/atomic.h>
#include <linux/hyperv.h>
+#include <linux/version.h>
/*
* The below CPUID leaves are present if VersionAndFeatures.HypervisorPresent
@@ -410,10 +411,21 @@ enum {
#define HV_PRESENT_BIT 0x80000000
-#define HV_LINUX_GUEST_ID_LO 0x00000000
-#define HV_LINUX_GUEST_ID_HI 0xB16B00B5
-#define HV_LINUX_GUEST_ID (((u64)HV_LINUX_GUEST_ID_HI << 32) | \
- HV_LINUX_GUEST_ID_LO)
+/* Linux vendor ID allocated by Microsoft */
+#define HV_LINUX_GUEST_ID_VENDOR 0xB16B
+
+/* Creating a guest ID conforming to the encoding recommended by
+ * Microsoft, with variant and build fixed to 0
+ *
+ * 63:48 - Vendor ID
+ * 47:40 - OS Variant
+ * 39:32 - Major Version
+ * 31:24 - Minor Version
+ * 23:16 - Service Version
+ * 15:0 - Build Number
+ */
+#define HV_LINUX_GUEST_ID ((u64)HV_LINUX_GUEST_ID_VENDOR << 48 | \
+ (u64)LINUX_VERSION_CODE << 16)
#define HV_CPU_POWER_MANAGEMENT (1 << 0)
#define HV_RECOMMENDATIONS_MAX 4
--
1.7.10.4
^ permalink raw reply related
* Re: [PATCH RESEND 5/5] vhost-blk: Add vhost-blk support
From: Stefan Hajnoczi @ 2012-07-20 10:31 UTC (permalink / raw)
To: Khoa Huynh
Cc: virtualization, Michael S. Tsirkin, linux-kernel, Anthony Liguori,
kvm
In-Reply-To: <20120719130957.GD9303@redhat.com>
On Thu, Jul 19, 2012 at 2:09 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> On Thu, Jul 19, 2012 at 08:05:42AM -0500, Anthony Liguori wrote:
>> Of course, the million dollar question is why would using AIO in the
>> kernel be faster than using AIO in userspace?
>
> Actually for me a more important question is how does it compare
> with virtio-blk dataplane?
Hi Khoa,
I think you have results of data-plane and vhost-blk? Is the
vhost-blk version identical to Asias' recent patches?
Stefan
^ permalink raw reply
* Re: [RFC-v3 4/4] tcm_vhost: Initial merge for vhost level target fabric driver
From: Michael S. Tsirkin @ 2012-07-20 12:03 UTC (permalink / raw)
To: Nicholas A. Bellinger
Cc: Jens Axboe, Stefan Hajnoczi, kvm-devel, Greg Kroah-Hartman,
Zhi Yong Wu, Anthony Liguori, target-devel, linux-scsi,
Paolo Bonzini, lf-virt, Christoph Hellwig
In-Reply-To: <1342646458.18004.637.camel@haakon2.linux-iscsi.org>
On Wed, Jul 18, 2012 at 02:20:58PM -0700, Nicholas A. Bellinger wrote:
> On Wed, 2012-07-18 at 19:09 +0300, Michael S. Tsirkin wrote:
> > On Wed, Jul 18, 2012 at 12:59:32AM +0000, Nicholas A. Bellinger wrote:
>
> <SNIP>
>
> > >
> > > Changelog v2 -> v3:
> > >
> > > Unlock on error in tcm_vhost_drop_nexus() (DanC)
> > > Fix strlen() doesn't count the terminator (DanC)
> > > Call kfree() on an error path (DanC)
> > > Convert tcm_vhost_write_pending to use target_execute_cmd (hch + nab)
> > > Fix another strlen() off by one in tcm_vhost_make_tport (DanC)
> > > Add option under drivers/staging/Kconfig, and move to drivers/vhost/tcm/
> > > as requested by MST (nab)
> > >
> > > ---
> > > drivers/staging/Kconfig | 2 +
> > > drivers/vhost/Makefile | 2 +
> > > drivers/vhost/tcm/Kconfig | 6 +
> > > drivers/vhost/tcm/Makefile | 1 +
> > > drivers/vhost/tcm/tcm_vhost.c | 1611 +++++++++++++++++++++++++++++++++++++++++
> > > drivers/vhost/tcm/tcm_vhost.h | 74 ++
> > > 6 files changed, 1696 insertions(+), 0 deletions(-)
> > > create mode 100644 drivers/vhost/tcm/Kconfig
> > > create mode 100644 drivers/vhost/tcm/Makefile
> > > create mode 100644 drivers/vhost/tcm/tcm_vhost.c
> > > create mode 100644 drivers/vhost/tcm/tcm_vhost.h
> > >
> >
> > Really sorry about making you run around like that,
> > I did not mean moving all of tcm to a directory,
> > just adding tcm/Kconfig or adding drivers/vhost/Kconfig.tcm
> > because eventually it's easier to keep it all together
> > in one place.
> >
>
> Er, apologies for the slight mis-understanding here.. Moving back now +
> fixing up the Kbuild bits.
I'm going offline in several hours and am on vacation for a week
starting tomorrow. So to make 3.6, and if you intend to merge through my
tree, the best bet is if you can send the final version real soon now.
--
MST
^ permalink raw reply
* Re: [net-next RFC V5 5/5] virtio_net: support negotiating the number of queues through ctrl vq
From: Michael S. Tsirkin @ 2012-07-20 12:33 UTC (permalink / raw)
To: Jason Wang
Cc: krkumar2, habanero, mashirle, kvm, netdev, linux-kernel,
virtualization, edumazet, tahm, jwhan, davem, sri
In-Reply-To: <1341484194-8108-6-git-send-email-jasowang@redhat.com>
On Thu, Jul 05, 2012 at 06:29:54PM +0800, Jason Wang wrote:
> This patch let the virtio_net driver can negotiate the number of queues it
> wishes to use through control virtqueue and export an ethtool interface to let
> use tweak it.
>
> As current multiqueue virtio-net implementation has optimizations on per-cpu
> virtuqueues, so only two modes were support:
>
> - single queue pair mode
> - multiple queue paris mode, the number of queues matches the number of vcpus
>
> The single queue mode were used by default currently due to regression of
> multiqueue mode in some test (especially in stream test).
>
> Since virtio core does not support paritially deleting virtqueues, so during
> mode switching the whole virtqueue were deleted and the driver would re-create
> the virtqueues it would used.
>
> btw. The queue number negotiating were defered to .ndo_open(), this is because
> only after feature negotitaion could we send the command to control virtqueue
> (as it may also use event index).
>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
> ---
> drivers/net/virtio_net.c | 171 ++++++++++++++++++++++++++++++++++---------
> include/linux/virtio_net.h | 7 ++
> 2 files changed, 142 insertions(+), 36 deletions(-)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index 7410187..3339eeb 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -88,6 +88,7 @@ struct receive_queue {
>
> struct virtnet_info {
> u16 num_queue_pairs; /* # of RX/TX vq pairs */
> + u16 total_queue_pairs;
>
> struct send_queue *sq[MAX_QUEUES] ____cacheline_aligned_in_smp;
> struct receive_queue *rq[MAX_QUEUES] ____cacheline_aligned_in_smp;
> @@ -137,6 +138,8 @@ struct padded_vnet_hdr {
> char padding[6];
> };
>
> +static const struct ethtool_ops virtnet_ethtool_ops;
> +
> static inline int txq_get_qnum(struct virtnet_info *vi, struct virtqueue *vq)
> {
> int ret = virtqueue_get_queue_index(vq);
> @@ -802,22 +805,6 @@ static void virtnet_netpoll(struct net_device *dev)
> }
> #endif
>
> -static int virtnet_open(struct net_device *dev)
> -{
> - struct virtnet_info *vi = netdev_priv(dev);
> - int i;
> -
> - for (i = 0; i < vi->num_queue_pairs; i++) {
> - /* Make sure we have some buffers: if oom use wq. */
> - if (!try_fill_recv(vi->rq[i], GFP_KERNEL))
> - queue_delayed_work(system_nrt_wq,
> - &vi->rq[i]->refill, 0);
> - virtnet_napi_enable(vi->rq[i]);
> - }
> -
> - return 0;
> -}
> -
> /*
> * Send command via the control virtqueue and check status. Commands
> * supported by the hypervisor, as indicated by feature bits, should
> @@ -873,6 +860,43 @@ static void virtnet_ack_link_announce(struct virtnet_info *vi)
> rtnl_unlock();
> }
>
> +static int virtnet_set_queues(struct virtnet_info *vi)
> +{
> + struct scatterlist sg;
> + struct net_device *dev = vi->dev;
> + sg_init_one(&sg, &vi->num_queue_pairs, sizeof(vi->num_queue_pairs));
> +
> + if (!vi->has_cvq)
> + return -EINVAL;
> +
> + if (!virtnet_send_command(vi, VIRTIO_NET_CTRL_MULTIQUEUE,
> + VIRTIO_NET_CTRL_MULTIQUEUE_QNUM, &sg, 1, 0)){
> + dev_warn(&dev->dev, "Fail to set the number of queue pairs to"
> + " %d\n", vi->num_queue_pairs);
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +static int virtnet_open(struct net_device *dev)
> +{
> + struct virtnet_info *vi = netdev_priv(dev);
> + int i;
> +
> + for (i = 0; i < vi->num_queue_pairs; i++) {
> + /* Make sure we have some buffers: if oom use wq. */
> + if (!try_fill_recv(vi->rq[i], GFP_KERNEL))
> + queue_delayed_work(system_nrt_wq,
> + &vi->rq[i]->refill, 0);
> + virtnet_napi_enable(vi->rq[i]);
> + }
> +
> + virtnet_set_queues(vi);
> +
> + return 0;
> +}
> +
> static int virtnet_close(struct net_device *dev)
> {
> struct virtnet_info *vi = netdev_priv(dev);
> @@ -1013,12 +1037,6 @@ static void virtnet_get_drvinfo(struct net_device *dev,
>
> }
>
> -static const struct ethtool_ops virtnet_ethtool_ops = {
> - .get_drvinfo = virtnet_get_drvinfo,
> - .get_link = ethtool_op_get_link,
> - .get_ringparam = virtnet_get_ringparam,
> -};
> -
> #define MIN_MTU 68
> #define MAX_MTU 65535
>
> @@ -1235,7 +1253,7 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
>
> err:
> if (ret && names)
> - for (i = 0; i < vi->num_queue_pairs * 2; i++)
> + for (i = 0; i < total_vqs * 2; i++)
> kfree(names[i]);
>
> kfree(names);
> @@ -1373,7 +1391,6 @@ static int virtnet_probe(struct virtio_device *vdev)
> mutex_init(&vi->config_lock);
> vi->config_enable = true;
> INIT_WORK(&vi->config_work, virtnet_config_changed_work);
> - vi->num_queue_pairs = num_queue_pairs;
>
> /* If we can receive ANY GSO packets, we must allocate large ones. */
> if (virtio_has_feature(vdev, VIRTIO_NET_F_GUEST_TSO4) ||
> @@ -1387,6 +1404,10 @@ static int virtnet_probe(struct virtio_device *vdev)
> if (virtio_has_feature(vdev, VIRTIO_NET_F_CTRL_VQ))
> vi->has_cvq = true;
>
> + /* Use single tx/rx queue pair as default */
> + vi->num_queue_pairs = 1;
> + vi->total_queue_pairs = num_queue_pairs;
> +
> /* Allocate/initialize the rx/tx queues, and invoke find_vqs */
> err = virtnet_setup_vqs(vi);
> if (err)
> @@ -1396,6 +1417,9 @@ static int virtnet_probe(struct virtio_device *vdev)
> virtio_has_feature(vi->vdev, VIRTIO_NET_F_CTRL_VLAN))
> dev->features |= NETIF_F_HW_VLAN_FILTER;
>
> + netif_set_real_num_tx_queues(dev, 1);
> + netif_set_real_num_rx_queues(dev, 1);
> +
> err = register_netdev(dev);
> if (err) {
> pr_debug("virtio_net: registering device failed\n");
> @@ -1403,7 +1427,7 @@ static int virtnet_probe(struct virtio_device *vdev)
> }
>
> /* Last of all, set up some receive buffers. */
> - for (i = 0; i < num_queue_pairs; i++) {
> + for (i = 0; i < vi->num_queue_pairs; i++) {
> try_fill_recv(vi->rq[i], GFP_KERNEL);
>
> /* If we didn't even get one input buffer, we're useless. */
> @@ -1474,10 +1498,8 @@ static void __devexit virtnet_remove(struct virtio_device *vdev)
> free_netdev(vi->dev);
> }
>
> -#ifdef CONFIG_PM
> -static int virtnet_freeze(struct virtio_device *vdev)
> +static void virtnet_stop(struct virtnet_info *vi)
> {
> - struct virtnet_info *vi = vdev->priv;
> int i;
>
> /* Prevent config work handler from accessing the device */
> @@ -1493,17 +1515,10 @@ static int virtnet_freeze(struct virtio_device *vdev)
> for (i = 0; i < vi->num_queue_pairs; i++)
> napi_disable(&vi->rq[i]->napi);
>
> -
> - remove_vq_common(vi);
> -
> - flush_work(&vi->config_work);
> -
> - return 0;
> }
>
> -static int virtnet_restore(struct virtio_device *vdev)
> +static int virtnet_start(struct virtnet_info *vi)
> {
> - struct virtnet_info *vi = vdev->priv;
> int err, i;
>
> err = virtnet_setup_vqs(vi);
> @@ -1527,6 +1542,29 @@ static int virtnet_restore(struct virtio_device *vdev)
>
> return 0;
> }
> +
> +#ifdef CONFIG_PM
> +static int virtnet_freeze(struct virtio_device *vdev)
> +{
> + struct virtnet_info *vi = vdev->priv;
> +
> + virtnet_stop(vi);
> +
> + remove_vq_common(vi);
> +
> + flush_work(&vi->config_work);
> +
> + return 0;
> +}
> +
> +static int virtnet_restore(struct virtio_device *vdev)
> +{
> + struct virtnet_info *vi = vdev->priv;
> +
> + virtnet_start(vi);
> +
> + return 0;
> +}
> #endif
>
> static struct virtio_device_id id_table[] = {
> @@ -1560,6 +1598,67 @@ static struct virtio_driver virtio_net_driver = {
> #endif
> };
>
> +static int virtnet_set_channels(struct net_device *dev,
> + struct ethtool_channels *channels)
> +{
> + struct virtnet_info *vi = netdev_priv(dev);
> + u16 queues = channels->rx_count;
> + unsigned status = VIRTIO_CONFIG_S_ACKNOWLEDGE | VIRTIO_CONFIG_S_DRIVER;
> +
> + if (channels->rx_count != channels->tx_count)
> + return -EINVAL;
> + /* Only two modes were support currently */
s/were/are/ ?
> + if (queues != vi->total_queue_pairs && queues != 1)
> + return -EINVAL;
So userspace has to get queue number right. How does it know
what the valid value is?
> + if (!vi->has_cvq)
> + return -EINVAL;
> +
> + virtnet_stop(vi);
> +
> + netif_set_real_num_tx_queues(dev, queues);
> + netif_set_real_num_rx_queues(dev, queues);
> +
> + remove_vq_common(vi);
> + flush_work(&vi->config_work);
> +
> + vi->num_queue_pairs = queues;
> + virtnet_start(vi);
> +
> + vi->vdev->config->finalize_features(vi->vdev);
> +
> + if (virtnet_set_queues(vi))
> + status |= VIRTIO_CONFIG_S_FAILED;
> + else
> + status |= VIRTIO_CONFIG_S_DRIVER_OK;
> +
> + vi->vdev->config->set_status(vi->vdev, status);
> +
Why do we need to tweak status like that?
Can we maybe just roll changes back on error?
> + return 0;
> +}
> +
> +static void virtnet_get_channels(struct net_device *dev,
> + struct ethtool_channels *channels)
> +{
> + struct virtnet_info *vi = netdev_priv(dev);
> +
> + channels->max_rx = vi->total_queue_pairs;
> + channels->max_tx = vi->total_queue_pairs;
> + channels->max_other = 0;
> + channels->max_combined = 0;
> + channels->rx_count = vi->num_queue_pairs;
> + channels->tx_count = vi->num_queue_pairs;
> + channels->other_count = 0;
> + channels->combined_count = 0;
> +}
> +
> +static const struct ethtool_ops virtnet_ethtool_ops = {
> + .get_drvinfo = virtnet_get_drvinfo,
> + .get_link = ethtool_op_get_link,
> + .get_ringparam = virtnet_get_ringparam,
> + .set_channels = virtnet_set_channels,
> + .get_channels = virtnet_get_channels,
> +};
> +
> static int __init init(void)
> {
> return register_virtio_driver(&virtio_net_driver);
> diff --git a/include/linux/virtio_net.h b/include/linux/virtio_net.h
> index 60f09ff..0d21e08 100644
> --- a/include/linux/virtio_net.h
> +++ b/include/linux/virtio_net.h
> @@ -169,4 +169,11 @@ struct virtio_net_ctrl_mac {
> #define VIRTIO_NET_CTRL_ANNOUNCE 3
> #define VIRTIO_NET_CTRL_ANNOUNCE_ACK 0
>
> +/*
> + * Control multiqueue
> + *
> + */
> +#define VIRTIO_NET_CTRL_MULTIQUEUE 4
> + #define VIRTIO_NET_CTRL_MULTIQUEUE_QNUM 0
> +
> #endif /* _LINUX_VIRTIO_NET_H */
> --
> 1.7.1
^ permalink raw reply
* Re: [net-next RFC V5 4/5] virtio_net: multiqueue support
From: Michael S. Tsirkin @ 2012-07-20 13:40 UTC (permalink / raw)
To: Jason Wang
Cc: krkumar2, habanero, mashirle, kvm, netdev, linux-kernel,
virtualization, edumazet, tahm, jwhan, davem, sri
In-Reply-To: <1341484194-8108-5-git-send-email-jasowang@redhat.com>
On Thu, Jul 05, 2012 at 06:29:53PM +0800, Jason Wang wrote:
> This patch converts virtio_net to a multi queue device. After negotiated
> VIRTIO_NET_F_MULTIQUEUE feature, the virtio device has many tx/rx queue pairs,
> and driver could read the number from config space.
>
> The driver expects the number of rx/tx queue paris is equal to the number of
> vcpus. To maximize the performance under this per-cpu rx/tx queue pairs, some
> optimization were introduced:
>
> - Txq selection is based on the processor id in order to avoid contending a lock
> whose owner may exits to host.
> - Since the txq/txq were per-cpu, affinity hint were set to the cpu that owns
> the queue pairs.
>
> Signed-off-by: Krishna Kumar <krkumar2@in.ibm.com>
> Signed-off-by: Jason Wang <jasowang@redhat.com>
Overall fine. I think it is best to smash the following patch
into this one, so that default behavior does not
jump to mq then back. some comments below: mostly nits, and a minor bug.
If you are worried the patch is too big, it can be split
differently
- rework to use send_queue/receive_queue structures, no
functional changes.
- add multiqueue
but this is not a must.
> ---
> drivers/net/virtio_net.c | 645 ++++++++++++++++++++++++++++++-------------
> include/linux/virtio_net.h | 2 +
> 2 files changed, 452 insertions(+), 195 deletions(-)
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index 1db445b..7410187 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -26,6 +26,7 @@
> #include <linux/scatterlist.h>
> #include <linux/if_vlan.h>
> #include <linux/slab.h>
> +#include <linux/interrupt.h>
>
> static int napi_weight = 128;
> module_param(napi_weight, int, 0444);
> @@ -41,6 +42,8 @@ module_param(gso, bool, 0444);
> #define VIRTNET_SEND_COMMAND_SG_MAX 2
> #define VIRTNET_DRIVER_VERSION "1.0.0"
>
> +#define MAX_QUEUES 256
> +
> struct virtnet_stats {
> struct u64_stats_sync tx_syncp;
> struct u64_stats_sync rx_syncp;
Would be a bit better not to have artificial limits like that.
Maybe allocate arrays at probe time, then we can
take whatever the device gives us?
> @@ -51,43 +54,69 @@ struct virtnet_stats {
> u64 rx_packets;
> };
>
> -struct virtnet_info {
> - struct virtio_device *vdev;
> - struct virtqueue *rvq, *svq, *cvq;
> - struct net_device *dev;
> +/* Internal representation of a send virtqueue */
> +struct send_queue {
> + /* Virtqueue associated with this send _queue */
> + struct virtqueue *vq;
> +
> + /* TX: fragments + linear part + virtio header */
> + struct scatterlist sg[MAX_SKB_FRAGS + 2];
> +};
> +
> +/* Internal representation of a receive virtqueue */
> +struct receive_queue {
> + /* Virtqueue associated with this receive_queue */
> + struct virtqueue *vq;
> +
> + /* Back pointer to the virtnet_info */
> + struct virtnet_info *vi;
> +
> struct napi_struct napi;
> - unsigned int status;
>
> /* Number of input buffers, and max we've ever had. */
> unsigned int num, max;
>
> + /* Work struct for refilling if we run low on memory. */
> + struct delayed_work refill;
> +
> + /* Chain pages by the private ptr. */
> + struct page *pages;
> +
> + /* RX: fragments + linear part + virtio header */
> + struct scatterlist sg[MAX_SKB_FRAGS + 2];
> +};
> +
> +struct virtnet_info {
> + u16 num_queue_pairs; /* # of RX/TX vq pairs */
> +
> + struct send_queue *sq[MAX_QUEUES] ____cacheline_aligned_in_smp;
> + struct receive_queue *rq[MAX_QUEUES] ____cacheline_aligned_in_smp;
The assumption is a tx/rx pair is handled on the same cpu, yes?
If yes maybe make it a single array to improve cache locality
a bit?
struct queue_pair {
struct send_queue sq;
struct receive_queue rq;
};
> + struct virtqueue *cvq;
> +
> + struct virtio_device *vdev;
> + struct net_device *dev;
> + unsigned int status;
> +
> /* I like... big packets and I cannot lie! */
> bool big_packets;
>
> /* Host will merge rx buffers for big packets (shake it! shake it!) */
> bool mergeable_rx_bufs;
>
> + /* Has control virtqueue */
> + bool has_cvq;
> +
won't checking (cvq != NULL) be enough?
> /* enable config space updates */
> bool config_enable;
>
> /* Active statistics */
> struct virtnet_stats __percpu *stats;
>
> - /* Work struct for refilling if we run low on memory. */
> - struct delayed_work refill;
> -
> /* Work struct for config space updates */
> struct work_struct config_work;
>
> /* Lock for config space updates */
> struct mutex config_lock;
> -
> - /* Chain pages by the private ptr. */
> - struct page *pages;
> -
> - /* fragments + linear part + virtio header */
> - struct scatterlist rx_sg[MAX_SKB_FRAGS + 2];
> - struct scatterlist tx_sg[MAX_SKB_FRAGS + 2];
> };
>
> struct skb_vnet_hdr {
> @@ -108,6 +137,22 @@ struct padded_vnet_hdr {
> char padding[6];
> };
>
> +static inline int txq_get_qnum(struct virtnet_info *vi, struct virtqueue *vq)
> +{
> + int ret = virtqueue_get_queue_index(vq);
> +
> + /* skip ctrl vq */
> + if (vi->has_cvq)
> + return (ret - 1) / 2;
> + else
> + return ret / 2;
> +}
> +
> +static inline int rxq_get_qnum(struct virtnet_info *vi, struct virtqueue *vq)
> +{
> + return virtqueue_get_queue_index(vq) / 2;
> +}
> +
> static inline struct skb_vnet_hdr *skb_vnet_hdr(struct sk_buff *skb)
> {
> return (struct skb_vnet_hdr *)skb->cb;
> @@ -117,22 +162,22 @@ static inline struct skb_vnet_hdr *skb_vnet_hdr(struct sk_buff *skb)
> * private is used to chain pages for big packets, put the whole
> * most recent used list in the beginning for reuse
> */
> -static void give_pages(struct virtnet_info *vi, struct page *page)
> +static void give_pages(struct receive_queue *rq, struct page *page)
> {
> struct page *end;
>
> /* Find end of list, sew whole thing into vi->pages. */
> for (end = page; end->private; end = (struct page *)end->private);
> - end->private = (unsigned long)vi->pages;
> - vi->pages = page;
> + end->private = (unsigned long)rq->pages;
> + rq->pages = page;
> }
>
> -static struct page *get_a_page(struct virtnet_info *vi, gfp_t gfp_mask)
> +static struct page *get_a_page(struct receive_queue *rq, gfp_t gfp_mask)
> {
> - struct page *p = vi->pages;
> + struct page *p = rq->pages;
>
> if (p) {
> - vi->pages = (struct page *)p->private;
> + rq->pages = (struct page *)p->private;
> /* clear private here, it is used to chain pages */
> p->private = 0;
> } else
> @@ -140,15 +185,15 @@ static struct page *get_a_page(struct virtnet_info *vi, gfp_t gfp_mask)
> return p;
> }
>
> -static void skb_xmit_done(struct virtqueue *svq)
> +static void skb_xmit_done(struct virtqueue *vq)
> {
> - struct virtnet_info *vi = svq->vdev->priv;
> + struct virtnet_info *vi = vq->vdev->priv;
>
> /* Suppress further interrupts. */
> - virtqueue_disable_cb(svq);
> + virtqueue_disable_cb(vq);
>
> /* We were probably waiting for more output buffers. */
> - netif_wake_queue(vi->dev);
> + netif_wake_subqueue(vi->dev, txq_get_qnum(vi, vq));
> }
>
> static void set_skb_frag(struct sk_buff *skb, struct page *page,
> @@ -167,9 +212,10 @@ static void set_skb_frag(struct sk_buff *skb, struct page *page,
> }
>
> /* Called from bottom half context */
> -static struct sk_buff *page_to_skb(struct virtnet_info *vi,
> +static struct sk_buff *page_to_skb(struct receive_queue *rq,
> struct page *page, unsigned int len)
> {
> + struct virtnet_info *vi = rq->vi;
> struct sk_buff *skb;
> struct skb_vnet_hdr *hdr;
> unsigned int copy, hdr_len, offset;
> @@ -225,12 +271,12 @@ static struct sk_buff *page_to_skb(struct virtnet_info *vi,
> }
>
> if (page)
> - give_pages(vi, page);
> + give_pages(rq, page);
>
> return skb;
> }
>
> -static int receive_mergeable(struct virtnet_info *vi, struct sk_buff *skb)
> +static int receive_mergeable(struct receive_queue *rq, struct sk_buff *skb)
> {
> struct skb_vnet_hdr *hdr = skb_vnet_hdr(skb);
> struct page *page;
> @@ -244,7 +290,7 @@ static int receive_mergeable(struct virtnet_info *vi, struct sk_buff *skb)
> skb->dev->stats.rx_length_errors++;
> return -EINVAL;
> }
> - page = virtqueue_get_buf(vi->rvq, &len);
> + page = virtqueue_get_buf(rq->vq, &len);
> if (!page) {
> pr_debug("%s: rx error: %d buffers missing\n",
> skb->dev->name, hdr->mhdr.num_buffers);
> @@ -257,13 +303,14 @@ static int receive_mergeable(struct virtnet_info *vi, struct sk_buff *skb)
>
> set_skb_frag(skb, page, 0, &len);
>
> - --vi->num;
> + --rq->num;
> }
> return 0;
> }
>
> -static void receive_buf(struct net_device *dev, void *buf, unsigned int len)
> +static void receive_buf(struct receive_queue *rq, void *buf, unsigned int len)
> {
> + struct net_device *dev = rq->vi->dev;
> struct virtnet_info *vi = netdev_priv(dev);
> struct virtnet_stats *stats = this_cpu_ptr(vi->stats);
> struct sk_buff *skb;
> @@ -274,7 +321,7 @@ static void receive_buf(struct net_device *dev, void *buf, unsigned int len)
> pr_debug("%s: short packet %i\n", dev->name, len);
> dev->stats.rx_length_errors++;
> if (vi->mergeable_rx_bufs || vi->big_packets)
> - give_pages(vi, buf);
> + give_pages(rq, buf);
> else
> dev_kfree_skb(buf);
> return;
> @@ -286,14 +333,14 @@ static void receive_buf(struct net_device *dev, void *buf, unsigned int len)
> skb_trim(skb, len);
> } else {
> page = buf;
> - skb = page_to_skb(vi, page, len);
> + skb = page_to_skb(rq, page, len);
> if (unlikely(!skb)) {
> dev->stats.rx_dropped++;
> - give_pages(vi, page);
> + give_pages(rq, page);
> return;
> }
> if (vi->mergeable_rx_bufs)
> - if (receive_mergeable(vi, skb)) {
> + if (receive_mergeable(rq, skb)) {
> dev_kfree_skb(skb);
> return;
> }
> @@ -363,90 +410,91 @@ frame_err:
> dev_kfree_skb(skb);
> }
>
> -static int add_recvbuf_small(struct virtnet_info *vi, gfp_t gfp)
> +static int add_recvbuf_small(struct receive_queue *rq, gfp_t gfp)
> {
> struct sk_buff *skb;
> struct skb_vnet_hdr *hdr;
> int err;
>
> - skb = __netdev_alloc_skb_ip_align(vi->dev, MAX_PACKET_LEN, gfp);
> + skb = __netdev_alloc_skb_ip_align(rq->vi->dev, MAX_PACKET_LEN, gfp);
> if (unlikely(!skb))
> return -ENOMEM;
>
> skb_put(skb, MAX_PACKET_LEN);
>
> hdr = skb_vnet_hdr(skb);
> - sg_set_buf(vi->rx_sg, &hdr->hdr, sizeof hdr->hdr);
> + sg_set_buf(rq->sg, &hdr->hdr, sizeof hdr->hdr);
> +
> + skb_to_sgvec(skb, rq->sg + 1, 0, skb->len);
>
> - skb_to_sgvec(skb, vi->rx_sg + 1, 0, skb->len);
> + err = virtqueue_add_buf(rq->vq, rq->sg, 0, 2, skb, gfp);
>
> - err = virtqueue_add_buf(vi->rvq, vi->rx_sg, 0, 2, skb, gfp);
> if (err < 0)
> dev_kfree_skb(skb);
>
> return err;
> }
>
> -static int add_recvbuf_big(struct virtnet_info *vi, gfp_t gfp)
> +static int add_recvbuf_big(struct receive_queue *rq, gfp_t gfp)
> {
> struct page *first, *list = NULL;
> char *p;
> int i, err, offset;
>
> - /* page in vi->rx_sg[MAX_SKB_FRAGS + 1] is list tail */
> + /* page in rq->sg[MAX_SKB_FRAGS + 1] is list tail */
> for (i = MAX_SKB_FRAGS + 1; i > 1; --i) {
> - first = get_a_page(vi, gfp);
> + first = get_a_page(rq, gfp);
> if (!first) {
> if (list)
> - give_pages(vi, list);
> + give_pages(rq, list);
> return -ENOMEM;
> }
> - sg_set_buf(&vi->rx_sg[i], page_address(first), PAGE_SIZE);
> + sg_set_buf(&rq->sg[i], page_address(first), PAGE_SIZE);
>
> /* chain new page in list head to match sg */
> first->private = (unsigned long)list;
> list = first;
> }
>
> - first = get_a_page(vi, gfp);
> + first = get_a_page(rq, gfp);
> if (!first) {
> - give_pages(vi, list);
> + give_pages(rq, list);
> return -ENOMEM;
> }
> p = page_address(first);
>
> - /* vi->rx_sg[0], vi->rx_sg[1] share the same page */
> - /* a separated vi->rx_sg[0] for virtio_net_hdr only due to QEMU bug */
> - sg_set_buf(&vi->rx_sg[0], p, sizeof(struct virtio_net_hdr));
> + /* rq->sg[0], rq->sg[1] share the same page */
> + /* a separated rq->sg[0] for virtio_net_hdr only due to QEMU bug */
> + sg_set_buf(&rq->sg[0], p, sizeof(struct virtio_net_hdr));
>
> - /* vi->rx_sg[1] for data packet, from offset */
> + /* rq->sg[1] for data packet, from offset */
> offset = sizeof(struct padded_vnet_hdr);
> - sg_set_buf(&vi->rx_sg[1], p + offset, PAGE_SIZE - offset);
> + sg_set_buf(&rq->sg[1], p + offset, PAGE_SIZE - offset);
>
> /* chain first in list head */
> first->private = (unsigned long)list;
> - err = virtqueue_add_buf(vi->rvq, vi->rx_sg, 0, MAX_SKB_FRAGS + 2,
> + err = virtqueue_add_buf(rq->vq, rq->sg, 0, MAX_SKB_FRAGS + 2,
> first, gfp);
> if (err < 0)
> - give_pages(vi, first);
> + give_pages(rq, first);
>
> return err;
> }
>
> -static int add_recvbuf_mergeable(struct virtnet_info *vi, gfp_t gfp)
> +static int add_recvbuf_mergeable(struct receive_queue *rq, gfp_t gfp)
> {
> struct page *page;
> int err;
>
> - page = get_a_page(vi, gfp);
> + page = get_a_page(rq, gfp);
> if (!page)
> return -ENOMEM;
>
> - sg_init_one(vi->rx_sg, page_address(page), PAGE_SIZE);
> + sg_init_one(rq->sg, page_address(page), PAGE_SIZE);
>
> - err = virtqueue_add_buf(vi->rvq, vi->rx_sg, 0, 1, page, gfp);
> + err = virtqueue_add_buf(rq->vq, rq->sg, 0, 1, page, gfp);
> if (err < 0)
> - give_pages(vi, page);
> + give_pages(rq, page);
>
> return err;
> }
> @@ -458,97 +506,104 @@ static int add_recvbuf_mergeable(struct virtnet_info *vi, gfp_t gfp)
> * before we're receiving packets, or from refill_work which is
> * careful to disable receiving (using napi_disable).
> */
> -static bool try_fill_recv(struct virtnet_info *vi, gfp_t gfp)
> +static bool try_fill_recv(struct receive_queue *rq, gfp_t gfp)
> {
> + struct virtnet_info *vi = rq->vi;
> int err;
> bool oom;
>
> do {
> if (vi->mergeable_rx_bufs)
> - err = add_recvbuf_mergeable(vi, gfp);
> + err = add_recvbuf_mergeable(rq, gfp);
> else if (vi->big_packets)
> - err = add_recvbuf_big(vi, gfp);
> + err = add_recvbuf_big(rq, gfp);
> else
> - err = add_recvbuf_small(vi, gfp);
> + err = add_recvbuf_small(rq, gfp);
>
> oom = err == -ENOMEM;
> if (err < 0)
> break;
> - ++vi->num;
> + ++rq->num;
> } while (err > 0);
> - if (unlikely(vi->num > vi->max))
> - vi->max = vi->num;
> - virtqueue_kick(vi->rvq);
> + if (unlikely(rq->num > rq->max))
> + rq->max = rq->num;
> + virtqueue_kick(rq->vq);
> return !oom;
> }
>
> -static void skb_recv_done(struct virtqueue *rvq)
> +static void skb_recv_done(struct virtqueue *vq)
> {
> - struct virtnet_info *vi = rvq->vdev->priv;
> + struct virtnet_info *vi = vq->vdev->priv;
> + struct napi_struct *napi = &vi->rq[rxq_get_qnum(vi, vq)]->napi;
> +
> /* Schedule NAPI, Suppress further interrupts if successful. */
> - if (napi_schedule_prep(&vi->napi)) {
> - virtqueue_disable_cb(rvq);
> - __napi_schedule(&vi->napi);
> + if (napi_schedule_prep(napi)) {
> + virtqueue_disable_cb(vq);
> + __napi_schedule(napi);
> }
> }
>
> -static void virtnet_napi_enable(struct virtnet_info *vi)
> +static void virtnet_napi_enable(struct receive_queue *rq)
> {
> - napi_enable(&vi->napi);
> + napi_enable(&rq->napi);
>
> /* If all buffers were filled by other side before we napi_enabled, we
> * won't get another interrupt, so process any outstanding packets
> * now. virtnet_poll wants re-enable the queue, so we disable here.
> * We synchronize against interrupts via NAPI_STATE_SCHED */
> - if (napi_schedule_prep(&vi->napi)) {
> - virtqueue_disable_cb(vi->rvq);
> + if (napi_schedule_prep(&rq->napi)) {
> + virtqueue_disable_cb(rq->vq);
> local_bh_disable();
> - __napi_schedule(&vi->napi);
> + __napi_schedule(&rq->napi);
> local_bh_enable();
> }
> }
>
> static void refill_work(struct work_struct *work)
> {
> - struct virtnet_info *vi;
> + struct napi_struct *napi;
> + struct receive_queue *rq;
> bool still_empty;
>
> - vi = container_of(work, struct virtnet_info, refill.work);
> - napi_disable(&vi->napi);
> - still_empty = !try_fill_recv(vi, GFP_KERNEL);
> - virtnet_napi_enable(vi);
> + rq = container_of(work, struct receive_queue, refill.work);
> + napi = &rq->napi;
> +
> + napi_disable(napi);
> + still_empty = !try_fill_recv(rq, GFP_KERNEL);
> + virtnet_napi_enable(rq);
>
> /* In theory, this can happen: if we don't get any buffers in
> * we will *never* try to fill again. */
> if (still_empty)
> - queue_delayed_work(system_nrt_wq, &vi->refill, HZ/2);
> + queue_delayed_work(system_nrt_wq, &rq->refill, HZ/2);
> }
>
> static int virtnet_poll(struct napi_struct *napi, int budget)
> {
> - struct virtnet_info *vi = container_of(napi, struct virtnet_info, napi);
> + struct receive_queue *rq = container_of(napi, struct receive_queue,
> + napi);
> void *buf;
> unsigned int len, received = 0;
>
> again:
> while (received < budget &&
> - (buf = virtqueue_get_buf(vi->rvq, &len)) != NULL) {
> - receive_buf(vi->dev, buf, len);
> - --vi->num;
> + (buf = virtqueue_get_buf(rq->vq, &len)) != NULL) {
> + receive_buf(rq, buf, len);
> + --rq->num;
> received++;
> }
>
> - if (vi->num < vi->max / 2) {
> - if (!try_fill_recv(vi, GFP_ATOMIC))
> - queue_delayed_work(system_nrt_wq, &vi->refill, 0);
> + if (rq->num < rq->max / 2) {
> + if (!try_fill_recv(rq, GFP_ATOMIC))
> + queue_delayed_work(system_nrt_wq, &rq->refill, 0);
> }
>
> /* Out of packets? */
> if (received < budget) {
> napi_complete(napi);
> - if (unlikely(!virtqueue_enable_cb(vi->rvq)) &&
> + if (unlikely(!virtqueue_enable_cb(rq->vq)) &&
> napi_schedule_prep(napi)) {
> - virtqueue_disable_cb(vi->rvq);
> + virtqueue_disable_cb(rq->vq);
> __napi_schedule(napi);
> goto again;
> }
> @@ -557,13 +612,14 @@ again:
> return received;
> }
>
> -static unsigned int free_old_xmit_skbs(struct virtnet_info *vi)
> +static unsigned int free_old_xmit_skbs(struct virtnet_info *vi,
> + struct virtqueue *vq)
> {
> struct sk_buff *skb;
> unsigned int len, tot_sgs = 0;
> struct virtnet_stats *stats = this_cpu_ptr(vi->stats);
>
> - while ((skb = virtqueue_get_buf(vi->svq, &len)) != NULL) {
> + while ((skb = virtqueue_get_buf(vq, &len)) != NULL) {
> pr_debug("Sent skb %p\n", skb);
>
> u64_stats_update_begin(&stats->tx_syncp);
> @@ -577,7 +633,8 @@ static unsigned int free_old_xmit_skbs(struct virtnet_info *vi)
> return tot_sgs;
> }
>
> -static int xmit_skb(struct virtnet_info *vi, struct sk_buff *skb)
> +static int xmit_skb(struct virtnet_info *vi, struct sk_buff *skb,
> + struct virtqueue *vq, struct scatterlist *sg)
> {
> struct skb_vnet_hdr *hdr = skb_vnet_hdr(skb);
> const unsigned char *dest = ((struct ethhdr *)skb->data)->h_dest;
> @@ -615,44 +672,47 @@ static int xmit_skb(struct virtnet_info *vi, struct sk_buff *skb)
>
> /* Encode metadata header at front. */
> if (vi->mergeable_rx_bufs)
> - sg_set_buf(vi->tx_sg, &hdr->mhdr, sizeof hdr->mhdr);
> + sg_set_buf(sg, &hdr->mhdr, sizeof hdr->mhdr);
> else
> - sg_set_buf(vi->tx_sg, &hdr->hdr, sizeof hdr->hdr);
> + sg_set_buf(sg, &hdr->hdr, sizeof hdr->hdr);
>
> - hdr->num_sg = skb_to_sgvec(skb, vi->tx_sg + 1, 0, skb->len) + 1;
> - return virtqueue_add_buf(vi->svq, vi->tx_sg, hdr->num_sg,
> + hdr->num_sg = skb_to_sgvec(skb, sg + 1, 0, skb->len) + 1;
> + return virtqueue_add_buf(vq, sg, hdr->num_sg,
> 0, skb, GFP_ATOMIC);
> }
>
> static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> {
> struct virtnet_info *vi = netdev_priv(dev);
> + int qnum = skb_get_queue_mapping(skb);
> + struct virtqueue *vq = vi->sq[qnum]->vq;
> int capacity;
>
> /* Free up any pending old buffers before queueing new ones. */
> - free_old_xmit_skbs(vi);
> + free_old_xmit_skbs(vi, vq);
>
> /* Try to transmit */
> - capacity = xmit_skb(vi, skb);
> + capacity = xmit_skb(vi, skb, vq, vi->sq[qnum]->sg);
>
> /* This can happen with OOM and indirect buffers. */
> if (unlikely(capacity < 0)) {
> if (likely(capacity == -ENOMEM)) {
> if (net_ratelimit())
> dev_warn(&dev->dev,
> - "TX queue failure: out of memory\n");
> + "TXQ (%d) failure: out of memory\n",
> + qnum);
> } else {
> dev->stats.tx_fifo_errors++;
> if (net_ratelimit())
> dev_warn(&dev->dev,
> - "Unexpected TX queue failure: %d\n",
> - capacity);
> + "Unexpected TXQ (%d) failure: %d\n",
> + qnum, capacity);
> }
> dev->stats.tx_dropped++;
> kfree_skb(skb);
> return NETDEV_TX_OK;
> }
> - virtqueue_kick(vi->svq);
> + virtqueue_kick(vq);
>
> /* Don't wait up for transmitted skbs to be freed. */
> skb_orphan(skb);
> @@ -661,13 +721,13 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> /* Apparently nice girls don't return TX_BUSY; stop the queue
> * before it gets out of hand. Naturally, this wastes entries. */
> if (capacity < 2+MAX_SKB_FRAGS) {
> - netif_stop_queue(dev);
> - if (unlikely(!virtqueue_enable_cb_delayed(vi->svq))) {
> + netif_stop_subqueue(dev, qnum);
> + if (unlikely(!virtqueue_enable_cb_delayed(vq))) {
> /* More just got used, free them then recheck. */
> - capacity += free_old_xmit_skbs(vi);
> + capacity += free_old_xmit_skbs(vi, vq);
> if (capacity >= 2+MAX_SKB_FRAGS) {
> - netif_start_queue(dev);
> - virtqueue_disable_cb(vi->svq);
> + netif_start_subqueue(dev, qnum);
> + virtqueue_disable_cb(vq);
> }
> }
> }
> @@ -700,7 +760,8 @@ static struct rtnl_link_stats64 *virtnet_stats(struct net_device *dev,
> unsigned int start;
>
> for_each_possible_cpu(cpu) {
> - struct virtnet_stats *stats = per_cpu_ptr(vi->stats, cpu);
> + struct virtnet_stats __percpu *stats
> + = per_cpu_ptr(vi->stats, cpu);
> u64 tpackets, tbytes, rpackets, rbytes;
>
> do {
> @@ -734,20 +795,26 @@ static struct rtnl_link_stats64 *virtnet_stats(struct net_device *dev,
> static void virtnet_netpoll(struct net_device *dev)
> {
> struct virtnet_info *vi = netdev_priv(dev);
> + int i;
>
> - napi_schedule(&vi->napi);
> + for (i = 0; i < vi->num_queue_pairs; i++)
> + napi_schedule(&vi->rq[i]->napi);
> }
> #endif
>
> static int virtnet_open(struct net_device *dev)
> {
> struct virtnet_info *vi = netdev_priv(dev);
> + int i;
>
> - /* Make sure we have some buffers: if oom use wq. */
> - if (!try_fill_recv(vi, GFP_KERNEL))
> - queue_delayed_work(system_nrt_wq, &vi->refill, 0);
> + for (i = 0; i < vi->num_queue_pairs; i++) {
> + /* Make sure we have some buffers: if oom use wq. */
> + if (!try_fill_recv(vi->rq[i], GFP_KERNEL))
> + queue_delayed_work(system_nrt_wq,
> + &vi->rq[i]->refill, 0);
> + virtnet_napi_enable(vi->rq[i]);
> + }
>
> - virtnet_napi_enable(vi);
> return 0;
> }
>
> @@ -809,10 +876,13 @@ static void virtnet_ack_link_announce(struct virtnet_info *vi)
> static int virtnet_close(struct net_device *dev)
> {
> struct virtnet_info *vi = netdev_priv(dev);
> + int i;
>
> /* Make sure refill_work doesn't re-enable napi! */
> - cancel_delayed_work_sync(&vi->refill);
> - napi_disable(&vi->napi);
> + for (i = 0; i < vi->num_queue_pairs; i++) {
> + cancel_delayed_work_sync(&vi->rq[i]->refill);
> + napi_disable(&vi->rq[i]->napi);
> + }
>
> return 0;
> }
> @@ -924,11 +994,10 @@ static void virtnet_get_ringparam(struct net_device *dev,
> {
> struct virtnet_info *vi = netdev_priv(dev);
>
> - ring->rx_max_pending = virtqueue_get_vring_size(vi->rvq);
> - ring->tx_max_pending = virtqueue_get_vring_size(vi->svq);
> + ring->rx_max_pending = virtqueue_get_vring_size(vi->rq[0]->vq);
> + ring->tx_max_pending = virtqueue_get_vring_size(vi->sq[0]->vq);
> ring->rx_pending = ring->rx_max_pending;
> ring->tx_pending = ring->tx_max_pending;
> -
> }
>
>
> @@ -961,6 +1030,19 @@ static int virtnet_change_mtu(struct net_device *dev, int new_mtu)
> return 0;
> }
>
> +/* To avoid contending a lock hold by a vcpu who would exit to host, select the
> + * txq based on the processor id.
> + */
> +static u16 virtnet_select_queue(struct net_device *dev, struct sk_buff *skb)
> +{
> + int txq = skb_rx_queue_recorded(skb) ? skb_get_rx_queue(skb) :
> + smp_processor_id();
> +
> + while (unlikely(txq >= dev->real_num_tx_queues))
> + txq -= dev->real_num_tx_queues;
> + return txq;
> +}
> +
> static const struct net_device_ops virtnet_netdev = {
> .ndo_open = virtnet_open,
> .ndo_stop = virtnet_close,
> @@ -972,6 +1054,7 @@ static const struct net_device_ops virtnet_netdev = {
> .ndo_get_stats64 = virtnet_stats,
> .ndo_vlan_rx_add_vid = virtnet_vlan_rx_add_vid,
> .ndo_vlan_rx_kill_vid = virtnet_vlan_rx_kill_vid,
> + .ndo_select_queue = virtnet_select_queue,
> #ifdef CONFIG_NET_POLL_CONTROLLER
> .ndo_poll_controller = virtnet_netpoll,
> #endif
> @@ -1007,10 +1090,10 @@ static void virtnet_config_changed_work(struct work_struct *work)
>
> if (vi->status & VIRTIO_NET_S_LINK_UP) {
> netif_carrier_on(vi->dev);
> - netif_wake_queue(vi->dev);
> + netif_tx_wake_all_queues(vi->dev);
> } else {
> netif_carrier_off(vi->dev);
> - netif_stop_queue(vi->dev);
> + netif_tx_stop_all_queues(vi->dev);
> }
> done:
> mutex_unlock(&vi->config_lock);
> @@ -1023,41 +1106,217 @@ static void virtnet_config_changed(struct virtio_device *vdev)
> queue_work(system_nrt_wq, &vi->config_work);
> }
>
> -static int init_vqs(struct virtnet_info *vi)
> +static void free_receive_bufs(struct virtnet_info *vi)
> +{
> + int i;
> +
> + for (i = 0; i < vi->num_queue_pairs; i++) {
> + while (vi->rq[i]->pages)
> + __free_pages(get_a_page(vi->rq[i], GFP_KERNEL), 0);
> + }
> +}
> +
> +/* Free memory allocated for send and receive queues */
> +static void virtnet_free_queues(struct virtnet_info *vi)
> {
> - struct virtqueue *vqs[3];
> - vq_callback_t *callbacks[] = { skb_recv_done, skb_xmit_done, NULL};
> - const char *names[] = { "input", "output", "control" };
> - int nvqs, err;
> + int i;
>
> - /* We expect two virtqueues, receive then send,
> - * and optionally control. */
> - nvqs = virtio_has_feature(vi->vdev, VIRTIO_NET_F_CTRL_VQ) ? 3 : 2;
> + for (i = 0; i < vi->num_queue_pairs; i++) {
> + kfree(vi->rq[i]);
> + vi->rq[i] = NULL;
> + kfree(vi->sq[i]);
> + vi->sq[i] = NULL;
> + }
> +}
>
> - err = vi->vdev->config->find_vqs(vi->vdev, nvqs, vqs, callbacks, names);
> - if (err)
> - return err;
> +static void free_unused_bufs(struct virtnet_info *vi)
> +{
> + void *buf;
> + int i;
> +
> + for (i = 0; i < vi->num_queue_pairs; i++) {
> + struct virtqueue *vq = vi->sq[i]->vq;
> +
> + while ((buf = virtqueue_detach_unused_buf(vq)) != NULL)
> + dev_kfree_skb(buf);
> + }
> +
> + for (i = 0; i < vi->num_queue_pairs; i++) {
> + struct virtqueue *vq = vi->rq[i]->vq;
> +
> + while ((buf = virtqueue_detach_unused_buf(vq)) != NULL) {
> + if (vi->mergeable_rx_bufs || vi->big_packets)
> + give_pages(vi->rq[i], buf);
> + else
> + dev_kfree_skb(buf);
> + --vi->rq[i]->num;
> + }
> + BUG_ON(vi->rq[i]->num != 0);
> + }
> +}
> +
> +static void virtnet_set_affinity(struct virtnet_info *vi, bool set)
> +{
> + int i;
> +
> + if (vi->num_queue_pairs == 1)
> + return;
> +
> + for (i = 0; i < vi->num_queue_pairs; i++) {
> + int cpu = set ? i : -1;
> + virtqueue_set_affinity(vi->rq[i]->vq, cpu);
> + virtqueue_set_affinity(vi->sq[i]->vq, cpu);
> + }
> + return;
> +}
> +
> +static void virtnet_del_vqs(struct virtnet_info *vi)
> +{
> + struct virtio_device *vdev = vi->vdev;
> +
> + virtnet_set_affinity(vi, false);
> +
> + vdev->config->del_vqs(vdev);
> +
> + virtnet_free_queues(vi);
> +}
> +
> +static int virtnet_find_vqs(struct virtnet_info *vi)
> +{
> + vq_callback_t **callbacks;
> + struct virtqueue **vqs;
> + int ret = -ENOMEM;
> + int i, total_vqs;
> + char **names;
>
> - vi->rvq = vqs[0];
> - vi->svq = vqs[1];
> + /*
> + * We expect 1 RX virtqueue followed by 1 TX virtqueue, followd by
> + * possible control virtqueue and followed by the same
> + * 'vi->num_queue_pairs-1' more times
> + */
> + total_vqs = vi->num_queue_pairs * 2 +
> + virtio_has_feature(vi->vdev, VIRTIO_NET_F_CTRL_VQ);
> +
> + /* Allocate space for find_vqs parameters */
> + vqs = kmalloc(total_vqs * sizeof(*vqs), GFP_KERNEL);
> + callbacks = kmalloc(total_vqs * sizeof(*callbacks), GFP_KERNEL);
> + names = kmalloc(total_vqs * sizeof(*names), GFP_KERNEL);
so this needs to be kzalloc otherwise on an error cleanup will
get uninitialized data and crash?
> + if (!vqs || !callbacks || !names)
> + goto err;
> +
> + /* Parameters for control virtqueue, if any */
> + if (vi->has_cvq) {
> + callbacks[2] = NULL;
> + names[2] = "control";
> + }
> +
> + /* Allocate/initialize parameters for send/receive virtqueues */
> + for (i = 0; i < vi->num_queue_pairs * 2; i += 2) {
> + int j = (i == 0 ? i : i + vi->has_cvq);
> + callbacks[j] = skb_recv_done;
> + callbacks[j + 1] = skb_xmit_done;
> + names[j] = kasprintf(GFP_KERNEL, "input.%d", i / 2);
> + names[j + 1] = kasprintf(GFP_KERNEL, "output.%d", i / 2);
This needs wrappers. E.g. virtnet_rx_vq(int queue_pair), virtnet_tx_vq(int queue_pair);
Then you would just scan 0 to num_queue_pairs, and i is queue pair
number.
> + }
>
> - if (virtio_has_feature(vi->vdev, VIRTIO_NET_F_CTRL_VQ)) {
> + ret = vi->vdev->config->find_vqs(vi->vdev, total_vqs, vqs, callbacks,
> + (const char **)names);
> + if (ret)
> + goto err;
> +
> + if (vi->has_cvq)
> vi->cvq = vqs[2];
>
> - if (virtio_has_feature(vi->vdev, VIRTIO_NET_F_CTRL_VLAN))
> - vi->dev->features |= NETIF_F_HW_VLAN_FILTER;
> + for (i = 0; i < vi->num_queue_pairs * 2; i += 2) {
> + int j = i == 0 ? i : i + vi->has_cvq;
> + vi->rq[i / 2]->vq = vqs[j];
> + vi->sq[i / 2]->vq = vqs[j + 1];
Same here.
> }
> - return 0;
> +
> +err:
> + if (ret && names)
If we are here ret != 0. For names, just add another label, don't
complicate cleanup.
> + for (i = 0; i < vi->num_queue_pairs * 2; i++)
> + kfree(names[i]);
> +
> + kfree(names);
> + kfree(callbacks);
> + kfree(vqs);
> +
> + return ret;
> +}
> +
> +static int virtnet_alloc_queues(struct virtnet_info *vi)
> +{
> + int ret = -ENOMEM;
> + int i;
> +
> + for (i = 0; i < vi->num_queue_pairs; i++) {
> + vi->rq[i] = kzalloc(sizeof(*vi->rq[i]), GFP_KERNEL);
> + vi->sq[i] = kzalloc(sizeof(*vi->sq[i]), GFP_KERNEL);
> + if (!vi->rq[i] || !vi->sq[i])
> + goto err;
> + }
> +
> + ret = 0;
> +
> + /* setup initial receive and send queue parameters */
> + for (i = 0; i < vi->num_queue_pairs; i++) {
> + vi->rq[i]->vi = vi;
> + vi->rq[i]->pages = NULL;
> + INIT_DELAYED_WORK(&vi->rq[i]->refill, refill_work);
> + netif_napi_add(vi->dev, &vi->rq[i]->napi, virtnet_poll,
> + napi_weight);
> +
> + sg_init_table(vi->rq[i]->sg, ARRAY_SIZE(vi->rq[i]->sg));
> + sg_init_table(vi->sq[i]->sg, ARRAY_SIZE(vi->sq[i]->sg));
> + }
> +
Add return 0 here, then ret = 0 will not be needed
above and if (ret) below.
> +err:
> + if (ret)
> + virtnet_free_queues(vi);
> +
> + return ret;
> +}
> +
> +static int virtnet_setup_vqs(struct virtnet_info *vi)
> +{
> + int ret;
> +
> + /* Allocate send & receive queues */
> + ret = virtnet_alloc_queues(vi);
> + if (!ret) {
> + ret = virtnet_find_vqs(vi);
> + if (ret)
> + virtnet_free_queues(vi);
> + else
> + virtnet_set_affinity(vi, true);
> + }
> +
> + return ret;
Add some labels for error handling, this if nesting is messy.
> }
>
> static int virtnet_probe(struct virtio_device *vdev)
> {
> - int err;
> + int i, err;
> struct net_device *dev;
> struct virtnet_info *vi;
> + u16 num_queues, num_queue_pairs;
> +
> + /* Find if host supports multiqueue virtio_net device */
> + err = virtio_config_val(vdev, VIRTIO_NET_F_MULTIQUEUE,
> + offsetof(struct virtio_net_config,
> + num_queues), &num_queues);
> +
> + /* We need atleast 2 queue's */
typo
> + if (err || num_queues < 2)
> + num_queues = 2;
> + if (num_queues > MAX_QUEUES * 2)
> + num_queues = MAX_QUEUES;
> +
> + num_queue_pairs = num_queues / 2;
>
> /* Allocate ourselves a network device with room for our info */
> - dev = alloc_etherdev(sizeof(struct virtnet_info));
> + dev = alloc_etherdev_mq(sizeof(struct virtnet_info), num_queue_pairs);
> if (!dev)
> return -ENOMEM;
>
> @@ -1103,22 +1362,18 @@ static int virtnet_probe(struct virtio_device *vdev)
>
> /* Set up our device-specific information */
> vi = netdev_priv(dev);
> - netif_napi_add(dev, &vi->napi, virtnet_poll, napi_weight);
> vi->dev = dev;
> vi->vdev = vdev;
> vdev->priv = vi;
> - vi->pages = NULL;
> vi->stats = alloc_percpu(struct virtnet_stats);
> err = -ENOMEM;
> if (vi->stats == NULL)
> - goto free;
> + goto free_netdev;
>
> - INIT_DELAYED_WORK(&vi->refill, refill_work);
> mutex_init(&vi->config_lock);
> vi->config_enable = true;
> INIT_WORK(&vi->config_work, virtnet_config_changed_work);
> - sg_init_table(vi->rx_sg, ARRAY_SIZE(vi->rx_sg));
> - sg_init_table(vi->tx_sg, ARRAY_SIZE(vi->tx_sg));
> + vi->num_queue_pairs = num_queue_pairs;
>
> /* If we can receive ANY GSO packets, we must allocate large ones. */
> if (virtio_has_feature(vdev, VIRTIO_NET_F_GUEST_TSO4) ||
> @@ -1129,9 +1384,17 @@ static int virtnet_probe(struct virtio_device *vdev)
> if (virtio_has_feature(vdev, VIRTIO_NET_F_MRG_RXBUF))
> vi->mergeable_rx_bufs = true;
>
> - err = init_vqs(vi);
> + if (virtio_has_feature(vdev, VIRTIO_NET_F_CTRL_VQ))
> + vi->has_cvq = true;
> +
How about we disable multiqueue if there's no cvq?
Will make logic a bit simpler, won't it?
> + /* Allocate/initialize the rx/tx queues, and invoke find_vqs */
> + err = virtnet_setup_vqs(vi);
> if (err)
> - goto free_stats;
> + goto free_netdev;
> +
> + if (virtio_has_feature(vi->vdev, VIRTIO_NET_F_CTRL_VQ) &&
> + virtio_has_feature(vi->vdev, VIRTIO_NET_F_CTRL_VLAN))
> + dev->features |= NETIF_F_HW_VLAN_FILTER;
>
> err = register_netdev(dev);
> if (err) {
> @@ -1140,12 +1403,15 @@ static int virtnet_probe(struct virtio_device *vdev)
> }
>
> /* Last of all, set up some receive buffers. */
> - try_fill_recv(vi, GFP_KERNEL);
> -
> - /* If we didn't even get one input buffer, we're useless. */
> - if (vi->num == 0) {
> - err = -ENOMEM;
> - goto unregister;
> + for (i = 0; i < num_queue_pairs; i++) {
> + try_fill_recv(vi->rq[i], GFP_KERNEL);
> +
> + /* If we didn't even get one input buffer, we're useless. */
> + if (vi->rq[i]->num == 0) {
> + free_unused_bufs(vi);
> + err = -ENOMEM;
> + goto free_recv_bufs;
> + }
> }
>
> /* Assume link up if device can't report link status,
> @@ -1158,42 +1424,25 @@ static int virtnet_probe(struct virtio_device *vdev)
> netif_carrier_on(dev);
> }
>
> - pr_debug("virtnet: registered device %s\n", dev->name);
> + pr_debug("virtnet: registered device %s with %d RX and TX vq's\n",
> + dev->name, num_queue_pairs);
> +
> return 0;
>
> -unregister:
> +free_recv_bufs:
> + free_receive_bufs(vi);
> unregister_netdev(dev);
> +
> free_vqs:
> - vdev->config->del_vqs(vdev);
> -free_stats:
> - free_percpu(vi->stats);
> -free:
> + for (i = 0; i < num_queue_pairs; i++)
> + cancel_delayed_work_sync(&vi->rq[i]->refill);
> + virtnet_del_vqs(vi);
> +
> +free_netdev:
> free_netdev(dev);
> return err;
> }
>
> -static void free_unused_bufs(struct virtnet_info *vi)
> -{
> - void *buf;
> - while (1) {
> - buf = virtqueue_detach_unused_buf(vi->svq);
> - if (!buf)
> - break;
> - dev_kfree_skb(buf);
> - }
> - while (1) {
> - buf = virtqueue_detach_unused_buf(vi->rvq);
> - if (!buf)
> - break;
> - if (vi->mergeable_rx_bufs || vi->big_packets)
> - give_pages(vi, buf);
> - else
> - dev_kfree_skb(buf);
> - --vi->num;
> - }
> - BUG_ON(vi->num != 0);
> -}
> -
> static void remove_vq_common(struct virtnet_info *vi)
> {
> vi->vdev->config->reset(vi->vdev);
> @@ -1201,10 +1450,9 @@ static void remove_vq_common(struct virtnet_info *vi)
> /* Free unused buffers in both send and recv, if any. */
> free_unused_bufs(vi);
>
> - vi->vdev->config->del_vqs(vi->vdev);
> + free_receive_bufs(vi);
>
> - while (vi->pages)
> - __free_pages(get_a_page(vi, GFP_KERNEL), 0);
> + virtnet_del_vqs(vi);
> }
>
> static void __devexit virtnet_remove(struct virtio_device *vdev)
> @@ -1230,6 +1478,7 @@ static void __devexit virtnet_remove(struct virtio_device *vdev)
> static int virtnet_freeze(struct virtio_device *vdev)
> {
> struct virtnet_info *vi = vdev->priv;
> + int i;
>
> /* Prevent config work handler from accessing the device */
> mutex_lock(&vi->config_lock);
> @@ -1237,10 +1486,13 @@ static int virtnet_freeze(struct virtio_device *vdev)
> mutex_unlock(&vi->config_lock);
>
> netif_device_detach(vi->dev);
> - cancel_delayed_work_sync(&vi->refill);
> + for (i = 0; i < vi->num_queue_pairs; i++)
> + cancel_delayed_work_sync(&vi->rq[i]->refill);
>
> if (netif_running(vi->dev))
> - napi_disable(&vi->napi);
> + for (i = 0; i < vi->num_queue_pairs; i++)
> + napi_disable(&vi->rq[i]->napi);
> +
>
> remove_vq_common(vi);
>
> @@ -1252,19 +1504,22 @@ static int virtnet_freeze(struct virtio_device *vdev)
> static int virtnet_restore(struct virtio_device *vdev)
> {
> struct virtnet_info *vi = vdev->priv;
> - int err;
> + int err, i;
>
> - err = init_vqs(vi);
> + err = virtnet_setup_vqs(vi);
> if (err)
> return err;
>
> if (netif_running(vi->dev))
> - virtnet_napi_enable(vi);
> + for (i = 0; i < vi->num_queue_pairs; i++)
> + virtnet_napi_enable(vi->rq[i]);
>
> netif_device_attach(vi->dev);
>
> - if (!try_fill_recv(vi, GFP_KERNEL))
> - queue_delayed_work(system_nrt_wq, &vi->refill, 0);
> + for (i = 0; i < vi->num_queue_pairs; i++)
> + if (!try_fill_recv(vi->rq[i], GFP_KERNEL))
> + queue_delayed_work(system_nrt_wq,
> + &vi->rq[i]->refill, 0);
>
> mutex_lock(&vi->config_lock);
> vi->config_enable = true;
> @@ -1287,7 +1542,7 @@ static unsigned int features[] = {
> VIRTIO_NET_F_GUEST_ECN, VIRTIO_NET_F_GUEST_UFO,
> VIRTIO_NET_F_MRG_RXBUF, VIRTIO_NET_F_STATUS, VIRTIO_NET_F_CTRL_VQ,
> VIRTIO_NET_F_CTRL_RX, VIRTIO_NET_F_CTRL_VLAN,
> - VIRTIO_NET_F_GUEST_ANNOUNCE,
> + VIRTIO_NET_F_GUEST_ANNOUNCE, VIRTIO_NET_F_MULTIQUEUE,
> };
>
> static struct virtio_driver virtio_net_driver = {
> diff --git a/include/linux/virtio_net.h b/include/linux/virtio_net.h
> index 1bc7e30..60f09ff 100644
> --- a/include/linux/virtio_net.h
> +++ b/include/linux/virtio_net.h
> @@ -61,6 +61,8 @@ struct virtio_net_config {
> __u8 mac[6];
> /* See VIRTIO_NET_F_STATUS and VIRTIO_NET_S_* above */
> __u16 status;
> + /* Total number of RX/TX queues */
> + __u16 num_queues;
> } __attribute__((packed));
>
> /* This is the first element of the scatter-gather list. If you don't
> --
> 1.7.1
^ permalink raw reply
* RE: 0xB16B00B5? Really? (was Re: Move hyperv out of the drivers/staging/ directory)
From: KY Srinivasan @ 2012-07-20 14:00 UTC (permalink / raw)
To: Bjørn Mork
Cc: Greg KH (gregkh@linuxfoundation.org), Paolo Bonzini,
devel@linuxdriverproject.org, linux-kernel@vger.kernel.org,
virtualization@lists.osdl.org
In-Reply-To: <87fw8mlv7e.fsf@nemi.mork.no>
> -----Original Message-----
> From: Bjørn Mork [mailto:bjorn@mork.no]
> Sent: Friday, July 20, 2012 3:59 AM
> To: KY Srinivasan
> Cc: Greg KH (gregkh@linuxfoundation.org); Paolo Bonzini;
> devel@linuxdriverproject.org; linux-kernel@vger.kernel.org;
> virtualization@lists.osdl.org
> Subject: Re: 0xB16B00B5? Really? (was Re: Move hyperv out of the
> drivers/staging/ directory)
>
> KY Srinivasan <kys@microsoft.com> writes:
>
> > Here is the link that describes how the guest ID should be composed:
> >
> > http://msdn.microsoft.com/en-
> us/library/windows/hardware/ff542653%28v=vs.85%29.aspx
>
> The unwanted value does not conform to those rules: Linux is not yet at
> major version 0xB5.
>
> I propose that you change this into a macro actually taking the OS
> major/minor version into account, keeping the 0xB16B vendor ID (which
> seems safest as the vendor ID registration process seems a bit unclear)
Thank you for your interest in fixing this problem. When we decide to change this
ID, we will conform to the MSFT guidelines on constructing this guest ID.
Regards,
K. Y
^ permalink raw reply
* Re: 0xB16B00B5? Really? (was Re: Move hyperv out of the drivers/staging/ directory)
From: richard -rw- weinberger @ 2012-07-20 15:03 UTC (permalink / raw)
To: KY Srinivasan
Cc: Bjørn Mork, Greg KH (gregkh@linuxfoundation.org),
Paolo Bonzini, devel@linuxdriverproject.org,
linux-kernel@vger.kernel.org, virtualization@lists.osdl.org
In-Reply-To: <426367E2313C2449837CD2DE46E7EAF923547782@SN2PRD0310MB382.namprd03.prod.outlook.com>
On Fri, Jul 20, 2012 at 4:00 PM, KY Srinivasan <kys@microsoft.com> wrote:
> Thank you for your interest in fixing this problem. When we decide to change this
> ID, we will conform to the MSFT guidelines on constructing this guest ID.
>
I'm wondering why it hasn't been conform to the MSFT guidelines from
the very beginning on?
--
Thanks,
//richard
^ permalink raw reply
* RE: 0xB16B00B5? Really? (was Re: Move hyperv out of the drivers/staging/ directory)
From: KY Srinivasan @ 2012-07-20 15:16 UTC (permalink / raw)
To: richard -rw- weinberger
Cc: Bjørn Mork, Greg KH (gregkh@linuxfoundation.org),
Paolo Bonzini, devel@linuxdriverproject.org,
linux-kernel@vger.kernel.org, virtualization@lists.osdl.org
In-Reply-To: <CAFLxGvwqKLhPWNfUJYm1KDXWEX1DNhPBGGApYFqMM0BsB87iig@mail.gmail.com>
> -----Original Message-----
> From: richard -rw- weinberger [mailto:richard.weinberger@gmail.com]
> Sent: Friday, July 20, 2012 11:04 AM
> To: KY Srinivasan
> Cc: Bjørn Mork; Greg KH (gregkh@linuxfoundation.org); Paolo Bonzini;
> devel@linuxdriverproject.org; linux-kernel@vger.kernel.org;
> virtualization@lists.osdl.org
> Subject: Re: 0xB16B00B5? Really? (was Re: Move hyperv out of the
> drivers/staging/ directory)
>
> On Fri, Jul 20, 2012 at 4:00 PM, KY Srinivasan <kys@microsoft.com> wrote:
> > Thank you for your interest in fixing this problem. When we decide to change
> this
> > ID, we will conform to the MSFT guidelines on constructing this guest ID.
> >
>
> I'm wondering why it hasn't been conform to the MSFT guidelines from
> the very beginning on?
Great question; this was before my time in MSFT and so I cannot answer that.
Regards,
^ permalink raw reply
* Re: [PATCH] drivers: hv: use Linux version in guest ID
From: Greg KH @ 2012-07-20 15:26 UTC (permalink / raw)
To: Bjørn Mork
Cc: KY Srinivasan, Paolo Bonzini, devel, linux-kernel, virtualization
In-Reply-To: <1342771388-24389-1-git-send-email-bjorn@mork.no>
On Fri, Jul 20, 2012 at 10:03:08AM +0200, Bjørn Mork wrote:
> Use OS version number in the guest ID as recommended by Microsoft
>
> Signed-off-by: Bjørn Mork <bjorn@mork.no>
> ---
> So how about something like this? Yes, I know including
> linux/version.h often is unwanted, but the spec does
> recommend using the actual OS version in the guest ID
>
> Yes, this will be offensive again when Linux is at
> version 181.0, but we might have found a better solution by
> then :-)
>
>
> drivers/hv/hyperv_vmbus.h | 20 ++++++++++++++++----
> 1 file changed, 16 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
> index b9426a6..cc0c3f0 100644
> --- a/drivers/hv/hyperv_vmbus.h
> +++ b/drivers/hv/hyperv_vmbus.h
> @@ -29,6 +29,7 @@
> #include <asm/sync_bitops.h>
> #include <linux/atomic.h>
> #include <linux/hyperv.h>
> +#include <linux/version.h>
>
> /*
> * The below CPUID leaves are present if VersionAndFeatures.HypervisorPresent
> @@ -410,10 +411,21 @@ enum {
>
> #define HV_PRESENT_BIT 0x80000000
>
> -#define HV_LINUX_GUEST_ID_LO 0x00000000
> -#define HV_LINUX_GUEST_ID_HI 0xB16B00B5
> -#define HV_LINUX_GUEST_ID (((u64)HV_LINUX_GUEST_ID_HI << 32) | \
> - HV_LINUX_GUEST_ID_LO)
> +/* Linux vendor ID allocated by Microsoft */
> +#define HV_LINUX_GUEST_ID_VENDOR 0xB16B
> +
> +/* Creating a guest ID conforming to the encoding recommended by
> + * Microsoft, with variant and build fixed to 0
We could use the kernel build number here as well, if we really wanted
it, right?
Otherwise, I like this patch, KY, any objection to me taking it?
thanks,
greg k-h
^ permalink raw reply
* RE: [PATCH] drivers: hv: use Linux version in guest ID
From: KY Srinivasan @ 2012-07-20 16:02 UTC (permalink / raw)
To: Greg KH, Bjørn Mork
Cc: Paolo Bonzini, devel@linuxdriverproject.org,
linux-kernel@vger.kernel.org, virtualization@lists.osdl.org
In-Reply-To: <20120720152650.GA5201@kroah.com>
> -----Original Message-----
> From: Greg KH [mailto:gregkh@linuxfoundation.org]
> Sent: Friday, July 20, 2012 11:27 AM
> To: Bjørn Mork
> Cc: KY Srinivasan; Paolo Bonzini; devel@linuxdriverproject.org; linux-
> kernel@vger.kernel.org; virtualization@lists.osdl.org
> Subject: Re: [PATCH] drivers: hv: use Linux version in guest ID
>
> On Fri, Jul 20, 2012 at 10:03:08AM +0200, Bjørn Mork wrote:
> > Use OS version number in the guest ID as recommended by Microsoft
> >
> > Signed-off-by: Bjørn Mork <bjorn@mork.no>
> > ---
> > So how about something like this? Yes, I know including
> > linux/version.h often is unwanted, but the spec does
> > recommend using the actual OS version in the guest ID
> >
> > Yes, this will be offensive again when Linux is at
> > version 181.0, but we might have found a better solution by
> > then :-)
> >
> >
> > drivers/hv/hyperv_vmbus.h | 20 ++++++++++++++++----
> > 1 file changed, 16 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
> > index b9426a6..cc0c3f0 100644
> > --- a/drivers/hv/hyperv_vmbus.h
> > +++ b/drivers/hv/hyperv_vmbus.h
> > @@ -29,6 +29,7 @@
> > #include <asm/sync_bitops.h>
> > #include <linux/atomic.h>
> > #include <linux/hyperv.h>
> > +#include <linux/version.h>
> >
> > /*
> > * The below CPUID leaves are present if
> VersionAndFeatures.HypervisorPresent
> > @@ -410,10 +411,21 @@ enum {
> >
> > #define HV_PRESENT_BIT 0x80000000
> >
> > -#define HV_LINUX_GUEST_ID_LO 0x00000000
> > -#define HV_LINUX_GUEST_ID_HI 0xB16B00B5
> > -#define HV_LINUX_GUEST_ID (((u64)HV_LINUX_GUEST_ID_HI
> << 32) | \
> > - HV_LINUX_GUEST_ID_LO)
> > +/* Linux vendor ID allocated by Microsoft */
> > +#define HV_LINUX_GUEST_ID_VENDOR 0xB16B
> > +
> > +/* Creating a guest ID conforming to the encoding recommended by
> > + * Microsoft, with variant and build fixed to 0
>
> We could use the kernel build number here as well, if we really wanted
> it, right?
>
> Otherwise, I like this patch, KY, any objection to me taking it?
Greg,
If it is ok with you can we wait a couple of days. I am pushing to get information
back from folks within MSFT. I am hoping we can fix this issue soon.
Regards,
K. Y
^ permalink raw reply
* Re: [RFC-v3 4/4] tcm_vhost: Initial merge for vhost level target fabric driver
From: Nicholas A. Bellinger @ 2012-07-20 18:00 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Jens Axboe, Stefan Hajnoczi, kvm-devel, Greg Kroah-Hartman,
Zhi Yong Wu, Anthony Liguori, target-devel, linux-scsi,
Paolo Bonzini, lf-virt, Christoph Hellwig
In-Reply-To: <20120720120347.GA16550@redhat.com>
On Fri, 2012-07-20 at 15:03 +0300, Michael S. Tsirkin wrote:
> On Wed, Jul 18, 2012 at 02:20:58PM -0700, Nicholas A. Bellinger wrote:
> > On Wed, 2012-07-18 at 19:09 +0300, Michael S. Tsirkin wrote:
> > > On Wed, Jul 18, 2012 at 12:59:32AM +0000, Nicholas A. Bellinger wrote:
> >
> > <SNIP>
> >
> > > >
> > > > Changelog v2 -> v3:
> > > >
> > > > Unlock on error in tcm_vhost_drop_nexus() (DanC)
> > > > Fix strlen() doesn't count the terminator (DanC)
> > > > Call kfree() on an error path (DanC)
> > > > Convert tcm_vhost_write_pending to use target_execute_cmd (hch + nab)
> > > > Fix another strlen() off by one in tcm_vhost_make_tport (DanC)
> > > > Add option under drivers/staging/Kconfig, and move to drivers/vhost/tcm/
> > > > as requested by MST (nab)
> > > >
> > > > ---
> > > > drivers/staging/Kconfig | 2 +
> > > > drivers/vhost/Makefile | 2 +
> > > > drivers/vhost/tcm/Kconfig | 6 +
> > > > drivers/vhost/tcm/Makefile | 1 +
> > > > drivers/vhost/tcm/tcm_vhost.c | 1611 +++++++++++++++++++++++++++++++++++++++++
> > > > drivers/vhost/tcm/tcm_vhost.h | 74 ++
> > > > 6 files changed, 1696 insertions(+), 0 deletions(-)
> > > > create mode 100644 drivers/vhost/tcm/Kconfig
> > > > create mode 100644 drivers/vhost/tcm/Makefile
> > > > create mode 100644 drivers/vhost/tcm/tcm_vhost.c
> > > > create mode 100644 drivers/vhost/tcm/tcm_vhost.h
> > > >
> > >
> > > Really sorry about making you run around like that,
> > > I did not mean moving all of tcm to a directory,
> > > just adding tcm/Kconfig or adding drivers/vhost/Kconfig.tcm
> > > because eventually it's easier to keep it all together
> > > in one place.
> > >
> >
> > Er, apologies for the slight mis-understanding here.. Moving back now +
> > fixing up the Kbuild bits.
>
> I'm going offline in several hours and am on vacation for a week
> starting tomorrow. So to make 3.6, and if you intend to merge through my
> tree, the best bet is if you can send the final version real soon now.
>
Ok, thanks for the heads up here..
So aside from Greg-KH's feedback to avoid the drivers/staging/ Kconfig
include usage, and one more bugfix from DanC from this morning those are
the only pending changes for RFC-v4.
If it's OK I'd prefer to take these via target-pending with the
necessary Acked-By's, especially if you'll be AFK next week..
Would you like to see a RFC-v4 with these changes included..?
Thank you,
--nab
^ permalink raw reply
* Re: [RFC-v3 4/4] tcm_vhost: Initial merge for vhost level target fabric driver
From: Nicholas A. Bellinger @ 2012-07-20 18:02 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Jens Axboe, Stefan Hajnoczi, kvm-devel, Greg Kroah-Hartman,
Zhi Yong Wu, Anthony Liguori, target-devel, linux-scsi,
Paolo Bonzini, lf-virt, Christoph Hellwig
In-Reply-To: <1342807247.25472.9.camel@haakon2.linux-iscsi.org>
On Fri, 2012-07-20 at 11:00 -0700, Nicholas A. Bellinger wrote:
> On Fri, 2012-07-20 at 15:03 +0300, Michael S. Tsirkin wrote:
> > On Wed, Jul 18, 2012 at 02:20:58PM -0700, Nicholas A. Bellinger wrote:
> > > On Wed, 2012-07-18 at 19:09 +0300, Michael S. Tsirkin wrote:
> > > > On Wed, Jul 18, 2012 at 12:59:32AM +0000, Nicholas A. Bellinger wrote:
> > >
> > > <SNIP>
> > >
> > > > >
> > > > > Changelog v2 -> v3:
> > > > >
> > > > > Unlock on error in tcm_vhost_drop_nexus() (DanC)
> > > > > Fix strlen() doesn't count the terminator (DanC)
> > > > > Call kfree() on an error path (DanC)
> > > > > Convert tcm_vhost_write_pending to use target_execute_cmd (hch + nab)
> > > > > Fix another strlen() off by one in tcm_vhost_make_tport (DanC)
> > > > > Add option under drivers/staging/Kconfig, and move to drivers/vhost/tcm/
> > > > > as requested by MST (nab)
> > > > >
> > > > > ---
> > > > > drivers/staging/Kconfig | 2 +
> > > > > drivers/vhost/Makefile | 2 +
> > > > > drivers/vhost/tcm/Kconfig | 6 +
> > > > > drivers/vhost/tcm/Makefile | 1 +
> > > > > drivers/vhost/tcm/tcm_vhost.c | 1611 +++++++++++++++++++++++++++++++++++++++++
> > > > > drivers/vhost/tcm/tcm_vhost.h | 74 ++
> > > > > 6 files changed, 1696 insertions(+), 0 deletions(-)
> > > > > create mode 100644 drivers/vhost/tcm/Kconfig
> > > > > create mode 100644 drivers/vhost/tcm/Makefile
> > > > > create mode 100644 drivers/vhost/tcm/tcm_vhost.c
> > > > > create mode 100644 drivers/vhost/tcm/tcm_vhost.h
> > > > >
> > > >
> > > > Really sorry about making you run around like that,
> > > > I did not mean moving all of tcm to a directory,
> > > > just adding tcm/Kconfig or adding drivers/vhost/Kconfig.tcm
> > > > because eventually it's easier to keep it all together
> > > > in one place.
> > > >
> > >
> > > Er, apologies for the slight mis-understanding here.. Moving back now +
> > > fixing up the Kbuild bits.
> >
> > I'm going offline in several hours and am on vacation for a week
> > starting tomorrow. So to make 3.6, and if you intend to merge through my
> > tree, the best bet is if you can send the final version real soon now.
> >
>
> Ok, thanks for the heads up here..
>
> So aside from Greg-KH's feedback to avoid the drivers/staging/ Kconfig
> include usage, and one more bugfix from DanC from this morning those are
> the only pending changes for RFC-v4.
>
> If it's OK I'd prefer to take these via target-pending with the
> necessary Acked-By's, especially if you'll be AFK next week..
>
> Would you like to see a RFC-v4 with these changes included..?
>
> Thank you,
>
Actually sorry, the patch from DanC is for target core, and not
tcm_vhost specific change..
So really the only thing left to resolve for an initial merge is
Greg-KH's comments wrt to drivers/staging Kconfig usage..
Are you OK with just adding CONFIG_STAGING following Greg-KH's
feedback..?
^ permalink raw reply
* Re: [PATCH RESEND 0/5] Add vhost-blk support
From: Michael S. Tsirkin @ 2012-07-20 19:30 UTC (permalink / raw)
To: Asias He
Cc: linux-aio, kvm, gregkh, linux-kernel, virtualization,
James Bottomley, Jeff Moyer, Benjamin LaHaise, Alexander Viro,
linux-fsdevel
In-Reply-To: <1342169711-12386-1-git-send-email-asias@redhat.com>
On Fri, Jul 13, 2012 at 04:55:06PM +0800, Asias He wrote:
>
> Hi folks,
>
> [I am resending to fix the broken thread in the previous one.]
>
> This patchset adds vhost-blk support. vhost-blk is a in kernel virito-blk
> device accelerator. Compared to userspace virtio-blk implementation, vhost-blk
> gives about 5% to 15% performance improvement.
>
> Asias He (5):
> aio: Export symbols and struct kiocb_batch for in kernel aio usage
> eventfd: Export symbol eventfd_file_create()
> vhost: Make vhost a separate module
> vhost-net: Use VHOST_NET_FEATURES for vhost-net
> vhost-blk: Add vhost-blk support
OK so given the state it's in, and assuming you think it is helpful
to let it mature in tree and not out of tree, I think it's
reasonable to try to do it like tcm_vhost is going to do it:
- send me changes to vhost core ASAP (and keep it minimal, e.g.
use your own header file to export to userspace)
- for other stuff - put in drivers/staging, and ask Greg to merge
--
MST
^ permalink raw reply
* Re: [PATCH v4 1/3] mm: introduce compaction and migration for virtio ballooned pages
From: Rafael Aquini @ 2012-07-20 19:48 UTC (permalink / raw)
To: Minchan Kim
Cc: Rik van Riel, Konrad Rzeszutek Wilk, Michael S. Tsirkin,
linux-kernel, virtualization, linux-mm, Andi Kleen, Andrew Morton,
Rafael Aquini
In-Reply-To: <20120718054824.GA32341@bbox>
Howdy Minchan,
Once again, thanks for raising such valuable feedback over here.
On Wed, Jul 18, 2012 at 02:48:24PM +0900, Minchan Kim wrote:
> > +/* __isolate_lru_page() counterpart for a ballooned page */
> > +static bool isolate_balloon_page(struct page *page)
> > +{
> > + if (WARN_ON(!is_balloon_page(page)))
> > + return false;
>
> I am not sure we need this because you alreay check it before calling
> isolate_balloon_page. If you really need it, it would be better to
> add likely in isolate_balloon_page, too.
>
This check point was set in place because isolate_balloon_page() was a publicly
visible function and while our current usage looks correct it would not hurt to
have something like that done -- think of it as an insurance policy, in case
someone else, in the future, attempts to use it on any other place outside this
specifc context.
Despite not seeing it as a dealbreaker for the patch as is, I do agree, however,
this snippet can _potentially_ be removed from isolate_balloon_page(), since
this function has become static to compaction.c.
> > +
> > + if (likely(get_page_unless_zero(page))) {
> > + /*
> > + * We can race against move_to_new_page() & __unmap_and_move().
> > + * If we stumble across a locked balloon page and succeed on
> > + * isolating it, the result tends to be disastrous.
> > + */
> > + if (likely(trylock_page(page))) {
>
> Hmm, I can't understand your comment.
> What does this lock protect? Could you elaborate it with code sequence?
>
As we are coping with a corner case -- balloon pages are not on LRU lists --
compaction concurrency can lead to a disastrous race which results in
isolate_balloon_page() re-isolating already isolated balloon pages, or isolating
a 'newpage' recently promoted to 'balloon page', while these pages are still
under migration. The only way we have to prevent that from happening is
attempting to grab the page lock, as pages under migration are already locked.
I had that comment rephrased to what follows (please, tell me how it sounds to
you now)
if (likely(get_page_unless_zero(page))) {
/*
- * We can race against move_to_new_page() & __unmap_and_move().
- * If we stumble across a locked balloon page and succeed on
- * isolating it, the result tends to be disastrous.
+ * As balloon pages are not isolated from LRU lists, several
+ * concurrent compaction threads will potentially race against
+ * page migration at move_to_new_page() & __unmap_and_move().
+ * In order to avoid having an already isolated balloon page
+ * being (wrongly) re-isolated while it is under migration,
+ * lets be sure we have the page lock before proceeding with
+ * the balloon page isolation steps.
*/
if (likely(trylock_page(page))) {
/*
> > +/* putback_lru_page() counterpart for a ballooned page */
> > +bool putback_balloon_page(struct page *page)
> > +{
> > + if (WARN_ON(!is_balloon_page(page)))
> > + return false;
>
> You already check WARN_ON in putback_lru_pages so we don't need it in here.
> And you can add likely in here, too.
>
This check point is in place by the same reason why it is for
isolate_balloon_page(). However, I don't think we should drop it here because
putback_balloon_page() is still a publicly visible symbol. Besides, the check
that is performed at putback_lru_pages() level has a different meaning, which is
to warn us when we fail on re-inserting an isolated (but not migrated) page back
to the balloon page list, thus it does not superceeds nor it's superceeded by
this checkpoint here.
> > + } else if (is_balloon_page(page)) {
>
> unlikely?
This can be done, for sure. Also, it reminds me that I had a
'likely(PageLRU(page))' set in place for this vary same patch, on v2 submission.
Do you recollect you've asked me to get rid of it?. Wouldn't it be better having
that suggestion of yours reverted, since PageLRU(page) is the likelihood case
here anyway? What about this?
" if (likely(PageLRU(page))) {
...
} else if (unlikely(is_balloon_page(page))) {
...
} else
continue;
"
>
> > @@ -78,7 +78,10 @@ void putback_lru_pages(struct list_head *l)
> > list_del(&page->lru);
> > dec_zone_page_state(page, NR_ISOLATED_ANON +
> > page_is_file_cache(page));
> > - putback_lru_page(page);
> > + if (unlikely(is_balloon_page(page)))
> > + WARN_ON(!putback_balloon_page(page));
> > + else
> > + putback_lru_page(page);
>
> Hmm, I don't like this.
> The function name says we putback *lru* pages, but balloon page isn't.
> How about this?
>
> diff --git a/mm/compaction.c b/mm/compaction.c
> index aad0a16..b07cd67 100644
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -298,6 +298,8 @@ static bool too_many_isolated(struct zone *zone)
> * Apart from cc->migratepages and cc->nr_migratetypes this function
> * does not modify any cc's fields, in particular it does not modify
> * (or read for that matter) cc->migrate_pfn.
> + *
> + * For returning page, you should use putback_pages instead of putback_lru_pages
> */
> unsigned long
> isolate_migratepages_range(struct zone *zone, struct compact_control *cc,
> @@ -827,7 +829,7 @@ static int compact_zone(struct zone *zone, struct compact_control *cc)
>
> /* Release LRU pages not migrated */
> if (err) {
> - putback_lru_pages(&cc->migratepages);
> + putback_pages(&cc->migratepages);
> cc->nr_migratepages = 0;
> if (err == -ENOMEM) {
> ret = COMPACT_PARTIAL;
> diff --git a/mm/migrate.c b/mm/migrate.c
> index 9705e70..a96b840 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -86,6 +86,22 @@ void putback_lru_pages(struct list_head *l)
> }
> }
>
> + /* blah blah .... */
> +void putback_pages(struct list_head *l)
> +{
> + struct page *page;
> + struct page *page2;
> +
> + list_for_each_entry_safe(page, page2, l, lru) {
> + list_del(&page->lru);
> + dec_zone_page_state(page, NR_ISOLATED_ANON +
> + page_is_file_cache(page));
> + if (unlikely(is_balloon_page(page)))
> + WARN_ON(!putback_balloon_page(page));
> + else
> + putback_lru_page(page);
> + }
> +}
> +
> /*
> * Restore a potential migration pte to a working pte entry
> */
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 32985dd..decb82a 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -5655,7 +5655,7 @@ static int __alloc_contig_migrate_range(unsigned long start, unsigned long end)
> 0, false, MIGRATE_SYNC);
> }
>
> - putback_lru_pages(&cc.migratepages);
> + putback_pages(&cc.migratepages);
> return ret > 0 ? 0 : ret;
> }
>
Despite being a very nice cleanup, this code refactoring has nothing to do with
this particular patch purpose. For the sake of this implementation, think about
the balloon page list acting as a particular LRU, so although ballooned pages
are not enlisted on real LRUs, per se, this doesn't mean we cannot have them
dealt as a corner case amongst putback_lru_pages() code for the sake of
simplicity and maintainability ease. OTOH, I do agree with your nice suggestion,
thus I can submit it as a separate clean-up patch attached to this series (if
you don't mind).
> >
> > + if (is_balloon_page(page)) {
>
> unlikely?
>
Yeah, why not? Will do it.
> > + if (is_balloon_page(newpage)) {
>
> unlikely?
>
ditto.
> > + /*
> > + * A ballooned page has been migrated already. Now, it is the
> > + * time to wrap-up counters, handle the old page back to Buddy
> > + * and return.
> > + */
> > + list_del(&page->lru);
> > + dec_zone_page_state(page, NR_ISOLATED_ANON +
> > + page_is_file_cache(page));
> > + put_page(page);
> > + __free_page(page);
>
> Why do you use __free_page instead of put_page?
>
Do you mean perform an extra put_page() and let putback_lru_page() do the rest?
Besides looking odd at code level, what other improvement we can potentially gain here?
> The feeling I look at your code in detail is that it makes compaction/migration
> code rather complicated because compaction/migration assumes source/target would
> be LRU pages.
>
> How often memory ballooning happens? Does it make sense to hook it in generic
> functions if it's very rare?
>
> Couldn't you implement it like huge page?
> It doesn't make current code complicated and doesn't affect performance
>
> In compaction,
> #ifdef CONFIG_VIRTIO_BALLOON
> static int compact_zone(struct zone *zone, struct compact_control *cc, bool balloon)
> {
> if (balloon) {
> isolate_balloonpages
>
> migrate_balloon_pages
> unmap_and_move_balloon_page
>
> putback_balloonpages
> }
> }
> #endif
>
> I'm not sure memory ballooning so it might be dumb question.
> Can we trigger it once only when ballooning happens?
I strongly believe, this is the simplest and easiest way to get this task
accomplished, mostly becasue:
i. It does not require any code duplication at all;
ii. It requires very few code insertion/surgery to be fully operative;
iii. It is embeded on already well established and maintained code;
iv. The approach makes easier to other balloon drivers leverage compaction
code;
v. It shows no significative impact to the entangled code paths;
It took me a little longer to address all your good questions because I
collected some data on this series impact for the ordinary average use case
(bare-metal boxes with no balloon at all). Here are some very simple numbers,
for the sake of comparison:
* Benchmark: "echo 1 > /proc/sys/vm/compact_memory" after a fresh reboot.
* Results represent the average of 4 sampling rounds for each test subject.
=== 3.5.0-rc7 (clean) ===
Measured overhead (perf record):
Events: 304 cycles
4.44% test.sh [kernel.kallsyms] [k] isolate_migratepages_range
1.69% test.sh [kernel.kallsyms] [k] migrate_pages
Performance counter stats for './test.sh':
314.794120 task-clock # 0.913 CPUs utilized
34 context-switches # 0.110 K/sec
1 CPU-migrations # 0.004 K/sec
354 page-faults # 0.001 M/sec
678,772,513 cycles # 2.173 GHz [49.92%]
<not supported> stalled-cycles-frontend
<not supported> stalled-cycles-backend
402,180,271 instructions # 0.59 insns per cycle [74.85%]
72,746,956 branches # 231.328 M/sec [75.05%]
455,274 branch-misses # 0.62% of all branches [75.13%]
0.344474338 seconds time elapsed
Compaction/Migration observed throughput:
-----------------------------------------
compact_blocks_moved 2988
compact_pages_moved 42219
compact_pagemigrate_failed 5996
compact_stall 0
compact_fail 0
compact_success 0
========================================
=== 3.5.0-rc7 (patch) ===
Measured overhead (perf record):
Events: 336 cycles
4.73% test.sh [kernel.kallsyms] [k] isolate_migratepages_range
1.77% test.sh [kernel.kallsyms] [k] migrate_pages
Performance counter stats for './test.sh':
369.363248 task-clock # 0.913 CPUs utilized
40 context-switches # 0.110 K/sec
1 CPU-migrations # 0.004 K/sec
354 page-faults # 0.001 K/sec
790,381,297 cycles # 2.173 GHz [49.94%]
<not supported> stalled-cycles-frontend
<not supported> stalled-cycles-backend
451,900,192 instructions # 0.59 insns per cycle [74.95%]
81,679,252 branches # 222.352 M/sec [74.99%]
531,265 branch-misses # 0.65% of all branches [75.12%]
0.404462151 seconds time elapsed
Compaction/Migration observed throughput:
-----------------------------------------
compact_blocks_moved 3287
compact_pages_moved 51794
compact_pagemigrate_failed 5971
compact_stall 0
compact_fail 0
compact_success 0
========================================
Despite the patched case apparently showing some low overhead, the numbers, in
proportion, are virtually the same, as one may notice.
In fact, I believe that applying your suggestions on better hinting the compiler
about the branch predictability can potentially turn this change into a win-win
deal.
Best regards!
-- Rafael
^ permalink raw reply
* Re: [PATCH RESEND 5/5] vhost-blk: Add vhost-blk support
From: Anthony Liguori @ 2012-07-20 20:56 UTC (permalink / raw)
To: Michael S. Tsirkin; +Cc: linux-kernel, kvm, virtualization
In-Reply-To: <20120719130957.GD9303@redhat.com>
"Michael S. Tsirkin" <mst@redhat.com> writes:
> On Thu, Jul 19, 2012 at 08:05:42AM -0500, Anthony Liguori wrote:
>> Of course, the million dollar question is why would using AIO in the
>> kernel be faster than using AIO in userspace?
>
> Actually for me a more important question is how does it compare
> with virtio-blk dataplane?
I'm not even asking for a benchmark comparision. It's the same API
being called from a kernel thread vs. a userspace thread. Why would
there be a 60% performance difference between the two? That doesn't
make any sense.
There's got to be a better justification for putting this in the kernel
than just that we can.
I completely understand why Christoph's suggestion of submitting BIOs
directly would be faster. There's no way to do that in userspace.
Regards,
Anthony Liguori
>
> --
> MST
^ permalink raw reply
* Re: [PATCH RESEND 5/5] vhost-blk: Add vhost-blk support
From: Asias He @ 2012-07-21 1:07 UTC (permalink / raw)
To: Anthony Liguori; +Cc: virtualization, linux-kernel, kvm, Michael S. Tsirkin
In-Reply-To: <87obnaup70.fsf@codemonkey.ws>
On 07/21/2012 04:56 AM, Anthony Liguori wrote:
> "Michael S. Tsirkin" <mst@redhat.com> writes:
>
>> On Thu, Jul 19, 2012 at 08:05:42AM -0500, Anthony Liguori wrote:
>>> Of course, the million dollar question is why would using AIO in the
>>> kernel be faster than using AIO in userspace?
>>
>> Actually for me a more important question is how does it compare
>> with virtio-blk dataplane?
>
> I'm not even asking for a benchmark comparision. It's the same API
> being called from a kernel thread vs. a userspace thread. Why would
> there be a 60% performance difference between the two? That doesn't
> make any sense.
Please read the commit log again. I am not saying vhost-blk v.s
userspace implementation gives 60% improvement. I am saying the
vhost-blk v.s original vhost-blk gives 60% improvement.
"""
This patch is based on Liu Yuan's implementation with various
improvements and bug fixes. Notably, this patch makes guest notify and
host completion processing in parallel which gives about 60% performance
improvement compared to Liu Yuan's implementation.
"""
>
> There's got to be a better justification for putting this in the kernel
> than just that we can.
>
> I completely understand why Christoph's suggestion of submitting BIOs
> directly would be faster. There's no way to do that in userspace.
Well. With Zach and Dave's new in-kernel aio API, the aio usage in
kernel is much simpler than in userspace. This a potential reason that
in kernel one is better than userspace one. I am working on it right
now. And for block based image, as suggested by Christoph, we can submit
bio directly. This is another potential reason.
Why can't we just go further to see if we can improve the IO stack from
guest kernel side all the way down to host kernel side. We can not do
that if we stick to doing everything in userspace (qemu).
--
Asias
^ permalink raw reply
* [RFC-v4 0/3] tcm_vhost+cmwq fabric driver code for-3.6
From: Nicholas A. Bellinger @ 2012-07-21 6:55 UTC (permalink / raw)
To: target-devel
Cc: Jens Axboe, Stefan Hajnoczi, kvm-devel, Michael S. Tsirkin,
Greg Kroah-Hartman, Zhi Yong Wu, Anthony Liguori, linux-scsi,
Paolo Bonzini, lf-virt, Christoph Hellwig
From: Nicholas Bellinger <nab@linux-iscsi.org>
Hi MST, Greg-KH & Co,
The following is -v4 of the in-flight TCM vhost fabric driver for-3.6 code.
This series has been rebased into target-pending.git/for-next-merge this
evening, and the changelog over the last days from v3 -> v4 has been:
*) Rename vhost_vring_target -> vhost_scsi_target (mst + nab)
*) Use TRANSPORT_IQN_LEN in vhost_scsi_target->vhost_wwpn[] def (nab)
*) Move back to drivers/vhost/, and just use drivers/vhost/Kconfig.tcm (mst)
*) Move TCM_VHOST related ioctl defines from include/linux/vhost.h ->
drivers/vhost/tcm_vhost.h as requested by MST (nab)
*) Move Kbuild.tcm include back from drivers/staging -> drivers/vhost/, and
just use 'if STAGING' around 'source drivers/vhost/Kbuild.tcm'
This series uses Greg-KH's last recommendation from the linux-next thread
to just KISS + use the STAGING bit for this driver short of moving wholesale
into drivers/staging/tcm_vhost/.
As mentioned in the same thread, MST would like to see this in staging so we
don't have to commit to a ABI for QEMU userspace just yet. Can we agree that
the STAGING bit usage here is enough to mark this code as staging for users..?
Pretty please..?
The reason that -v4 currently avoids a drivers/staging/tcm_vhost/ move is to
prevent the staging tree needing to merge vhost + target-pending/for-next for
tcm_vhost build requirements. It is easier to merge via target-pending w/ the
necessary ACKs for the drivers/vhost/ bits, but I'm fine with posting a -v5
series ASAP to move this code into drivers/staging/tcm_vhost + let staging
handle the necessary vhost + target merge dependencies.
I'll let the staging folks (Greg-KH..?) decided if they are OK with the
extra tree merges here before nominating them. ;)
So aside from the 'marked as staging' directory location back <-> forth,
please consider ACK'ing the main bits for an initial merge.
Thank you!
Nicholas Bellinger (1):
tcm_vhost: Initial merge for vhost level target fabric driver
Stefan Hajnoczi (2):
vhost: Separate vhost-net features from vhost features
vhost: make vhost work queue visible
drivers/vhost/Kconfig | 3 +
drivers/vhost/Kconfig.tcm | 6 +
drivers/vhost/Makefile | 2 +
drivers/vhost/net.c | 4 +-
drivers/vhost/tcm_vhost.c | 1611 +++++++++++++++++++++++++++++++++++++++++++++
drivers/vhost/tcm_vhost.h | 90 +++
drivers/vhost/test.c | 4 +-
drivers/vhost/vhost.c | 5 +-
drivers/vhost/vhost.h | 6 +-
9 files changed, 1723 insertions(+), 8 deletions(-)
create mode 100644 drivers/vhost/Kconfig.tcm
create mode 100644 drivers/vhost/tcm_vhost.c
create mode 100644 drivers/vhost/tcm_vhost.h
--
1.7.2.5
^ permalink raw reply
* [RFC-v4 1/3] vhost: Separate vhost-net features from vhost features
From: Nicholas A. Bellinger @ 2012-07-21 6:55 UTC (permalink / raw)
To: target-devel
Cc: Jens Axboe, Stefan Hajnoczi, kvm-devel, Michael S. Tsirkin,
Greg Kroah-Hartman, Zhi Yong Wu, Anthony Liguori, linux-scsi,
Paolo Bonzini, lf-virt, Christoph Hellwig
In-Reply-To: <1342853738-27682-1-git-send-email-nab@linux-iscsi.org>
From: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
In order for other vhost devices to use the VHOST_FEATURES bits the
vhost-net specific bits need to be moved to their own VHOST_NET_FEATURES
constant.
(Asias: Update drivers/vhost/test.c to use VHOST_NET_FEATURES)
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Cc: Zhi Yong Wu <wuzhy@cn.ibm.com>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Asias He <asias@redhat.com>
Signed-off-by: Nicholas A. Bellinger <nab@risingtidesystems.com>
---
drivers/vhost/net.c | 4 ++--
drivers/vhost/test.c | 4 ++--
drivers/vhost/vhost.h | 3 ++-
3 files changed, 6 insertions(+), 5 deletions(-)
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index f82a739..072cbba 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -823,14 +823,14 @@ static long vhost_net_ioctl(struct file *f, unsigned int ioctl,
return -EFAULT;
return vhost_net_set_backend(n, backend.index, backend.fd);
case VHOST_GET_FEATURES:
- features = VHOST_FEATURES;
+ features = VHOST_NET_FEATURES;
if (copy_to_user(featurep, &features, sizeof features))
return -EFAULT;
return 0;
case VHOST_SET_FEATURES:
if (copy_from_user(&features, featurep, sizeof features))
return -EFAULT;
- if (features & ~VHOST_FEATURES)
+ if (features & ~VHOST_NET_FEATURES)
return -EOPNOTSUPP;
return vhost_net_set_features(n, features);
case VHOST_RESET_OWNER:
diff --git a/drivers/vhost/test.c b/drivers/vhost/test.c
index 3de00d9..91d6f06 100644
--- a/drivers/vhost/test.c
+++ b/drivers/vhost/test.c
@@ -261,14 +261,14 @@ static long vhost_test_ioctl(struct file *f, unsigned int ioctl,
return -EFAULT;
return vhost_test_run(n, test);
case VHOST_GET_FEATURES:
- features = VHOST_FEATURES;
+ features = VHOST_NET_FEATURES;
if (copy_to_user(featurep, &features, sizeof features))
return -EFAULT;
return 0;
case VHOST_SET_FEATURES:
if (copy_from_user(&features, featurep, sizeof features))
return -EFAULT;
- if (features & ~VHOST_FEATURES)
+ if (features & ~VHOST_NET_FEATURES)
return -EOPNOTSUPP;
return vhost_test_set_features(n, features);
case VHOST_RESET_OWNER:
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index 8de1fd5..07b9763 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -201,7 +201,8 @@ enum {
VHOST_FEATURES = (1ULL << VIRTIO_F_NOTIFY_ON_EMPTY) |
(1ULL << VIRTIO_RING_F_INDIRECT_DESC) |
(1ULL << VIRTIO_RING_F_EVENT_IDX) |
- (1ULL << VHOST_F_LOG_ALL) |
+ (1ULL << VHOST_F_LOG_ALL),
+ VHOST_NET_FEATURES = VHOST_FEATURES |
(1ULL << VHOST_NET_F_VIRTIO_NET_HDR) |
(1ULL << VIRTIO_NET_F_MRG_RXBUF),
};
--
1.7.2.5
^ permalink raw reply related
* [RFC-v4 2/3] vhost: make vhost work queue visible
From: Nicholas A. Bellinger @ 2012-07-21 6:55 UTC (permalink / raw)
To: target-devel
Cc: Jens Axboe, Stefan Hajnoczi, kvm-devel, Michael S. Tsirkin,
Greg Kroah-Hartman, Zhi Yong Wu, Anthony Liguori, linux-scsi,
Paolo Bonzini, lf-virt, Christoph Hellwig
In-Reply-To: <1342853738-27682-1-git-send-email-nab@linux-iscsi.org>
From: Stefan Hajnoczi <stefanha@gmail.com>
The vhost work queue allows processing to be done in vhost worker thread
context, which uses the owner process mm. Access to the vring and guest
memory is typically only possible from vhost worker context so it is
useful to allow work to be queued directly by users.
Currently vhost_net only uses the poll wrappers which do not expose the
work queue functions. However, for tcm_vhost (vhost_scsi) it will be
necessary to queue custom work.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Cc: Zhi Yong Wu <wuzhy@cn.ibm.com>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
drivers/vhost/vhost.c | 5 ++---
drivers/vhost/vhost.h | 3 +++
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index 94dbd25..1aab08b 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -64,7 +64,7 @@ static int vhost_poll_wakeup(wait_queue_t *wait, unsigned mode, int sync,
return 0;
}
-static void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn)
+void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn)
{
INIT_LIST_HEAD(&work->node);
work->fn = fn;
@@ -137,8 +137,7 @@ void vhost_poll_flush(struct vhost_poll *poll)
vhost_work_flush(poll->dev, &poll->work);
}
-static inline void vhost_work_queue(struct vhost_dev *dev,
- struct vhost_work *work)
+void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work)
{
unsigned long flags;
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index 07b9763..1125af3 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -43,6 +43,9 @@ struct vhost_poll {
struct vhost_dev *dev;
};
+void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn);
+void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work);
+
void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn,
unsigned long mask, struct vhost_dev *dev);
void vhost_poll_start(struct vhost_poll *poll, struct file *file);
--
1.7.2.5
^ permalink raw reply related
* [RFC-v4 3/3] tcm_vhost: Initial merge for vhost level target fabric driver
From: Nicholas A. Bellinger @ 2012-07-21 6:55 UTC (permalink / raw)
To: target-devel
Cc: Jens Axboe, Stefan Hajnoczi, kvm-devel, Michael S. Tsirkin,
Greg Kroah-Hartman, Zhi Yong Wu, Anthony Liguori, linux-scsi,
Paolo Bonzini, lf-virt, Christoph Hellwig
In-Reply-To: <1342853738-27682-1-git-send-email-nab@linux-iscsi.org>
From: Nicholas Bellinger <nab@linux-iscsi.org>
This patch adds the initial code for tcm_vhost, a Vhost level TCM
fabric driver for virtio SCSI initiators into KVM guest.
This code is currently up and running on v3.5-rc2 host+guest along
with the virtio-scsi vdev->scan() patch to allow a proper
scsi_scan_host() to occur once the tcm_vhost nexus has been established
by the paravirtualized virtio-scsi client here:
virtio-scsi: Add vdrv->scan for post VIRTIO_CONFIG_S_DRIVER_OK LUN scanning
http://marc.info/?l=linux-scsi&m=134160609212542&w=2
Using tcm_vhost requires Zhi's -> Stefan's qemu vhost-scsi tree here:
https://github.com/wuzhy/qemu/tree/vhost-scsi
along with the recent QEMU patch to hw/virtio-scsi.c to set max_target=0
during vhost-scsi operation.
Changelog v3 -> v4:
Rename vhost_vring_target -> vhost_scsi_target (mst + nab)
Use TRANSPORT_IQN_LEN in vhost_scsi_target->vhost_wwpn[] def (nab)
Move back to drivers/vhost/, and just use drivers/vhost/Kconfig.tcm (mst)
Move TCM_VHOST related ioctl defines from include/linux/vhost.h ->
drivers/vhost/tcm_vhost.h as requested by MST (nab)
Move Kbuild.tcm include from drivers/staging -> drivers/vhost/, and
just use 'if STAGING' around 'source drivers/vhost/Kbuild.tcm'
Changelog v2 -> v3:
Unlock on error in tcm_vhost_drop_nexus() (DanC)
Fix strlen() doesn't count the terminator (DanC)
Call kfree() on an error path (DanC)
Convert tcm_vhost_write_pending to use target_execute_cmd (hch + nab)
Fix another strlen() off by one in tcm_vhost_make_tport (DanC)
Add option under drivers/staging/Kconfig, and move to drivers/vhost/tcm/
as requested by MST (nab)
Changelog v1 -> v2:
Fix tv_cmd completion -> release SGL memory leak (nab)
Fix sparse warnings for static variable usage ((Fengguang Wu)
Fix sparse warnings for min() typing + printk format specs (Fengguang Wu)
Convert to cmwq submission for I/O dispatch (nab + hch)
Changelog v0 -> v1:
Merge into single source + header file, and move to drivers/vhost/
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Cc: Zhi Yong Wu <wuzhy@cn.ibm.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
drivers/vhost/Kconfig | 3 +
drivers/vhost/Kconfig.tcm | 6 +
drivers/vhost/Makefile | 2 +
drivers/vhost/tcm_vhost.c | 1611 +++++++++++++++++++++++++++++++++++++++++++++
drivers/vhost/tcm_vhost.h | 90 +++
5 files changed, 1712 insertions(+), 0 deletions(-)
create mode 100644 drivers/vhost/Kconfig.tcm
create mode 100644 drivers/vhost/tcm_vhost.c
create mode 100644 drivers/vhost/tcm_vhost.h
diff --git a/drivers/vhost/Kconfig b/drivers/vhost/Kconfig
index e4e2fd1..202bba6 100644
--- a/drivers/vhost/Kconfig
+++ b/drivers/vhost/Kconfig
@@ -9,3 +9,6 @@ config VHOST_NET
To compile this driver as a module, choose M here: the module will
be called vhost_net.
+if STAGING
+source "drivers/vhost/Kconfig.tcm"
+endif
diff --git a/drivers/vhost/Kconfig.tcm b/drivers/vhost/Kconfig.tcm
new file mode 100644
index 0000000..a9c6f76
--- /dev/null
+++ b/drivers/vhost/Kconfig.tcm
@@ -0,0 +1,6 @@
+config TCM_VHOST
+ tristate "TCM_VHOST fabric module (EXPERIMENTAL)"
+ depends on TARGET_CORE && EVENTFD && EXPERIMENTAL && m
+ default n
+ ---help---
+ Say M here to enable the TCM_VHOST fabric module for use with virtio-scsi guests
diff --git a/drivers/vhost/Makefile b/drivers/vhost/Makefile
index 72dd020..a27b053 100644
--- a/drivers/vhost/Makefile
+++ b/drivers/vhost/Makefile
@@ -1,2 +1,4 @@
obj-$(CONFIG_VHOST_NET) += vhost_net.o
vhost_net-y := vhost.o net.o
+
+obj-$(CONFIG_TCM_VHOST) += tcm_vhost.o
diff --git a/drivers/vhost/tcm_vhost.c b/drivers/vhost/tcm_vhost.c
new file mode 100644
index 0000000..dc7e024
--- /dev/null
+++ b/drivers/vhost/tcm_vhost.c
@@ -0,0 +1,1611 @@
+/*******************************************************************************
+ * Vhost kernel TCM fabric driver for virtio SCSI initiators
+ *
+ * (C) Copyright 2010-2012 RisingTide Systems LLC.
+ * (C) Copyright 2010-2012 IBM Corp.
+ *
+ * Licensed to the Linux Foundation under the General Public License (GPL) version 2.
+ *
+ * Authors: Nicholas A. Bellinger <nab@risingtidesystems.com>
+ * Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ ****************************************************************************/
+
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <generated/utsrelease.h>
+#include <linux/utsname.h>
+#include <linux/init.h>
+#include <linux/slab.h>
+#include <linux/kthread.h>
+#include <linux/types.h>
+#include <linux/string.h>
+#include <linux/configfs.h>
+#include <linux/ctype.h>
+#include <linux/compat.h>
+#include <linux/eventfd.h>
+#include <linux/vhost.h>
+#include <linux/fs.h>
+#include <linux/miscdevice.h>
+#include <asm/unaligned.h>
+#include <scsi/scsi.h>
+#include <scsi/scsi_tcq.h>
+#include <target/target_core_base.h>
+#include <target/target_core_fabric.h>
+#include <target/target_core_fabric_configfs.h>
+#include <target/target_core_configfs.h>
+#include <target/configfs_macros.h>
+#include <linux/vhost.h>
+#include <linux/virtio_net.h> /* TODO vhost.h currently depends on this */
+#include <linux/virtio_scsi.h>
+
+#include "vhost.c"
+#include "vhost.h"
+#include "tcm_vhost.h"
+
+struct vhost_scsi {
+ atomic_t vhost_ref_cnt;
+ struct tcm_vhost_tpg *vs_tpg;
+ struct vhost_dev dev;
+ struct vhost_virtqueue vqs[3];
+
+ struct vhost_work vs_completion_work; /* cmd completion work item */
+ struct list_head vs_completion_list; /* cmd completion queue */
+ spinlock_t vs_completion_lock; /* protects s_completion_list */
+};
+
+/* Local pointer to allocated TCM configfs fabric module */
+static struct target_fabric_configfs *tcm_vhost_fabric_configfs;
+
+static struct workqueue_struct *tcm_vhost_workqueue;
+
+/* Global spinlock to protect tcm_vhost TPG list for vhost IOCTL access */
+static DEFINE_MUTEX(tcm_vhost_mutex);
+static LIST_HEAD(tcm_vhost_list);
+
+static int tcm_vhost_check_true(struct se_portal_group *se_tpg)
+{
+ return 1;
+}
+
+static int tcm_vhost_check_false(struct se_portal_group *se_tpg)
+{
+ return 0;
+}
+
+static char *tcm_vhost_get_fabric_name(void)
+{
+ return "vhost";
+}
+
+static u8 tcm_vhost_get_fabric_proto_ident(struct se_portal_group *se_tpg)
+{
+ struct tcm_vhost_tpg *tpg = container_of(se_tpg,
+ struct tcm_vhost_tpg, se_tpg);
+ struct tcm_vhost_tport *tport = tpg->tport;
+
+ switch (tport->tport_proto_id) {
+ case SCSI_PROTOCOL_SAS:
+ return sas_get_fabric_proto_ident(se_tpg);
+ case SCSI_PROTOCOL_FCP:
+ return fc_get_fabric_proto_ident(se_tpg);
+ case SCSI_PROTOCOL_ISCSI:
+ return iscsi_get_fabric_proto_ident(se_tpg);
+ default:
+ pr_err("Unknown tport_proto_id: 0x%02x, using"
+ " SAS emulation\n", tport->tport_proto_id);
+ break;
+ }
+
+ return sas_get_fabric_proto_ident(se_tpg);
+}
+
+static char *tcm_vhost_get_fabric_wwn(struct se_portal_group *se_tpg)
+{
+ struct tcm_vhost_tpg *tpg = container_of(se_tpg,
+ struct tcm_vhost_tpg, se_tpg);
+ struct tcm_vhost_tport *tport = tpg->tport;
+
+ return &tport->tport_name[0];
+}
+
+static u16 tcm_vhost_get_tag(struct se_portal_group *se_tpg)
+{
+ struct tcm_vhost_tpg *tpg = container_of(se_tpg,
+ struct tcm_vhost_tpg, se_tpg);
+ return tpg->tport_tpgt;
+}
+
+static u32 tcm_vhost_get_default_depth(struct se_portal_group *se_tpg)
+{
+ return 1;
+}
+
+static u32 tcm_vhost_get_pr_transport_id(
+ struct se_portal_group *se_tpg,
+ struct se_node_acl *se_nacl,
+ struct t10_pr_registration *pr_reg,
+ int *format_code,
+ unsigned char *buf)
+{
+ struct tcm_vhost_tpg *tpg = container_of(se_tpg,
+ struct tcm_vhost_tpg, se_tpg);
+ struct tcm_vhost_tport *tport = tpg->tport;
+
+ switch (tport->tport_proto_id) {
+ case SCSI_PROTOCOL_SAS:
+ return sas_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
+ format_code, buf);
+ case SCSI_PROTOCOL_FCP:
+ return fc_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
+ format_code, buf);
+ case SCSI_PROTOCOL_ISCSI:
+ return iscsi_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
+ format_code, buf);
+ default:
+ pr_err("Unknown tport_proto_id: 0x%02x, using"
+ " SAS emulation\n", tport->tport_proto_id);
+ break;
+ }
+
+ return sas_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
+ format_code, buf);
+}
+
+static u32 tcm_vhost_get_pr_transport_id_len(
+ struct se_portal_group *se_tpg,
+ struct se_node_acl *se_nacl,
+ struct t10_pr_registration *pr_reg,
+ int *format_code)
+{
+ struct tcm_vhost_tpg *tpg = container_of(se_tpg,
+ struct tcm_vhost_tpg, se_tpg);
+ struct tcm_vhost_tport *tport = tpg->tport;
+
+ switch (tport->tport_proto_id) {
+ case SCSI_PROTOCOL_SAS:
+ return sas_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
+ format_code);
+ case SCSI_PROTOCOL_FCP:
+ return fc_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
+ format_code);
+ case SCSI_PROTOCOL_ISCSI:
+ return iscsi_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
+ format_code);
+ default:
+ pr_err("Unknown tport_proto_id: 0x%02x, using"
+ " SAS emulation\n", tport->tport_proto_id);
+ break;
+ }
+
+ return sas_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
+ format_code);
+}
+
+static char *tcm_vhost_parse_pr_out_transport_id(
+ struct se_portal_group *se_tpg,
+ const char *buf,
+ u32 *out_tid_len,
+ char **port_nexus_ptr)
+{
+ struct tcm_vhost_tpg *tpg = container_of(se_tpg,
+ struct tcm_vhost_tpg, se_tpg);
+ struct tcm_vhost_tport *tport = tpg->tport;
+
+ switch (tport->tport_proto_id) {
+ case SCSI_PROTOCOL_SAS:
+ return sas_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
+ port_nexus_ptr);
+ case SCSI_PROTOCOL_FCP:
+ return fc_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
+ port_nexus_ptr);
+ case SCSI_PROTOCOL_ISCSI:
+ return iscsi_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
+ port_nexus_ptr);
+ default:
+ pr_err("Unknown tport_proto_id: 0x%02x, using"
+ " SAS emulation\n", tport->tport_proto_id);
+ break;
+ }
+
+ return sas_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
+ port_nexus_ptr);
+}
+
+static struct se_node_acl *tcm_vhost_alloc_fabric_acl(struct se_portal_group *se_tpg)
+{
+ struct tcm_vhost_nacl *nacl;
+
+ nacl = kzalloc(sizeof(struct tcm_vhost_nacl), GFP_KERNEL);
+ if (!nacl) {
+ pr_err("Unable to alocate struct tcm_vhost_nacl\n");
+ return NULL;
+ }
+
+ return &nacl->se_node_acl;
+}
+
+static void tcm_vhost_release_fabric_acl(
+ struct se_portal_group *se_tpg,
+ struct se_node_acl *se_nacl)
+{
+ struct tcm_vhost_nacl *nacl = container_of(se_nacl,
+ struct tcm_vhost_nacl, se_node_acl);
+ kfree(nacl);
+}
+
+static u32 tcm_vhost_tpg_get_inst_index(struct se_portal_group *se_tpg)
+{
+ return 1;
+}
+
+static void tcm_vhost_release_cmd(struct se_cmd *se_cmd)
+{
+ return;
+}
+
+static int tcm_vhost_shutdown_session(struct se_session *se_sess)
+{
+ return 0;
+}
+
+static void tcm_vhost_close_session(struct se_session *se_sess)
+{
+ return;
+}
+
+static u32 tcm_vhost_sess_get_index(struct se_session *se_sess)
+{
+ return 0;
+}
+
+static int tcm_vhost_write_pending(struct se_cmd *se_cmd)
+{
+ /* Go ahead and process the write immediately */
+ target_execute_cmd(se_cmd);
+ return 0;
+}
+
+static int tcm_vhost_write_pending_status(struct se_cmd *se_cmd)
+{
+ return 0;
+}
+
+static void tcm_vhost_set_default_node_attrs(struct se_node_acl *nacl)
+{
+ return;
+}
+
+static u32 tcm_vhost_get_task_tag(struct se_cmd *se_cmd)
+{
+ return 0;
+}
+
+static int tcm_vhost_get_cmd_state(struct se_cmd *se_cmd)
+{
+ return 0;
+}
+
+static void vhost_scsi_complete_cmd(struct tcm_vhost_cmd *);
+
+static int tcm_vhost_queue_data_in(struct se_cmd *se_cmd)
+{
+ struct tcm_vhost_cmd *tv_cmd = container_of(se_cmd,
+ struct tcm_vhost_cmd, tvc_se_cmd);
+ vhost_scsi_complete_cmd(tv_cmd);
+ return 0;
+}
+
+static int tcm_vhost_queue_status(struct se_cmd *se_cmd)
+{
+ struct tcm_vhost_cmd *tv_cmd = container_of(se_cmd,
+ struct tcm_vhost_cmd, tvc_se_cmd);
+ vhost_scsi_complete_cmd(tv_cmd);
+ return 0;
+}
+
+static int tcm_vhost_queue_tm_rsp(struct se_cmd *se_cmd)
+{
+ return 0;
+}
+
+static u16 tcm_vhost_set_fabric_sense_len(struct se_cmd *se_cmd, u32 sense_length)
+{
+ return 0;
+}
+
+static u16 tcm_vhost_get_fabric_sense_len(void)
+{
+ return 0;
+}
+
+static void vhost_scsi_free_cmd(struct tcm_vhost_cmd *tv_cmd)
+{
+ struct se_cmd *se_cmd = &tv_cmd->tvc_se_cmd;
+
+ /* TODO locking against target/backend threads? */
+ transport_generic_free_cmd(se_cmd, 1);
+
+ if (tv_cmd->tvc_sgl_count) {
+ u32 i;
+ for (i = 0; i < tv_cmd->tvc_sgl_count; i++)
+ put_page(sg_page(&tv_cmd->tvc_sgl[i]));
+
+ kfree(tv_cmd->tvc_sgl);
+ }
+
+ kfree(tv_cmd);
+}
+
+/* Dequeue a command from the completion list */
+static struct tcm_vhost_cmd *vhost_scsi_get_cmd_from_completion(struct vhost_scsi *vs)
+{
+ struct tcm_vhost_cmd *tv_cmd = NULL;
+
+ spin_lock_bh(&vs->vs_completion_lock);
+ if (list_empty(&vs->vs_completion_list)) {
+ spin_unlock_bh(&vs->vs_completion_lock);
+ return NULL;
+ }
+
+ list_for_each_entry(tv_cmd, &vs->vs_completion_list,
+ tvc_completion_list) {
+ list_del(&tv_cmd->tvc_completion_list);
+ break;
+ }
+ spin_unlock_bh(&vs->vs_completion_lock);
+ return tv_cmd;
+}
+
+/* Fill in status and signal that we are done processing this command
+ *
+ * This is scheduled in the vhost work queue so we are called with the owner
+ * process mm and can access the vring.
+ */
+static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
+{
+ struct vhost_scsi *vs = container_of(work, struct vhost_scsi,
+ vs_completion_work);
+ struct tcm_vhost_cmd *tv_cmd;
+
+ while ((tv_cmd = vhost_scsi_get_cmd_from_completion(vs)) != NULL) {
+ struct virtio_scsi_cmd_resp v_rsp;
+ struct se_cmd *se_cmd = &tv_cmd->tvc_se_cmd;
+ int ret;
+
+ pr_debug("%s tv_cmd %p resid %u status %#02x\n", __func__,
+ tv_cmd, se_cmd->residual_count, se_cmd->scsi_status);
+
+ memset(&v_rsp, 0, sizeof(v_rsp));
+ v_rsp.resid = se_cmd->residual_count;
+ /* TODO is status_qualifier field needed? */
+ v_rsp.status = se_cmd->scsi_status;
+ v_rsp.sense_len = se_cmd->scsi_sense_length;
+ memcpy(v_rsp.sense, tv_cmd->tvc_sense_buf,
+ v_rsp.sense_len);
+ ret = copy_to_user(tv_cmd->tvc_resp, &v_rsp, sizeof(v_rsp));
+ if (likely(ret == 0))
+ vhost_add_used(&vs->vqs[2], tv_cmd->tvc_vq_desc, 0);
+ else
+ pr_err("Faulted on virtio_scsi_cmd_resp\n");
+
+ vhost_scsi_free_cmd(tv_cmd);
+ }
+
+ vhost_signal(&vs->dev, &vs->vqs[2]);
+}
+
+static void vhost_scsi_complete_cmd(struct tcm_vhost_cmd *tv_cmd)
+{
+ struct vhost_scsi *vs = tv_cmd->tvc_vhost;
+
+ pr_debug("%s tv_cmd %p\n", __func__, tv_cmd);
+
+ spin_lock_bh(&vs->vs_completion_lock);
+ list_add_tail(&tv_cmd->tvc_completion_list, &vs->vs_completion_list);
+ spin_unlock_bh(&vs->vs_completion_lock);
+
+ vhost_work_queue(&vs->dev, &vs->vs_completion_work);
+}
+
+static struct tcm_vhost_cmd *vhost_scsi_allocate_cmd(
+ struct tcm_vhost_tpg *tv_tpg,
+ struct virtio_scsi_cmd_req *v_req,
+ u32 exp_data_len,
+ int data_direction)
+{
+ struct tcm_vhost_cmd *tv_cmd;
+ struct tcm_vhost_nexus *tv_nexus;
+ struct se_portal_group *se_tpg = &tv_tpg->se_tpg;
+ struct se_session *se_sess;
+ struct se_cmd *se_cmd;
+ int sam_task_attr;
+
+ tv_nexus = tv_tpg->tpg_nexus;
+ if (!tv_nexus) {
+ pr_err("Unable to locate active struct tcm_vhost_nexus\n");
+ return ERR_PTR(-EIO);
+ }
+ se_sess = tv_nexus->tvn_se_sess;
+
+ tv_cmd = kzalloc(sizeof(struct tcm_vhost_cmd), GFP_ATOMIC);
+ if (!tv_cmd) {
+ pr_err("Unable to allocate struct tcm_vhost_cmd\n");
+ return ERR_PTR(-ENOMEM);
+ }
+ INIT_LIST_HEAD(&tv_cmd->tvc_completion_list);
+ tv_cmd->tvc_tag = v_req->tag;
+
+ se_cmd = &tv_cmd->tvc_se_cmd;
+ /*
+ * Locate the SAM Task Attr from virtio_scsi_cmd_req
+ */
+ sam_task_attr = v_req->task_attr;
+ /*
+ * Initialize struct se_cmd descriptor from target_core_mod infrastructure
+ */
+ transport_init_se_cmd(se_cmd, se_tpg->se_tpg_tfo, se_sess, exp_data_len,
+ data_direction, sam_task_attr,
+ &tv_cmd->tvc_sense_buf[0]);
+
+#if 0 /* FIXME: vhost_scsi_allocate_cmd() BIDI operation */
+ if (bidi)
+ se_cmd->se_cmd_flags |= SCF_BIDI;
+#endif
+ return tv_cmd;
+}
+
+/*
+ * Map a user memory range into a scatterlist
+ *
+ * Returns the number of scatterlist entries used or -errno on error.
+ */
+static int vhost_scsi_map_to_sgl(struct scatterlist *sgl,
+ unsigned int sgl_count,
+ void __user *ptr, size_t len, int write)
+{
+ struct scatterlist *sg = sgl;
+ unsigned int npages = 0;
+ int ret;
+
+ while (len > 0) {
+ struct page *page;
+ unsigned int offset = (uintptr_t)ptr & ~PAGE_MASK;
+ unsigned int nbytes = min_t(unsigned int,
+ PAGE_SIZE - offset, len);
+
+ if (npages == sgl_count) {
+ ret = -ENOBUFS;
+ goto err;
+ }
+
+ ret = get_user_pages_fast((unsigned long)ptr, 1, write, &page);
+ BUG_ON(ret == 0); /* we should either get our page or fail */
+ if (ret < 0)
+ goto err;
+
+ sg_set_page(sg, page, nbytes, offset);
+ ptr += nbytes;
+ len -= nbytes;
+ sg++;
+ npages++;
+ }
+ return npages;
+
+err:
+ /* Put pages that we hold */
+ for (sg = sgl; sg != &sgl[npages]; sg++)
+ put_page(sg_page(sg));
+ return ret;
+}
+
+static int vhost_scsi_map_iov_to_sgl(struct tcm_vhost_cmd *tv_cmd,
+ struct iovec *iov, unsigned int niov,
+ int write)
+{
+ int ret;
+ unsigned int i;
+ u32 sgl_count;
+ struct scatterlist *sg;
+
+ /*
+ * Find out how long sglist needs to be
+ */
+ sgl_count = 0;
+ for (i = 0; i < niov; i++) {
+ sgl_count += (((uintptr_t)iov[i].iov_base + iov[i].iov_len +
+ PAGE_SIZE - 1) >> PAGE_SHIFT) -
+ ((uintptr_t)iov[i].iov_base >> PAGE_SHIFT);
+ }
+ /* TODO overflow checking */
+
+ sg = kmalloc(sizeof(tv_cmd->tvc_sgl[0]) * sgl_count, GFP_ATOMIC);
+ if (!sg)
+ return -ENOMEM;
+ pr_debug("%s sg %p sgl_count %u is_err %ld\n", __func__,
+ sg, sgl_count, IS_ERR(sg));
+ sg_init_table(sg, sgl_count);
+
+ tv_cmd->tvc_sgl = sg;
+ tv_cmd->tvc_sgl_count = sgl_count;
+
+ pr_debug("Mapping %u iovecs for %u pages\n", niov, sgl_count);
+ for (i = 0; i < niov; i++) {
+ ret = vhost_scsi_map_to_sgl(sg, sgl_count, iov[i].iov_base,
+ iov[i].iov_len, write);
+ if (ret < 0) {
+ for (i = 0; i < tv_cmd->tvc_sgl_count; i++)
+ put_page(sg_page(&tv_cmd->tvc_sgl[i]));
+ kfree(tv_cmd->tvc_sgl);
+ tv_cmd->tvc_sgl = NULL;
+ tv_cmd->tvc_sgl_count = 0;
+ return ret;
+ }
+
+ sg += ret;
+ sgl_count -= ret;
+ }
+ return 0;
+}
+
+static void tcm_vhost_submission_work(struct work_struct *work)
+{
+ struct tcm_vhost_cmd *tv_cmd =
+ container_of(work, struct tcm_vhost_cmd, work);
+ struct se_cmd *se_cmd = &tv_cmd->tvc_se_cmd;
+ struct scatterlist *sg_ptr, *sg_bidi_ptr = NULL;
+ int rc, sg_no_bidi = 0;
+ /*
+ * Locate the struct se_lun pointer based on v_req->lun, and
+ * attach it to struct se_cmd
+ */
+ rc = transport_lookup_cmd_lun(&tv_cmd->tvc_se_cmd, tv_cmd->tvc_lun);
+ if (rc < 0) {
+ pr_err("Failed to look up lun: %d\n", tv_cmd->tvc_lun);
+ transport_send_check_condition_and_sense(&tv_cmd->tvc_se_cmd,
+ tv_cmd->tvc_se_cmd.scsi_sense_reason, 0);
+ transport_generic_free_cmd(se_cmd, 0);
+ return;
+ }
+
+ rc = target_setup_cmd_from_cdb(se_cmd, tv_cmd->tvc_cdb);
+ if (rc == -ENOMEM) {
+ transport_send_check_condition_and_sense(se_cmd,
+ TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE, 0);
+ transport_generic_free_cmd(se_cmd, 0);
+ return;
+ } else if (rc < 0) {
+ if (se_cmd->se_cmd_flags & SCF_SCSI_RESERVATION_CONFLICT)
+ tcm_vhost_queue_status(se_cmd);
+ else
+ transport_send_check_condition_and_sense(se_cmd,
+ se_cmd->scsi_sense_reason, 0);
+ transport_generic_free_cmd(se_cmd, 0);
+ return;
+ }
+
+ if (tv_cmd->tvc_sgl_count) {
+ sg_ptr = tv_cmd->tvc_sgl;
+ /*
+ * For BIDI commands, pass in the extra READ buffer
+ * to transport_generic_map_mem_to_cmd() below..
+ */
+/* FIXME: Fix BIDI operation in tcm_vhost_submission_work() */
+#if 0
+ if (se_cmd->se_cmd_flags & SCF_BIDI) {
+ sg_bidi_ptr = NULL;
+ sg_no_bidi = 0;
+ }
+#endif
+ } else {
+ sg_ptr = NULL;
+ }
+
+ rc = transport_generic_map_mem_to_cmd(se_cmd, sg_ptr,
+ tv_cmd->tvc_sgl_count, sg_bidi_ptr,
+ sg_no_bidi);
+ if (rc < 0) {
+ transport_send_check_condition_and_sense(se_cmd,
+ se_cmd->scsi_sense_reason, 0);
+ transport_generic_free_cmd(se_cmd, 0);
+ return;
+ }
+ transport_handle_cdb_direct(se_cmd);
+}
+
+static void vhost_scsi_handle_vq(struct vhost_scsi *vs)
+{
+ struct vhost_virtqueue *vq = &vs->vqs[2];
+ struct virtio_scsi_cmd_req v_req;
+ struct tcm_vhost_tpg *tv_tpg;
+ struct tcm_vhost_cmd *tv_cmd;
+ u32 exp_data_len, data_first, data_num, data_direction;
+ unsigned out, in, i;
+ int head, ret;
+
+ /* Must use ioctl VHOST_SCSI_SET_ENDPOINT */
+ tv_tpg = vs->vs_tpg;
+ if (unlikely(!tv_tpg)) {
+ pr_err("%s endpoint not set\n", __func__);
+ return;
+ }
+
+ mutex_lock(&vq->mutex);
+ vhost_disable_notify(&vs->dev, vq);
+
+ for (;;) {
+ head = vhost_get_vq_desc(&vs->dev, vq, vq->iov,
+ ARRAY_SIZE(vq->iov), &out, &in,
+ NULL, NULL);
+ pr_debug("vhost_get_vq_desc: head: %d, out: %u in: %u\n", head, out, in);
+ /* On error, stop handling until the next kick. */
+ if (unlikely(head < 0))
+ break;
+ /* Nothing new? Wait for eventfd to tell us they refilled. */
+ if (head == vq->num) {
+ if (unlikely(vhost_enable_notify(&vs->dev, vq))) {
+ vhost_disable_notify(&vs->dev, vq);
+ continue;
+ }
+ break;
+ }
+
+/* FIXME: BIDI operation */
+ if (out == 1 && in == 1) {
+ data_direction = DMA_NONE;
+ data_first = 0;
+ data_num = 0;
+ } else if (out == 1 && in > 1) {
+ data_direction = DMA_FROM_DEVICE;
+ data_first = out + 1;
+ data_num = in - 1;
+ } else if (out > 1 && in == 1) {
+ data_direction = DMA_TO_DEVICE;
+ data_first = 1;
+ data_num = out - 1;
+ } else {
+ pr_err("Invalid buffer layout out: %u in: %u\n", out, in);
+ break;
+ }
+
+ /*
+ * Check for a sane resp buffer so we can report errors to
+ * the guest.
+ */
+ if (unlikely(vq->iov[out].iov_len !=
+ sizeof(struct virtio_scsi_cmd_resp))) {
+ pr_err("Expecting virtio_scsi_cmd_resp, got %zu bytes\n",
+ vq->iov[out].iov_len);
+ break;
+ }
+
+ if (unlikely(vq->iov[0].iov_len != sizeof(v_req))) {
+ pr_err("Expecting virtio_scsi_cmd_req, got %zu bytes\n",
+ vq->iov[0].iov_len);
+ break;
+ }
+ pr_debug("Calling __copy_from_user: vq->iov[0].iov_base: %p, len: %zu\n",
+ vq->iov[0].iov_base, sizeof(v_req));
+ ret = __copy_from_user(&v_req, vq->iov[0].iov_base, sizeof(v_req));
+ if (unlikely(ret)) {
+ pr_err("Faulted on virtio_scsi_cmd_req\n");
+ break;
+ }
+
+ exp_data_len = 0;
+ for (i = 0; i < data_num; i++) {
+ exp_data_len += vq->iov[data_first + i].iov_len;
+ }
+
+ tv_cmd = vhost_scsi_allocate_cmd(tv_tpg, &v_req,
+ exp_data_len, data_direction);
+ if (IS_ERR(tv_cmd)) {
+ pr_err("vhost_scsi_allocate_cmd failed %ld\n", PTR_ERR(tv_cmd));
+ break;
+ }
+ pr_debug("Allocated tv_cmd: %p exp_data_len: %d, data_direction: %d\n",
+ tv_cmd, exp_data_len, data_direction);
+
+ tv_cmd->tvc_vhost = vs;
+
+ if (unlikely(vq->iov[out].iov_len !=
+ sizeof(struct virtio_scsi_cmd_resp))) {
+ pr_err("Expecting virtio_scsi_cmd_resp, "
+ " got %zu bytes, out: %d, in: %d\n", vq->iov[out].iov_len, out, in);
+ break;
+ }
+
+ tv_cmd->tvc_resp = vq->iov[out].iov_base;
+
+ /*
+ * Copy in the recieved CDB descriptor into tv_cmd->tvc_cdb
+ * that will be used by tcm_vhost_new_cmd_map() and down into
+ * target_setup_cmd_from_cdb()
+ */
+ memcpy(tv_cmd->tvc_cdb, v_req.cdb, TCM_VHOST_MAX_CDB_SIZE);
+ /*
+ * Check that the recieved CDB size does not exceeded our
+ * hardcoded max for tcm_vhost
+ */
+ /* TODO what if cdb was too small for varlen cdb header? */
+ if (unlikely(scsi_command_size(tv_cmd->tvc_cdb) > TCM_VHOST_MAX_CDB_SIZE)) {
+ pr_err("Received SCSI CDB with command_size: %d that exceeds"
+ " SCSI_MAX_VARLEN_CDB_SIZE: %d\n",
+ scsi_command_size(tv_cmd->tvc_cdb), TCM_VHOST_MAX_CDB_SIZE);
+ break; /* TODO */
+ }
+ tv_cmd->tvc_lun = ((v_req.lun[2] << 8) | v_req.lun[3]) & 0x3FFF;
+
+ pr_debug("vhost_scsi got command opcode: %#02x, lun: %d\n",
+ tv_cmd->tvc_cdb[0], tv_cmd->tvc_lun);
+
+ if (data_direction != DMA_NONE) {
+ ret = vhost_scsi_map_iov_to_sgl(tv_cmd, &vq->iov[data_first],
+ data_num, data_direction == DMA_TO_DEVICE);
+ if (unlikely(ret)) {
+ pr_err("Failed to map iov to sgl\n");
+ break; /* TODO */
+ }
+ }
+
+ /*
+ * Save the descriptor from vhost_get_vq_desc() to be used to
+ * complete the virtio-scsi request in TCM callback context via
+ * tcm_vhost_queue_data_in() and tcm_vhost_queue_status()
+ */
+ tv_cmd->tvc_vq_desc = head;
+ /*
+ * Dispatch tv_cmd descriptor for cmwq execution in process
+ * context provided by tcm_vhost_workqueue. This also ensures
+ * tv_cmd is executed on the same kworker CPU as this vhost
+ * thread to gain positive L2 cache locality effects..
+ */
+ INIT_WORK(&tv_cmd->work, tcm_vhost_submission_work);
+ queue_work(tcm_vhost_workqueue, &tv_cmd->work);
+ }
+
+ mutex_unlock(&vq->mutex);
+}
+
+static void vhost_scsi_ctl_handle_kick(struct vhost_work *work)
+{
+ pr_err("%s: The handling func for control queue.\n", __func__);
+}
+
+static void vhost_scsi_evt_handle_kick(struct vhost_work *work)
+{
+ pr_err("%s: The handling func for event queue.\n", __func__);
+}
+
+static void vhost_scsi_handle_kick(struct vhost_work *work)
+{
+ struct vhost_virtqueue *vq = container_of(work, struct vhost_virtqueue,
+ poll.work);
+ struct vhost_scsi *vs = container_of(vq->dev, struct vhost_scsi, dev);
+
+ vhost_scsi_handle_vq(vs);
+}
+
+/*
+ * Called from vhost_scsi_ioctl() context to walk the list of available tcm_vhost_tpg
+ * with an active struct tcm_vhost_nexus
+ */
+static int vhost_scsi_set_endpoint(
+ struct vhost_scsi *vs,
+ struct vhost_scsi_target *t)
+{
+ struct tcm_vhost_tport *tv_tport;
+ struct tcm_vhost_tpg *tv_tpg;
+ int index;
+
+ mutex_lock(&vs->dev.mutex);
+ /* Verify that ring has been setup correctly. */
+ for (index = 0; index < vs->dev.nvqs; ++index) {
+ /* Verify that ring has been setup correctly. */
+ if (!vhost_vq_access_ok(&vs->vqs[index])) {
+ mutex_unlock(&vs->dev.mutex);
+ return -EFAULT;
+ }
+ }
+
+ if (vs->vs_tpg) {
+ mutex_unlock(&vs->dev.mutex);
+ return -EEXIST;
+ }
+ mutex_unlock(&vs->dev.mutex);
+
+ mutex_lock(&tcm_vhost_mutex);
+ list_for_each_entry(tv_tpg, &tcm_vhost_list, tv_tpg_list) {
+ mutex_lock(&tv_tpg->tv_tpg_mutex);
+ if (!tv_tpg->tpg_nexus) {
+ mutex_unlock(&tv_tpg->tv_tpg_mutex);
+ continue;
+ }
+ if (atomic_read(&tv_tpg->tv_tpg_vhost_count)) {
+ mutex_unlock(&tv_tpg->tv_tpg_mutex);
+ continue;
+ }
+ tv_tport = tv_tpg->tport;
+
+ if (!strcmp(tv_tport->tport_name, t->vhost_wwpn) &&
+ (tv_tpg->tport_tpgt == t->vhost_tpgt)) {
+ atomic_inc(&tv_tpg->tv_tpg_vhost_count);
+ smp_mb__after_atomic_inc();
+ mutex_unlock(&tv_tpg->tv_tpg_mutex);
+ mutex_unlock(&tcm_vhost_mutex);
+
+ mutex_lock(&vs->dev.mutex);
+ vs->vs_tpg = tv_tpg;
+ atomic_inc(&vs->vhost_ref_cnt);
+ smp_mb__after_atomic_inc();
+ mutex_unlock(&vs->dev.mutex);
+ return 0;
+ }
+ mutex_unlock(&tv_tpg->tv_tpg_mutex);
+ }
+ mutex_unlock(&tcm_vhost_mutex);
+ return -EINVAL;
+}
+
+static int vhost_scsi_clear_endpoint(
+ struct vhost_scsi *vs,
+ struct vhost_scsi_target *t)
+{
+ struct tcm_vhost_tport *tv_tport;
+ struct tcm_vhost_tpg *tv_tpg;
+ int index;
+
+ mutex_lock(&vs->dev.mutex);
+ /* Verify that ring has been setup correctly. */
+ for (index = 0; index < vs->dev.nvqs; ++index) {
+ if (!vhost_vq_access_ok(&vs->vqs[index])) {
+ mutex_unlock(&vs->dev.mutex);
+ return -EFAULT;
+ }
+ }
+
+ if (!vs->vs_tpg) {
+ mutex_unlock(&vs->dev.mutex);
+ return -ENODEV;
+ }
+ tv_tpg = vs->vs_tpg;
+ tv_tport = tv_tpg->tport;
+
+ if (strcmp(tv_tport->tport_name, t->vhost_wwpn) ||
+ (tv_tpg->tport_tpgt != t->vhost_tpgt)) {
+ mutex_unlock(&vs->dev.mutex);
+ pr_warn("tv_tport->tport_name: %s, tv_tpg->tport_tpgt: %hu"
+ " does not match t->vhost_wwpn: %s, t->vhost_tpgt: %hu\n",
+ tv_tport->tport_name, tv_tpg->tport_tpgt,
+ t->vhost_wwpn, t->vhost_tpgt);
+ return -EINVAL;
+ }
+ atomic_dec(&tv_tpg->tv_tpg_vhost_count);
+ vs->vs_tpg = NULL;
+ mutex_unlock(&vs->dev.mutex);
+
+ return 0;
+}
+
+static int vhost_scsi_open(struct inode *inode, struct file *f)
+{
+ struct vhost_scsi *s;
+ int r;
+
+ s = kzalloc(sizeof(*s), GFP_KERNEL);
+ if (!s)
+ return -ENOMEM;
+
+ vhost_work_init(&s->vs_completion_work, vhost_scsi_complete_cmd_work);
+ INIT_LIST_HEAD(&s->vs_completion_list);
+ spin_lock_init(&s->vs_completion_lock);
+
+ s->vqs[0].handle_kick = vhost_scsi_ctl_handle_kick;
+ s->vqs[1].handle_kick = vhost_scsi_evt_handle_kick;
+ s->vqs[2].handle_kick = vhost_scsi_handle_kick;
+ r = vhost_dev_init(&s->dev, s->vqs, 3);
+ if (r < 0) {
+ kfree(s);
+ return r;
+ }
+
+ f->private_data = s;
+ return 0;
+}
+
+static int vhost_scsi_release(struct inode *inode, struct file *f)
+{
+ struct vhost_scsi *s = f->private_data;
+
+ if (s->vs_tpg && s->vs_tpg->tport) {
+ struct vhost_scsi_target backend;
+ memcpy(backend.vhost_wwpn, s->vs_tpg->tport->tport_name, sizeof(backend.vhost_wwpn));
+ backend.vhost_tpgt = s->vs_tpg->tport_tpgt;
+ vhost_scsi_clear_endpoint(s, &backend);
+ }
+
+ vhost_dev_cleanup(&s->dev, false);
+ kfree(s);
+ return 0;
+}
+
+static int vhost_scsi_set_features(struct vhost_scsi *vs, u64 features)
+{
+ if (features & ~VHOST_FEATURES)
+ return -EOPNOTSUPP;
+
+ mutex_lock(&vs->dev.mutex);
+ if ((features & (1 << VHOST_F_LOG_ALL)) &&
+ !vhost_log_access_ok(&vs->dev)) {
+ mutex_unlock(&vs->dev.mutex);
+ return -EFAULT;
+ }
+ vs->dev.acked_features = features;
+ /* TODO possibly smp_wmb() and flush vqs */
+ mutex_unlock(&vs->dev.mutex);
+ return 0;
+}
+
+static long vhost_scsi_ioctl(struct file *f, unsigned int ioctl,
+ unsigned long arg)
+{
+ struct vhost_scsi *vs = f->private_data;
+ struct vhost_scsi_target backend;
+ void __user *argp = (void __user *)arg;
+ u64 __user *featurep = argp;
+ u64 features;
+ int r;
+
+ switch (ioctl) {
+ case VHOST_SCSI_SET_ENDPOINT:
+ if (copy_from_user(&backend, argp, sizeof backend))
+ return -EFAULT;
+
+ return vhost_scsi_set_endpoint(vs, &backend);
+ case VHOST_SCSI_CLEAR_ENDPOINT:
+ if (copy_from_user(&backend, argp, sizeof backend))
+ return -EFAULT;
+
+ return vhost_scsi_clear_endpoint(vs, &backend);
+ case VHOST_GET_FEATURES:
+ features = VHOST_FEATURES;
+ if (copy_to_user(featurep, &features, sizeof features))
+ return -EFAULT;
+ return 0;
+ case VHOST_SET_FEATURES:
+ if (copy_from_user(&features, featurep, sizeof features))
+ return -EFAULT;
+ return vhost_scsi_set_features(vs, features);
+ default:
+ mutex_lock(&vs->dev.mutex);
+ r = vhost_dev_ioctl(&vs->dev, ioctl, arg);
+ mutex_unlock(&vs->dev.mutex);
+ return r;
+ }
+}
+
+static const struct file_operations vhost_scsi_fops = {
+ .owner = THIS_MODULE,
+ .release = vhost_scsi_release,
+ .unlocked_ioctl = vhost_scsi_ioctl,
+ /* TODO compat ioctl? */
+ .open = vhost_scsi_open,
+ .llseek = noop_llseek,
+};
+
+static struct miscdevice vhost_scsi_misc = {
+ MISC_DYNAMIC_MINOR,
+ "vhost-scsi",
+ &vhost_scsi_fops,
+};
+
+static int __init vhost_scsi_register(void)
+{
+ return misc_register(&vhost_scsi_misc);
+}
+
+static int vhost_scsi_deregister(void)
+{
+ return misc_deregister(&vhost_scsi_misc);
+}
+
+static char *tcm_vhost_dump_proto_id(struct tcm_vhost_tport *tport)
+{
+ switch (tport->tport_proto_id) {
+ case SCSI_PROTOCOL_SAS:
+ return "SAS";
+ case SCSI_PROTOCOL_FCP:
+ return "FCP";
+ case SCSI_PROTOCOL_ISCSI:
+ return "iSCSI";
+ default:
+ break;
+ }
+
+ return "Unknown";
+}
+
+static int tcm_vhost_port_link(
+ struct se_portal_group *se_tpg,
+ struct se_lun *lun)
+{
+ struct tcm_vhost_tpg *tv_tpg = container_of(se_tpg,
+ struct tcm_vhost_tpg, se_tpg);
+
+ atomic_inc(&tv_tpg->tv_tpg_port_count);
+ smp_mb__after_atomic_inc();
+
+ return 0;
+}
+
+static void tcm_vhost_port_unlink(
+ struct se_portal_group *se_tpg,
+ struct se_lun *se_lun)
+{
+ struct tcm_vhost_tpg *tv_tpg = container_of(se_tpg,
+ struct tcm_vhost_tpg, se_tpg);
+
+ atomic_dec(&tv_tpg->tv_tpg_port_count);
+ smp_mb__after_atomic_dec();
+}
+
+static struct se_node_acl *tcm_vhost_make_nodeacl(
+ struct se_portal_group *se_tpg,
+ struct config_group *group,
+ const char *name)
+{
+ struct se_node_acl *se_nacl, *se_nacl_new;
+ struct tcm_vhost_nacl *nacl;
+ u64 wwpn = 0;
+ u32 nexus_depth;
+
+ /* tcm_vhost_parse_wwn(name, &wwpn, 1) < 0)
+ return ERR_PTR(-EINVAL); */
+ se_nacl_new = tcm_vhost_alloc_fabric_acl(se_tpg);
+ if (!se_nacl_new)
+ return ERR_PTR(-ENOMEM);
+//#warning FIXME: Hardcoded nexus depth in tcm_vhost_make_nodeacl()
+ nexus_depth = 1;
+ /*
+ * se_nacl_new may be released by core_tpg_add_initiator_node_acl()
+ * when converting a NodeACL from demo mode -> explict
+ */
+ se_nacl = core_tpg_add_initiator_node_acl(se_tpg, se_nacl_new,
+ name, nexus_depth);
+ if (IS_ERR(se_nacl)) {
+ tcm_vhost_release_fabric_acl(se_tpg, se_nacl_new);
+ return se_nacl;
+ }
+ /*
+ * Locate our struct tcm_vhost_nacl and set the FC Nport WWPN
+ */
+ nacl = container_of(se_nacl, struct tcm_vhost_nacl, se_node_acl);
+ nacl->iport_wwpn = wwpn;
+ /* tcm_vhost_format_wwn(&nacl->iport_name[0], TCM_VHOST_NAMELEN, wwpn); */
+
+ return se_nacl;
+}
+
+static void tcm_vhost_drop_nodeacl(struct se_node_acl *se_acl)
+{
+ struct tcm_vhost_nacl *nacl = container_of(se_acl,
+ struct tcm_vhost_nacl, se_node_acl);
+ core_tpg_del_initiator_node_acl(se_acl->se_tpg, se_acl, 1);
+ kfree(nacl);
+}
+
+static int tcm_vhost_make_nexus(
+ struct tcm_vhost_tpg *tv_tpg,
+ const char *name)
+{
+ struct se_portal_group *se_tpg;
+ struct tcm_vhost_nexus *tv_nexus;
+
+ mutex_lock(&tv_tpg->tv_tpg_mutex);
+ if (tv_tpg->tpg_nexus) {
+ mutex_unlock(&tv_tpg->tv_tpg_mutex);
+ pr_debug("tv_tpg->tpg_nexus already exists\n");
+ return -EEXIST;
+ }
+ se_tpg = &tv_tpg->se_tpg;
+
+ tv_nexus = kzalloc(sizeof(struct tcm_vhost_nexus), GFP_KERNEL);
+ if (!tv_nexus) {
+ mutex_unlock(&tv_tpg->tv_tpg_mutex);
+ pr_err("Unable to allocate struct tcm_vhost_nexus\n");
+ return -ENOMEM;
+ }
+ /*
+ * Initialize the struct se_session pointer
+ */
+ tv_nexus->tvn_se_sess = transport_init_session();
+ if (IS_ERR(tv_nexus->tvn_se_sess)) {
+ mutex_unlock(&tv_tpg->tv_tpg_mutex);
+ kfree(tv_nexus);
+ return -ENOMEM;
+ }
+ /*
+ * Since we are running in 'demo mode' this call with generate a
+ * struct se_node_acl for the tcm_vhost struct se_portal_group with
+ * the SCSI Initiator port name of the passed configfs group 'name'.
+ */
+ tv_nexus->tvn_se_sess->se_node_acl = core_tpg_check_initiator_node_acl(
+ se_tpg, (unsigned char *)name);
+ if (!tv_nexus->tvn_se_sess->se_node_acl) {
+ mutex_unlock(&tv_tpg->tv_tpg_mutex);
+ pr_debug("core_tpg_check_initiator_node_acl() failed"
+ " for %s\n", name);
+ transport_free_session(tv_nexus->tvn_se_sess);
+ kfree(tv_nexus);
+ return -ENOMEM;
+ }
+ /*
+ * Now register the TCM vHost virtual I_T Nexus as active with the
+ * call to __transport_register_session()
+ */
+ __transport_register_session(se_tpg, tv_nexus->tvn_se_sess->se_node_acl,
+ tv_nexus->tvn_se_sess, tv_nexus);
+ tv_tpg->tpg_nexus = tv_nexus;
+
+ mutex_unlock(&tv_tpg->tv_tpg_mutex);
+ return 0;
+}
+
+static int tcm_vhost_drop_nexus(
+ struct tcm_vhost_tpg *tpg)
+{
+ struct se_session *se_sess;
+ struct tcm_vhost_nexus *tv_nexus;
+
+ mutex_lock(&tpg->tv_tpg_mutex);
+ tv_nexus = tpg->tpg_nexus;
+ if (!tv_nexus) {
+ mutex_unlock(&tpg->tv_tpg_mutex);
+ return -ENODEV;
+ }
+
+ se_sess = tv_nexus->tvn_se_sess;
+ if (!se_sess) {
+ mutex_unlock(&tpg->tv_tpg_mutex);
+ return -ENODEV;
+ }
+
+ if (atomic_read(&tpg->tv_tpg_port_count)) {
+ mutex_unlock(&tpg->tv_tpg_mutex);
+ pr_err("Unable to remove TCM_vHost I_T Nexus with"
+ " active TPG port count: %d\n",
+ atomic_read(&tpg->tv_tpg_port_count));
+ return -EPERM;
+ }
+
+ if (atomic_read(&tpg->tv_tpg_vhost_count)) {
+ mutex_unlock(&tpg->tv_tpg_mutex);
+ pr_err("Unable to remove TCM_vHost I_T Nexus with"
+ " active TPG vhost count: %d\n",
+ atomic_read(&tpg->tv_tpg_vhost_count));
+ return -EPERM;
+ }
+
+ pr_debug("TCM_vHost_ConfigFS: Removing I_T Nexus to emulated"
+ " %s Initiator Port: %s\n", tcm_vhost_dump_proto_id(tpg->tport),
+ tv_nexus->tvn_se_sess->se_node_acl->initiatorname);
+ /*
+ * Release the SCSI I_T Nexus to the emulated vHost Target Port
+ */
+ transport_deregister_session(tv_nexus->tvn_se_sess);
+ tpg->tpg_nexus = NULL;
+ mutex_unlock(&tpg->tv_tpg_mutex);
+
+ kfree(tv_nexus);
+ return 0;
+}
+
+static ssize_t tcm_vhost_tpg_show_nexus(
+ struct se_portal_group *se_tpg,
+ char *page)
+{
+ struct tcm_vhost_tpg *tv_tpg = container_of(se_tpg,
+ struct tcm_vhost_tpg, se_tpg);
+ struct tcm_vhost_nexus *tv_nexus;
+ ssize_t ret;
+
+ mutex_lock(&tv_tpg->tv_tpg_mutex);
+ tv_nexus = tv_tpg->tpg_nexus;
+ if (!tv_nexus) {
+ mutex_unlock(&tv_tpg->tv_tpg_mutex);
+ return -ENODEV;
+ }
+ ret = snprintf(page, PAGE_SIZE, "%s\n",
+ tv_nexus->tvn_se_sess->se_node_acl->initiatorname);
+ mutex_unlock(&tv_tpg->tv_tpg_mutex);
+
+ return ret;
+}
+
+static ssize_t tcm_vhost_tpg_store_nexus(
+ struct se_portal_group *se_tpg,
+ const char *page,
+ size_t count)
+{
+ struct tcm_vhost_tpg *tv_tpg = container_of(se_tpg,
+ struct tcm_vhost_tpg, se_tpg);
+ struct tcm_vhost_tport *tport_wwn = tv_tpg->tport;
+ unsigned char i_port[TCM_VHOST_NAMELEN], *ptr, *port_ptr;
+ int ret;
+ /*
+ * Shutdown the active I_T nexus if 'NULL' is passed..
+ */
+ if (!strncmp(page, "NULL", 4)) {
+ ret = tcm_vhost_drop_nexus(tv_tpg);
+ return (!ret) ? count : ret;
+ }
+ /*
+ * Otherwise make sure the passed virtual Initiator port WWN matches
+ * the fabric protocol_id set in tcm_vhost_make_tport(), and call
+ * tcm_vhost_make_nexus().
+ */
+ if (strlen(page) >= TCM_VHOST_NAMELEN) {
+ pr_err("Emulated NAA Sas Address: %s, exceeds"
+ " max: %d\n", page, TCM_VHOST_NAMELEN);
+ return -EINVAL;
+ }
+ snprintf(&i_port[0], TCM_VHOST_NAMELEN, "%s", page);
+
+ ptr = strstr(i_port, "naa.");
+ if (ptr) {
+ if (tport_wwn->tport_proto_id != SCSI_PROTOCOL_SAS) {
+ pr_err("Passed SAS Initiator Port %s does not"
+ " match target port protoid: %s\n", i_port,
+ tcm_vhost_dump_proto_id(tport_wwn));
+ return -EINVAL;
+ }
+ port_ptr = &i_port[0];
+ goto check_newline;
+ }
+ ptr = strstr(i_port, "fc.");
+ if (ptr) {
+ if (tport_wwn->tport_proto_id != SCSI_PROTOCOL_FCP) {
+ pr_err("Passed FCP Initiator Port %s does not"
+ " match target port protoid: %s\n", i_port,
+ tcm_vhost_dump_proto_id(tport_wwn));
+ return -EINVAL;
+ }
+ port_ptr = &i_port[3]; /* Skip over "fc." */
+ goto check_newline;
+ }
+ ptr = strstr(i_port, "iqn.");
+ if (ptr) {
+ if (tport_wwn->tport_proto_id != SCSI_PROTOCOL_ISCSI) {
+ pr_err("Passed iSCSI Initiator Port %s does not"
+ " match target port protoid: %s\n", i_port,
+ tcm_vhost_dump_proto_id(tport_wwn));
+ return -EINVAL;
+ }
+ port_ptr = &i_port[0];
+ goto check_newline;
+ }
+ pr_err("Unable to locate prefix for emulated Initiator Port:"
+ " %s\n", i_port);
+ return -EINVAL;
+ /*
+ * Clear any trailing newline for the NAA WWN
+ */
+check_newline:
+ if (i_port[strlen(i_port)-1] == '\n')
+ i_port[strlen(i_port)-1] = '\0';
+
+ ret = tcm_vhost_make_nexus(tv_tpg, port_ptr);
+ if (ret < 0)
+ return ret;
+
+ return count;
+}
+
+TF_TPG_BASE_ATTR(tcm_vhost, nexus, S_IRUGO | S_IWUSR);
+
+static struct configfs_attribute *tcm_vhost_tpg_attrs[] = {
+ &tcm_vhost_tpg_nexus.attr,
+ NULL,
+};
+
+static struct se_portal_group *tcm_vhost_make_tpg(
+ struct se_wwn *wwn,
+ struct config_group *group,
+ const char *name)
+{
+ struct tcm_vhost_tport*tport = container_of(wwn,
+ struct tcm_vhost_tport, tport_wwn);
+
+ struct tcm_vhost_tpg *tpg;
+ unsigned long tpgt;
+ int ret;
+
+ if (strstr(name, "tpgt_") != name)
+ return ERR_PTR(-EINVAL);
+ if (strict_strtoul(name + 5, 10, &tpgt) || tpgt > UINT_MAX)
+ return ERR_PTR(-EINVAL);
+
+ tpg = kzalloc(sizeof(struct tcm_vhost_tpg), GFP_KERNEL);
+ if (!tpg) {
+ pr_err("Unable to allocate struct tcm_vhost_tpg");
+ return ERR_PTR(-ENOMEM);
+ }
+ mutex_init(&tpg->tv_tpg_mutex);
+ INIT_LIST_HEAD(&tpg->tv_tpg_list);
+ tpg->tport = tport;
+ tpg->tport_tpgt = tpgt;
+
+ ret = core_tpg_register(&tcm_vhost_fabric_configfs->tf_ops, wwn,
+ &tpg->se_tpg, tpg, TRANSPORT_TPG_TYPE_NORMAL);
+ if (ret < 0) {
+ kfree(tpg);
+ return NULL;
+ }
+ mutex_lock(&tcm_vhost_mutex);
+ list_add_tail(&tpg->tv_tpg_list, &tcm_vhost_list);
+ mutex_unlock(&tcm_vhost_mutex);
+
+ return &tpg->se_tpg;
+}
+
+static void tcm_vhost_drop_tpg(struct se_portal_group *se_tpg)
+{
+ struct tcm_vhost_tpg *tpg = container_of(se_tpg,
+ struct tcm_vhost_tpg, se_tpg);
+
+ mutex_lock(&tcm_vhost_mutex);
+ list_del(&tpg->tv_tpg_list);
+ mutex_unlock(&tcm_vhost_mutex);
+ /*
+ * Release the virtual I_T Nexus for this vHost TPG
+ */
+ tcm_vhost_drop_nexus(tpg);
+ /*
+ * Deregister the se_tpg from TCM..
+ */
+ core_tpg_deregister(se_tpg);
+ kfree(tpg);
+}
+
+static struct se_wwn *tcm_vhost_make_tport(
+ struct target_fabric_configfs *tf,
+ struct config_group *group,
+ const char *name)
+{
+ struct tcm_vhost_tport *tport;
+ char *ptr;
+ u64 wwpn = 0;
+ int off = 0;
+
+ /* if (tcm_vhost_parse_wwn(name, &wwpn, 1) < 0)
+ return ERR_PTR(-EINVAL); */
+
+ tport = kzalloc(sizeof(struct tcm_vhost_tport), GFP_KERNEL);
+ if (!tport) {
+ pr_err("Unable to allocate struct tcm_vhost_tport");
+ return ERR_PTR(-ENOMEM);
+ }
+ tport->tport_wwpn = wwpn;
+ /* tcm_vhost_format_wwn(&tport->tport_name[0], TCM_VHOST__NAMELEN, wwpn); */
+ /*
+ * Determine the emulated Protocol Identifier and Target Port Name
+ * based on the incoming configfs directory name.
+ */
+ ptr = strstr(name, "naa.");
+ if (ptr) {
+ tport->tport_proto_id = SCSI_PROTOCOL_SAS;
+ goto check_len;
+ }
+ ptr = strstr(name, "fc.");
+ if (ptr) {
+ tport->tport_proto_id = SCSI_PROTOCOL_FCP;
+ off = 3; /* Skip over "fc." */
+ goto check_len;
+ }
+ ptr = strstr(name, "iqn.");
+ if (ptr) {
+ tport->tport_proto_id = SCSI_PROTOCOL_ISCSI;
+ goto check_len;
+ }
+
+ pr_err("Unable to locate prefix for emulated Target Port:"
+ " %s\n", name);
+ kfree(tport);
+ return ERR_PTR(-EINVAL);
+
+check_len:
+ if (strlen(name) >= TCM_VHOST_NAMELEN) {
+ pr_err("Emulated %s Address: %s, exceeds"
+ " max: %d\n", name, tcm_vhost_dump_proto_id(tport),
+ TCM_VHOST_NAMELEN);
+ kfree(tport);
+ return ERR_PTR(-EINVAL);
+ }
+ snprintf(&tport->tport_name[0], TCM_VHOST_NAMELEN, "%s", &name[off]);
+
+ pr_debug("TCM_VHost_ConfigFS: Allocated emulated Target"
+ " %s Address: %s\n", tcm_vhost_dump_proto_id(tport), name);
+
+ return &tport->tport_wwn;
+}
+
+static void tcm_vhost_drop_tport(struct se_wwn *wwn)
+{
+ struct tcm_vhost_tport *tport = container_of(wwn,
+ struct tcm_vhost_tport, tport_wwn);
+
+ pr_debug("TCM_VHost_ConfigFS: Deallocating emulated Target"
+ " %s Address: %s\n", tcm_vhost_dump_proto_id(tport),
+ tport->tport_name);;
+
+ kfree(tport);
+}
+
+static ssize_t tcm_vhost_wwn_show_attr_version(
+ struct target_fabric_configfs *tf,
+ char *page)
+{
+ return sprintf(page, "TCM_VHOST fabric module %s on %s/%s"
+ "on "UTS_RELEASE"\n", TCM_VHOST_VERSION, utsname()->sysname,
+ utsname()->machine);
+}
+
+TF_WWN_ATTR_RO(tcm_vhost, version);
+
+static struct configfs_attribute *tcm_vhost_wwn_attrs[] = {
+ &tcm_vhost_wwn_version.attr,
+ NULL,
+};
+
+static struct target_core_fabric_ops tcm_vhost_ops = {
+ .get_fabric_name = tcm_vhost_get_fabric_name,
+ .get_fabric_proto_ident = tcm_vhost_get_fabric_proto_ident,
+ .tpg_get_wwn = tcm_vhost_get_fabric_wwn,
+ .tpg_get_tag = tcm_vhost_get_tag,
+ .tpg_get_default_depth = tcm_vhost_get_default_depth,
+ .tpg_get_pr_transport_id = tcm_vhost_get_pr_transport_id,
+ .tpg_get_pr_transport_id_len = tcm_vhost_get_pr_transport_id_len,
+ .tpg_parse_pr_out_transport_id = tcm_vhost_parse_pr_out_transport_id,
+ .tpg_check_demo_mode = tcm_vhost_check_true,
+ .tpg_check_demo_mode_cache = tcm_vhost_check_true,
+ .tpg_check_demo_mode_write_protect = tcm_vhost_check_false,
+ .tpg_check_prod_mode_write_protect = tcm_vhost_check_false,
+ .tpg_alloc_fabric_acl = tcm_vhost_alloc_fabric_acl,
+ .tpg_release_fabric_acl = tcm_vhost_release_fabric_acl,
+ .tpg_get_inst_index = tcm_vhost_tpg_get_inst_index,
+ .release_cmd = tcm_vhost_release_cmd,
+ .shutdown_session = tcm_vhost_shutdown_session,
+ .close_session = tcm_vhost_close_session,
+ .sess_get_index = tcm_vhost_sess_get_index,
+ .sess_get_initiator_sid = NULL,
+ .write_pending = tcm_vhost_write_pending,
+ .write_pending_status = tcm_vhost_write_pending_status,
+ .set_default_node_attributes = tcm_vhost_set_default_node_attrs,
+ .get_task_tag = tcm_vhost_get_task_tag,
+ .get_cmd_state = tcm_vhost_get_cmd_state,
+ .queue_data_in = tcm_vhost_queue_data_in,
+ .queue_status = tcm_vhost_queue_status,
+ .queue_tm_rsp = tcm_vhost_queue_tm_rsp,
+ .get_fabric_sense_len = tcm_vhost_get_fabric_sense_len,
+ .set_fabric_sense_len = tcm_vhost_set_fabric_sense_len,
+ /*
+ * Setup function pointers for generic logic in target_core_fabric_configfs.c
+ */
+ .fabric_make_wwn = tcm_vhost_make_tport,
+ .fabric_drop_wwn = tcm_vhost_drop_tport,
+ .fabric_make_tpg = tcm_vhost_make_tpg,
+ .fabric_drop_tpg = tcm_vhost_drop_tpg,
+ .fabric_post_link = tcm_vhost_port_link,
+ .fabric_pre_unlink = tcm_vhost_port_unlink,
+ .fabric_make_np = NULL,
+ .fabric_drop_np = NULL,
+ .fabric_make_nodeacl = tcm_vhost_make_nodeacl,
+ .fabric_drop_nodeacl = tcm_vhost_drop_nodeacl,
+};
+
+static int tcm_vhost_register_configfs(void)
+{
+ struct target_fabric_configfs *fabric;
+ int ret;
+
+ pr_debug("TCM_VHOST fabric module %s on %s/%s"
+ " on "UTS_RELEASE"\n",TCM_VHOST_VERSION, utsname()->sysname,
+ utsname()->machine);
+ /*
+ * Register the top level struct config_item_type with TCM core
+ */
+ fabric = target_fabric_configfs_init(THIS_MODULE, "vhost");
+ if (IS_ERR(fabric)) {
+ pr_err("target_fabric_configfs_init() failed\n");
+ return PTR_ERR(fabric);
+ }
+ /*
+ * Setup fabric->tf_ops from our local tcm_vhost_ops
+ */
+ fabric->tf_ops = tcm_vhost_ops;
+ /*
+ * Setup default attribute lists for various fabric->tf_cit_tmpl
+ */
+ TF_CIT_TMPL(fabric)->tfc_wwn_cit.ct_attrs = tcm_vhost_wwn_attrs;
+ TF_CIT_TMPL(fabric)->tfc_tpg_base_cit.ct_attrs = tcm_vhost_tpg_attrs;
+ TF_CIT_TMPL(fabric)->tfc_tpg_attrib_cit.ct_attrs = NULL;
+ TF_CIT_TMPL(fabric)->tfc_tpg_param_cit.ct_attrs = NULL;
+ TF_CIT_TMPL(fabric)->tfc_tpg_np_base_cit.ct_attrs = NULL;
+ TF_CIT_TMPL(fabric)->tfc_tpg_nacl_base_cit.ct_attrs = NULL;
+ TF_CIT_TMPL(fabric)->tfc_tpg_nacl_attrib_cit.ct_attrs = NULL;
+ TF_CIT_TMPL(fabric)->tfc_tpg_nacl_auth_cit.ct_attrs = NULL;
+ TF_CIT_TMPL(fabric)->tfc_tpg_nacl_param_cit.ct_attrs = NULL;
+ /*
+ * Register the fabric for use within TCM
+ */
+ ret = target_fabric_configfs_register(fabric);
+ if (ret < 0) {
+ pr_err("target_fabric_configfs_register() failed"
+ " for TCM_VHOST\n");
+ return ret;
+ }
+ /*
+ * Setup our local pointer to *fabric
+ */
+ tcm_vhost_fabric_configfs = fabric;
+ pr_debug("TCM_VHOST[0] - Set fabric -> tcm_vhost_fabric_configfs\n");
+ return 0;
+};
+
+static void tcm_vhost_deregister_configfs(void)
+{
+ if (!tcm_vhost_fabric_configfs)
+ return;
+
+ target_fabric_configfs_deregister(tcm_vhost_fabric_configfs);
+ tcm_vhost_fabric_configfs = NULL;
+ pr_debug("TCM_VHOST[0] - Cleared tcm_vhost_fabric_configfs\n");
+};
+
+static int __init tcm_vhost_init(void)
+{
+ int ret = -ENOMEM;
+
+ tcm_vhost_workqueue = alloc_workqueue("tcm_vhost", 0, 0);
+ if (!tcm_vhost_workqueue)
+ goto out;
+
+ ret = vhost_scsi_register();
+ if (ret < 0)
+ goto out_destroy_workqueue;
+
+ ret = tcm_vhost_register_configfs();
+ if (ret < 0)
+ goto out_vhost_scsi_deregister;
+
+ return 0;
+
+out_vhost_scsi_deregister:
+ vhost_scsi_deregister();
+out_destroy_workqueue:
+ destroy_workqueue(tcm_vhost_workqueue);
+out:
+ return ret;
+};
+
+static void tcm_vhost_exit(void)
+{
+ tcm_vhost_deregister_configfs();
+ vhost_scsi_deregister();
+ destroy_workqueue(tcm_vhost_workqueue);
+};
+
+MODULE_DESCRIPTION("TCM_VHOST series fabric driver");
+MODULE_LICENSE("GPL");
+module_init(tcm_vhost_init);
+module_exit(tcm_vhost_exit);
diff --git a/drivers/vhost/tcm_vhost.h b/drivers/vhost/tcm_vhost.h
new file mode 100644
index 0000000..e942df9
--- /dev/null
+++ b/drivers/vhost/tcm_vhost.h
@@ -0,0 +1,90 @@
+#define TCM_VHOST_VERSION "v0.1"
+#define TCM_VHOST_NAMELEN 256
+#define TCM_VHOST_MAX_CDB_SIZE 32
+
+struct tcm_vhost_cmd {
+ /* Descriptor from vhost_get_vq_desc() for virt_queue segment */
+ int tvc_vq_desc;
+ /* The Tag from include/linux/virtio_scsi.h:struct virtio_scsi_cmd_req */
+ u64 tvc_tag;
+ /* The number of scatterlists associated with this cmd */
+ u32 tvc_sgl_count;
+ /* Saved unpacked SCSI LUN for tcm_vhost_submission_work() */
+ u32 tvc_lun;
+ /* Pointer to the SGL formatted memory from virtio-scsi */
+ struct scatterlist *tvc_sgl;
+ /* Pointer to response */
+ struct virtio_scsi_cmd_resp __user *tvc_resp;
+ /* Pointer to vhost_scsi for our device */
+ struct vhost_scsi *tvc_vhost;
+ /* The TCM I/O descriptor that is accessed via container_of() */
+ struct se_cmd tvc_se_cmd;
+ /* work item used for cmwq dispatch to tcm_vhost_submission_work() */
+ struct work_struct work;
+ /* Copy of the incoming SCSI command descriptor block (CDB) */
+ unsigned char tvc_cdb[TCM_VHOST_MAX_CDB_SIZE];
+ /* Sense buffer that will be mapped into outgoing status */
+ unsigned char tvc_sense_buf[TRANSPORT_SENSE_BUFFER];
+ /* Completed commands list, serviced from vhost worker thread */
+ struct list_head tvc_completion_list;
+};
+
+struct tcm_vhost_nexus {
+ /* Pointer to TCM session for I_T Nexus */
+ struct se_session *tvn_se_sess;
+};
+
+struct tcm_vhost_nacl {
+ /* Binary World Wide unique Port Name for Vhost Initiator port */
+ u64 iport_wwpn;
+ /* ASCII formatted WWPN for Sas Initiator port */
+ char iport_name[TCM_VHOST_NAMELEN];
+ /* Returned by tcm_vhost_make_nodeacl() */
+ struct se_node_acl se_node_acl;
+};
+
+struct tcm_vhost_tpg {
+ /* Vhost port target portal group tag for TCM */
+ u16 tport_tpgt;
+ /* Used to track number of TPG Port/Lun Links wrt to explict I_T Nexus shutdown */
+ atomic_t tv_tpg_port_count;
+ /* Used for vhost_scsi device reference to tpg_nexus */
+ atomic_t tv_tpg_vhost_count;
+ /* list for tcm_vhost_list */
+ struct list_head tv_tpg_list;
+ /* Used to protect access for tpg_nexus */
+ struct mutex tv_tpg_mutex;
+ /* Pointer to the TCM VHost I_T Nexus for this TPG endpoint */
+ struct tcm_vhost_nexus *tpg_nexus;
+ /* Pointer back to tcm_vhost_tport */
+ struct tcm_vhost_tport *tport;
+ /* Returned by tcm_vhost_make_tpg() */
+ struct se_portal_group se_tpg;
+};
+
+struct tcm_vhost_tport {
+ /* SCSI protocol the tport is providing */
+ u8 tport_proto_id;
+ /* Binary World Wide unique Port Name for Vhost Target port */
+ u64 tport_wwpn;
+ /* ASCII formatted WWPN for Vhost Target port */
+ char tport_name[TCM_VHOST_NAMELEN];
+ /* Returned by tcm_vhost_make_tport() */
+ struct se_wwn tport_wwn;
+};
+
+/*
+ * As per request from MST, keep TCM_VHOST related ioctl defines out of
+ * linux/vhost.h (user-space) for now..
+ */
+
+#include <linux/vhost.h>
+
+struct vhost_scsi_target {
+ unsigned char vhost_wwpn[TRANSPORT_IQN_LEN];
+ unsigned short vhost_tpgt;
+};
+
+/* VHOST_SCSI specific defines */
+#define VHOST_SCSI_SET_ENDPOINT _IOW(VHOST_VIRTIO, 0x40, struct vhost_scsi_target)
+#define VHOST_SCSI_CLEAR_ENDPOINT _IOW(VHOST_VIRTIO, 0x41, struct vhost_scsi_target)
--
1.7.2.5
^ permalink raw reply related
* Re: [net-next RFC V5 4/5] virtio_net: multiqueue support
From: Sasha Levin @ 2012-07-21 12:02 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: krkumar2, habanero, kvm, netdev, mashirle, linux-kernel,
virtualization, edumazet, tahm, jwhan, davem, sri
In-Reply-To: <20120720134014.GD16550@redhat.com>
On 07/20/2012 03:40 PM, Michael S. Tsirkin wrote:
>> - err = init_vqs(vi);
>> > + if (virtio_has_feature(vdev, VIRTIO_NET_F_CTRL_VQ))
>> > + vi->has_cvq = true;
>> > +
> How about we disable multiqueue if there's no cvq?
> Will make logic a bit simpler, won't it?
multiqueues don't really depend on cvq. Does this added complexity really justifies adding an artificial limit?
^ permalink raw reply
* Re: [RFC-v4 1/3] vhost: Separate vhost-net features from vhost features
From: Michael S. Tsirkin @ 2012-07-21 22:26 UTC (permalink / raw)
To: Nicholas A. Bellinger
Cc: Jens Axboe, Stefan Hajnoczi, kvm-devel, Greg Kroah-Hartman,
Zhi Yong Wu, Anthony Liguori, target-devel, linux-scsi,
Paolo Bonzini, lf-virt, Christoph Hellwig
In-Reply-To: <1342853738-27682-2-git-send-email-nab@linux-iscsi.org>
On Sat, Jul 21, 2012 at 06:55:36AM +0000, Nicholas A. Bellinger wrote:
> From: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
>
> In order for other vhost devices to use the VHOST_FEATURES bits the
> vhost-net specific bits need to be moved to their own VHOST_NET_FEATURES
> constant.
>
> (Asias: Update drivers/vhost/test.c to use VHOST_NET_FEATURES)
>
> Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> Cc: Zhi Yong Wu <wuzhy@cn.ibm.com>
> Cc: Michael S. Tsirkin <mst@redhat.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Asias He <asias@redhat.com>
> Signed-off-by: Nicholas A. Bellinger <nab@risingtidesystems.com>
Applied to vhost-next.
> ---
> drivers/vhost/net.c | 4 ++--
> drivers/vhost/test.c | 4 ++--
> drivers/vhost/vhost.h | 3 ++-
> 3 files changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
> index f82a739..072cbba 100644
> --- a/drivers/vhost/net.c
> +++ b/drivers/vhost/net.c
> @@ -823,14 +823,14 @@ static long vhost_net_ioctl(struct file *f, unsigned int ioctl,
> return -EFAULT;
> return vhost_net_set_backend(n, backend.index, backend.fd);
> case VHOST_GET_FEATURES:
> - features = VHOST_FEATURES;
> + features = VHOST_NET_FEATURES;
> if (copy_to_user(featurep, &features, sizeof features))
> return -EFAULT;
> return 0;
> case VHOST_SET_FEATURES:
> if (copy_from_user(&features, featurep, sizeof features))
> return -EFAULT;
> - if (features & ~VHOST_FEATURES)
> + if (features & ~VHOST_NET_FEATURES)
> return -EOPNOTSUPP;
> return vhost_net_set_features(n, features);
> case VHOST_RESET_OWNER:
> diff --git a/drivers/vhost/test.c b/drivers/vhost/test.c
> index 3de00d9..91d6f06 100644
> --- a/drivers/vhost/test.c
> +++ b/drivers/vhost/test.c
> @@ -261,14 +261,14 @@ static long vhost_test_ioctl(struct file *f, unsigned int ioctl,
> return -EFAULT;
> return vhost_test_run(n, test);
> case VHOST_GET_FEATURES:
> - features = VHOST_FEATURES;
> + features = VHOST_NET_FEATURES;
> if (copy_to_user(featurep, &features, sizeof features))
> return -EFAULT;
> return 0;
> case VHOST_SET_FEATURES:
> if (copy_from_user(&features, featurep, sizeof features))
> return -EFAULT;
> - if (features & ~VHOST_FEATURES)
> + if (features & ~VHOST_NET_FEATURES)
> return -EOPNOTSUPP;
> return vhost_test_set_features(n, features);
> case VHOST_RESET_OWNER:
> diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
> index 8de1fd5..07b9763 100644
> --- a/drivers/vhost/vhost.h
> +++ b/drivers/vhost/vhost.h
> @@ -201,7 +201,8 @@ enum {
> VHOST_FEATURES = (1ULL << VIRTIO_F_NOTIFY_ON_EMPTY) |
> (1ULL << VIRTIO_RING_F_INDIRECT_DESC) |
> (1ULL << VIRTIO_RING_F_EVENT_IDX) |
> - (1ULL << VHOST_F_LOG_ALL) |
> + (1ULL << VHOST_F_LOG_ALL),
> + VHOST_NET_FEATURES = VHOST_FEATURES |
> (1ULL << VHOST_NET_F_VIRTIO_NET_HDR) |
> (1ULL << VIRTIO_NET_F_MRG_RXBUF),
> };
> --
> 1.7.2.5
^ permalink raw reply
* Re: [RFC-v4 2/3] vhost: make vhost work queue visible
From: Michael S. Tsirkin @ 2012-07-21 22:26 UTC (permalink / raw)
To: Nicholas A. Bellinger
Cc: Jens Axboe, Stefan Hajnoczi, kvm-devel, Greg Kroah-Hartman,
Zhi Yong Wu, Anthony Liguori, target-devel, linux-scsi,
Paolo Bonzini, lf-virt, Christoph Hellwig
In-Reply-To: <1342853738-27682-3-git-send-email-nab@linux-iscsi.org>
On Sat, Jul 21, 2012 at 06:55:37AM +0000, Nicholas A. Bellinger wrote:
> From: Stefan Hajnoczi <stefanha@gmail.com>
>
> The vhost work queue allows processing to be done in vhost worker thread
> context, which uses the owner process mm. Access to the vring and guest
> memory is typically only possible from vhost worker context so it is
> useful to allow work to be queued directly by users.
>
> Currently vhost_net only uses the poll wrappers which do not expose the
> work queue functions. However, for tcm_vhost (vhost_scsi) it will be
> necessary to queue custom work.
>
> Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> Cc: Zhi Yong Wu <wuzhy@cn.ibm.com>
> Cc: Michael S. Tsirkin <mst@redhat.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
Applied to vhost-next.
> ---
> drivers/vhost/vhost.c | 5 ++---
> drivers/vhost/vhost.h | 3 +++
> 2 files changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
> index 94dbd25..1aab08b 100644
> --- a/drivers/vhost/vhost.c
> +++ b/drivers/vhost/vhost.c
> @@ -64,7 +64,7 @@ static int vhost_poll_wakeup(wait_queue_t *wait, unsigned mode, int sync,
> return 0;
> }
>
> -static void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn)
> +void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn)
> {
> INIT_LIST_HEAD(&work->node);
> work->fn = fn;
> @@ -137,8 +137,7 @@ void vhost_poll_flush(struct vhost_poll *poll)
> vhost_work_flush(poll->dev, &poll->work);
> }
>
> -static inline void vhost_work_queue(struct vhost_dev *dev,
> - struct vhost_work *work)
> +void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work)
> {
> unsigned long flags;
>
> diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
> index 07b9763..1125af3 100644
> --- a/drivers/vhost/vhost.h
> +++ b/drivers/vhost/vhost.h
> @@ -43,6 +43,9 @@ struct vhost_poll {
> struct vhost_dev *dev;
> };
>
> +void vhost_work_init(struct vhost_work *work, vhost_work_fn_t fn);
> +void vhost_work_queue(struct vhost_dev *dev, struct vhost_work *work);
> +
> void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn,
> unsigned long mask, struct vhost_dev *dev);
> void vhost_poll_start(struct vhost_poll *poll, struct file *file);
> --
> 1.7.2.5
^ permalink raw reply
* Re: [RFC-v4 3/3] tcm_vhost: Initial merge for vhost level target fabric driver
From: Michael S. Tsirkin @ 2012-07-21 22:50 UTC (permalink / raw)
To: Nicholas A. Bellinger
Cc: Jens Axboe, Stefan Hajnoczi, kvm-devel, Greg Kroah-Hartman,
Zhi Yong Wu, Anthony Liguori, target-devel, linux-scsi,
Paolo Bonzini, lf-virt, Christoph Hellwig
In-Reply-To: <1342853738-27682-4-git-send-email-nab@linux-iscsi.org>
On Sat, Jul 21, 2012 at 06:55:38AM +0000, Nicholas A. Bellinger wrote:
> From: Nicholas Bellinger <nab@linux-iscsi.org>
>
> This patch adds the initial code for tcm_vhost, a Vhost level TCM
> fabric driver for virtio SCSI initiators into KVM guest.
>
> This code is currently up and running on v3.5-rc2 host+guest along
> with the virtio-scsi vdev->scan() patch to allow a proper
> scsi_scan_host() to occur once the tcm_vhost nexus has been established
> by the paravirtualized virtio-scsi client here:
>
> virtio-scsi: Add vdrv->scan for post VIRTIO_CONFIG_S_DRIVER_OK LUN scanning
> http://marc.info/?l=linux-scsi&m=134160609212542&w=2
>
> Using tcm_vhost requires Zhi's -> Stefan's qemu vhost-scsi tree here:
>
> https://github.com/wuzhy/qemu/tree/vhost-scsi
>
> along with the recent QEMU patch to hw/virtio-scsi.c to set max_target=0
> during vhost-scsi operation.
>
> Changelog v3 -> v4:
>
> Rename vhost_vring_target -> vhost_scsi_target (mst + nab)
> Use TRANSPORT_IQN_LEN in vhost_scsi_target->vhost_wwpn[] def (nab)
> Move back to drivers/vhost/, and just use drivers/vhost/Kconfig.tcm (mst)
> Move TCM_VHOST related ioctl defines from include/linux/vhost.h ->
> drivers/vhost/tcm_vhost.h as requested by MST (nab)
> Move Kbuild.tcm include from drivers/staging -> drivers/vhost/, and
> just use 'if STAGING' around 'source drivers/vhost/Kbuild.tcm'
>
> Changelog v2 -> v3:
>
> Unlock on error in tcm_vhost_drop_nexus() (DanC)
> Fix strlen() doesn't count the terminator (DanC)
> Call kfree() on an error path (DanC)
> Convert tcm_vhost_write_pending to use target_execute_cmd (hch + nab)
> Fix another strlen() off by one in tcm_vhost_make_tport (DanC)
> Add option under drivers/staging/Kconfig, and move to drivers/vhost/tcm/
> as requested by MST (nab)
>
> Changelog v1 -> v2:
>
> Fix tv_cmd completion -> release SGL memory leak (nab)
> Fix sparse warnings for static variable usage ((Fengguang Wu)
> Fix sparse warnings for min() typing + printk format specs (Fengguang Wu)
> Convert to cmwq submission for I/O dispatch (nab + hch)
>
> Changelog v0 -> v1:
>
> Merge into single source + header file, and move to drivers/vhost/
>
> Cc: Michael S. Tsirkin <mst@redhat.com>
> Cc: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> Cc: Zhi Yong Wu <wuzhy@cn.ibm.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Hannes Reinecke <hare@suse.de>
> Cc: Jens Axboe <axboe@kernel.dk>
> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
There are some comments that I sent that have not been addressed yet,
in particular it's easy for userspace to flood the host log.
But I think it's in an OK state for a staging driver.
I don't know if the dependency on CONFIG_STAGING is enough:
it's easy for someone to miss, if it was in the menu with other
stagung drivers it woul be more obvious. If the way to do this is to
move it to drivers/staging/tcp_vhost it might be worth it, I'm fine
either way. Anyway, I think we both agree it should go in through
Greg's tree so he's the boss.
I put the dependency patches 1 and 2 on my tree
git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git vhost-net-next
and sending pull request to dave ASAP.
> ---
> drivers/vhost/Kconfig | 3 +
> drivers/vhost/Kconfig.tcm | 6 +
> drivers/vhost/Makefile | 2 +
> drivers/vhost/tcm_vhost.c | 1611 +++++++++++++++++++++++++++++++++++++++++++++
> drivers/vhost/tcm_vhost.h | 90 +++
> 5 files changed, 1712 insertions(+), 0 deletions(-)
> create mode 100644 drivers/vhost/Kconfig.tcm
> create mode 100644 drivers/vhost/tcm_vhost.c
> create mode 100644 drivers/vhost/tcm_vhost.h
>
> diff --git a/drivers/vhost/Kconfig b/drivers/vhost/Kconfig
> index e4e2fd1..202bba6 100644
> --- a/drivers/vhost/Kconfig
> +++ b/drivers/vhost/Kconfig
> @@ -9,3 +9,6 @@ config VHOST_NET
> To compile this driver as a module, choose M here: the module will
> be called vhost_net.
>
> +if STAGING
> +source "drivers/vhost/Kconfig.tcm"
> +endif
> diff --git a/drivers/vhost/Kconfig.tcm b/drivers/vhost/Kconfig.tcm
> new file mode 100644
> index 0000000..a9c6f76
> --- /dev/null
> +++ b/drivers/vhost/Kconfig.tcm
> @@ -0,0 +1,6 @@
> +config TCM_VHOST
> + tristate "TCM_VHOST fabric module (EXPERIMENTAL)"
> + depends on TARGET_CORE && EVENTFD && EXPERIMENTAL && m
> + default n
> + ---help---
> + Say M here to enable the TCM_VHOST fabric module for use with virtio-scsi guests
> diff --git a/drivers/vhost/Makefile b/drivers/vhost/Makefile
> index 72dd020..a27b053 100644
> --- a/drivers/vhost/Makefile
> +++ b/drivers/vhost/Makefile
> @@ -1,2 +1,4 @@
> obj-$(CONFIG_VHOST_NET) += vhost_net.o
> vhost_net-y := vhost.o net.o
> +
> +obj-$(CONFIG_TCM_VHOST) += tcm_vhost.o
> diff --git a/drivers/vhost/tcm_vhost.c b/drivers/vhost/tcm_vhost.c
> new file mode 100644
> index 0000000..dc7e024
> --- /dev/null
> +++ b/drivers/vhost/tcm_vhost.c
> @@ -0,0 +1,1611 @@
> +/*******************************************************************************
> + * Vhost kernel TCM fabric driver for virtio SCSI initiators
> + *
> + * (C) Copyright 2010-2012 RisingTide Systems LLC.
> + * (C) Copyright 2010-2012 IBM Corp.
> + *
> + * Licensed to the Linux Foundation under the General Public License (GPL) version 2.
> + *
> + * Authors: Nicholas A. Bellinger <nab@risingtidesystems.com>
> + * Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + ****************************************************************************/
> +
> +#include <linux/module.h>
> +#include <linux/moduleparam.h>
> +#include <generated/utsrelease.h>
> +#include <linux/utsname.h>
> +#include <linux/init.h>
> +#include <linux/slab.h>
> +#include <linux/kthread.h>
> +#include <linux/types.h>
> +#include <linux/string.h>
> +#include <linux/configfs.h>
> +#include <linux/ctype.h>
> +#include <linux/compat.h>
> +#include <linux/eventfd.h>
> +#include <linux/vhost.h>
> +#include <linux/fs.h>
> +#include <linux/miscdevice.h>
> +#include <asm/unaligned.h>
> +#include <scsi/scsi.h>
> +#include <scsi/scsi_tcq.h>
> +#include <target/target_core_base.h>
> +#include <target/target_core_fabric.h>
> +#include <target/target_core_fabric_configfs.h>
> +#include <target/target_core_configfs.h>
> +#include <target/configfs_macros.h>
> +#include <linux/vhost.h>
> +#include <linux/virtio_net.h> /* TODO vhost.h currently depends on this */
> +#include <linux/virtio_scsi.h>
> +
> +#include "vhost.c"
> +#include "vhost.h"
> +#include "tcm_vhost.h"
> +
> +struct vhost_scsi {
> + atomic_t vhost_ref_cnt;
> + struct tcm_vhost_tpg *vs_tpg;
> + struct vhost_dev dev;
> + struct vhost_virtqueue vqs[3];
> +
> + struct vhost_work vs_completion_work; /* cmd completion work item */
> + struct list_head vs_completion_list; /* cmd completion queue */
> + spinlock_t vs_completion_lock; /* protects s_completion_list */
> +};
> +
> +/* Local pointer to allocated TCM configfs fabric module */
> +static struct target_fabric_configfs *tcm_vhost_fabric_configfs;
> +
> +static struct workqueue_struct *tcm_vhost_workqueue;
> +
> +/* Global spinlock to protect tcm_vhost TPG list for vhost IOCTL access */
> +static DEFINE_MUTEX(tcm_vhost_mutex);
> +static LIST_HEAD(tcm_vhost_list);
> +
> +static int tcm_vhost_check_true(struct se_portal_group *se_tpg)
> +{
> + return 1;
> +}
> +
> +static int tcm_vhost_check_false(struct se_portal_group *se_tpg)
> +{
> + return 0;
> +}
> +
> +static char *tcm_vhost_get_fabric_name(void)
> +{
> + return "vhost";
> +}
> +
> +static u8 tcm_vhost_get_fabric_proto_ident(struct se_portal_group *se_tpg)
> +{
> + struct tcm_vhost_tpg *tpg = container_of(se_tpg,
> + struct tcm_vhost_tpg, se_tpg);
> + struct tcm_vhost_tport *tport = tpg->tport;
> +
> + switch (tport->tport_proto_id) {
> + case SCSI_PROTOCOL_SAS:
> + return sas_get_fabric_proto_ident(se_tpg);
> + case SCSI_PROTOCOL_FCP:
> + return fc_get_fabric_proto_ident(se_tpg);
> + case SCSI_PROTOCOL_ISCSI:
> + return iscsi_get_fabric_proto_ident(se_tpg);
> + default:
> + pr_err("Unknown tport_proto_id: 0x%02x, using"
> + " SAS emulation\n", tport->tport_proto_id);
> + break;
> + }
> +
> + return sas_get_fabric_proto_ident(se_tpg);
> +}
> +
> +static char *tcm_vhost_get_fabric_wwn(struct se_portal_group *se_tpg)
> +{
> + struct tcm_vhost_tpg *tpg = container_of(se_tpg,
> + struct tcm_vhost_tpg, se_tpg);
> + struct tcm_vhost_tport *tport = tpg->tport;
> +
> + return &tport->tport_name[0];
> +}
> +
> +static u16 tcm_vhost_get_tag(struct se_portal_group *se_tpg)
> +{
> + struct tcm_vhost_tpg *tpg = container_of(se_tpg,
> + struct tcm_vhost_tpg, se_tpg);
> + return tpg->tport_tpgt;
> +}
> +
> +static u32 tcm_vhost_get_default_depth(struct se_portal_group *se_tpg)
> +{
> + return 1;
> +}
> +
> +static u32 tcm_vhost_get_pr_transport_id(
> + struct se_portal_group *se_tpg,
> + struct se_node_acl *se_nacl,
> + struct t10_pr_registration *pr_reg,
> + int *format_code,
> + unsigned char *buf)
> +{
> + struct tcm_vhost_tpg *tpg = container_of(se_tpg,
> + struct tcm_vhost_tpg, se_tpg);
> + struct tcm_vhost_tport *tport = tpg->tport;
> +
> + switch (tport->tport_proto_id) {
> + case SCSI_PROTOCOL_SAS:
> + return sas_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
> + format_code, buf);
> + case SCSI_PROTOCOL_FCP:
> + return fc_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
> + format_code, buf);
> + case SCSI_PROTOCOL_ISCSI:
> + return iscsi_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
> + format_code, buf);
> + default:
> + pr_err("Unknown tport_proto_id: 0x%02x, using"
> + " SAS emulation\n", tport->tport_proto_id);
> + break;
> + }
> +
> + return sas_get_pr_transport_id(se_tpg, se_nacl, pr_reg,
> + format_code, buf);
> +}
> +
> +static u32 tcm_vhost_get_pr_transport_id_len(
> + struct se_portal_group *se_tpg,
> + struct se_node_acl *se_nacl,
> + struct t10_pr_registration *pr_reg,
> + int *format_code)
> +{
> + struct tcm_vhost_tpg *tpg = container_of(se_tpg,
> + struct tcm_vhost_tpg, se_tpg);
> + struct tcm_vhost_tport *tport = tpg->tport;
> +
> + switch (tport->tport_proto_id) {
> + case SCSI_PROTOCOL_SAS:
> + return sas_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
> + format_code);
> + case SCSI_PROTOCOL_FCP:
> + return fc_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
> + format_code);
> + case SCSI_PROTOCOL_ISCSI:
> + return iscsi_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
> + format_code);
> + default:
> + pr_err("Unknown tport_proto_id: 0x%02x, using"
> + " SAS emulation\n", tport->tport_proto_id);
> + break;
> + }
> +
> + return sas_get_pr_transport_id_len(se_tpg, se_nacl, pr_reg,
> + format_code);
> +}
> +
> +static char *tcm_vhost_parse_pr_out_transport_id(
> + struct se_portal_group *se_tpg,
> + const char *buf,
> + u32 *out_tid_len,
> + char **port_nexus_ptr)
> +{
> + struct tcm_vhost_tpg *tpg = container_of(se_tpg,
> + struct tcm_vhost_tpg, se_tpg);
> + struct tcm_vhost_tport *tport = tpg->tport;
> +
> + switch (tport->tport_proto_id) {
> + case SCSI_PROTOCOL_SAS:
> + return sas_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
> + port_nexus_ptr);
> + case SCSI_PROTOCOL_FCP:
> + return fc_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
> + port_nexus_ptr);
> + case SCSI_PROTOCOL_ISCSI:
> + return iscsi_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
> + port_nexus_ptr);
> + default:
> + pr_err("Unknown tport_proto_id: 0x%02x, using"
> + " SAS emulation\n", tport->tport_proto_id);
> + break;
> + }
> +
> + return sas_parse_pr_out_transport_id(se_tpg, buf, out_tid_len,
> + port_nexus_ptr);
> +}
> +
> +static struct se_node_acl *tcm_vhost_alloc_fabric_acl(struct se_portal_group *se_tpg)
> +{
> + struct tcm_vhost_nacl *nacl;
> +
> + nacl = kzalloc(sizeof(struct tcm_vhost_nacl), GFP_KERNEL);
> + if (!nacl) {
> + pr_err("Unable to alocate struct tcm_vhost_nacl\n");
> + return NULL;
> + }
> +
> + return &nacl->se_node_acl;
> +}
> +
> +static void tcm_vhost_release_fabric_acl(
> + struct se_portal_group *se_tpg,
> + struct se_node_acl *se_nacl)
> +{
> + struct tcm_vhost_nacl *nacl = container_of(se_nacl,
> + struct tcm_vhost_nacl, se_node_acl);
> + kfree(nacl);
> +}
> +
> +static u32 tcm_vhost_tpg_get_inst_index(struct se_portal_group *se_tpg)
> +{
> + return 1;
> +}
> +
> +static void tcm_vhost_release_cmd(struct se_cmd *se_cmd)
> +{
> + return;
> +}
> +
> +static int tcm_vhost_shutdown_session(struct se_session *se_sess)
> +{
> + return 0;
> +}
> +
> +static void tcm_vhost_close_session(struct se_session *se_sess)
> +{
> + return;
> +}
> +
> +static u32 tcm_vhost_sess_get_index(struct se_session *se_sess)
> +{
> + return 0;
> +}
> +
> +static int tcm_vhost_write_pending(struct se_cmd *se_cmd)
> +{
> + /* Go ahead and process the write immediately */
> + target_execute_cmd(se_cmd);
> + return 0;
> +}
> +
> +static int tcm_vhost_write_pending_status(struct se_cmd *se_cmd)
> +{
> + return 0;
> +}
> +
> +static void tcm_vhost_set_default_node_attrs(struct se_node_acl *nacl)
> +{
> + return;
> +}
> +
> +static u32 tcm_vhost_get_task_tag(struct se_cmd *se_cmd)
> +{
> + return 0;
> +}
> +
> +static int tcm_vhost_get_cmd_state(struct se_cmd *se_cmd)
> +{
> + return 0;
> +}
> +
> +static void vhost_scsi_complete_cmd(struct tcm_vhost_cmd *);
> +
> +static int tcm_vhost_queue_data_in(struct se_cmd *se_cmd)
> +{
> + struct tcm_vhost_cmd *tv_cmd = container_of(se_cmd,
> + struct tcm_vhost_cmd, tvc_se_cmd);
> + vhost_scsi_complete_cmd(tv_cmd);
> + return 0;
> +}
> +
> +static int tcm_vhost_queue_status(struct se_cmd *se_cmd)
> +{
> + struct tcm_vhost_cmd *tv_cmd = container_of(se_cmd,
> + struct tcm_vhost_cmd, tvc_se_cmd);
> + vhost_scsi_complete_cmd(tv_cmd);
> + return 0;
> +}
> +
> +static int tcm_vhost_queue_tm_rsp(struct se_cmd *se_cmd)
> +{
> + return 0;
> +}
> +
> +static u16 tcm_vhost_set_fabric_sense_len(struct se_cmd *se_cmd, u32 sense_length)
> +{
> + return 0;
> +}
> +
> +static u16 tcm_vhost_get_fabric_sense_len(void)
> +{
> + return 0;
> +}
> +
> +static void vhost_scsi_free_cmd(struct tcm_vhost_cmd *tv_cmd)
> +{
> + struct se_cmd *se_cmd = &tv_cmd->tvc_se_cmd;
> +
> + /* TODO locking against target/backend threads? */
> + transport_generic_free_cmd(se_cmd, 1);
> +
> + if (tv_cmd->tvc_sgl_count) {
> + u32 i;
> + for (i = 0; i < tv_cmd->tvc_sgl_count; i++)
> + put_page(sg_page(&tv_cmd->tvc_sgl[i]));
> +
> + kfree(tv_cmd->tvc_sgl);
> + }
> +
> + kfree(tv_cmd);
> +}
> +
> +/* Dequeue a command from the completion list */
> +static struct tcm_vhost_cmd *vhost_scsi_get_cmd_from_completion(struct vhost_scsi *vs)
> +{
> + struct tcm_vhost_cmd *tv_cmd = NULL;
> +
> + spin_lock_bh(&vs->vs_completion_lock);
> + if (list_empty(&vs->vs_completion_list)) {
> + spin_unlock_bh(&vs->vs_completion_lock);
> + return NULL;
> + }
> +
> + list_for_each_entry(tv_cmd, &vs->vs_completion_list,
> + tvc_completion_list) {
> + list_del(&tv_cmd->tvc_completion_list);
> + break;
> + }
> + spin_unlock_bh(&vs->vs_completion_lock);
> + return tv_cmd;
> +}
> +
> +/* Fill in status and signal that we are done processing this command
> + *
> + * This is scheduled in the vhost work queue so we are called with the owner
> + * process mm and can access the vring.
> + */
> +static void vhost_scsi_complete_cmd_work(struct vhost_work *work)
> +{
> + struct vhost_scsi *vs = container_of(work, struct vhost_scsi,
> + vs_completion_work);
> + struct tcm_vhost_cmd *tv_cmd;
> +
> + while ((tv_cmd = vhost_scsi_get_cmd_from_completion(vs)) != NULL) {
> + struct virtio_scsi_cmd_resp v_rsp;
> + struct se_cmd *se_cmd = &tv_cmd->tvc_se_cmd;
> + int ret;
> +
> + pr_debug("%s tv_cmd %p resid %u status %#02x\n", __func__,
> + tv_cmd, se_cmd->residual_count, se_cmd->scsi_status);
> +
> + memset(&v_rsp, 0, sizeof(v_rsp));
> + v_rsp.resid = se_cmd->residual_count;
> + /* TODO is status_qualifier field needed? */
> + v_rsp.status = se_cmd->scsi_status;
> + v_rsp.sense_len = se_cmd->scsi_sense_length;
> + memcpy(v_rsp.sense, tv_cmd->tvc_sense_buf,
> + v_rsp.sense_len);
> + ret = copy_to_user(tv_cmd->tvc_resp, &v_rsp, sizeof(v_rsp));
> + if (likely(ret == 0))
> + vhost_add_used(&vs->vqs[2], tv_cmd->tvc_vq_desc, 0);
> + else
> + pr_err("Faulted on virtio_scsi_cmd_resp\n");
> +
> + vhost_scsi_free_cmd(tv_cmd);
> + }
> +
> + vhost_signal(&vs->dev, &vs->vqs[2]);
> +}
> +
> +static void vhost_scsi_complete_cmd(struct tcm_vhost_cmd *tv_cmd)
> +{
> + struct vhost_scsi *vs = tv_cmd->tvc_vhost;
> +
> + pr_debug("%s tv_cmd %p\n", __func__, tv_cmd);
> +
> + spin_lock_bh(&vs->vs_completion_lock);
> + list_add_tail(&tv_cmd->tvc_completion_list, &vs->vs_completion_list);
> + spin_unlock_bh(&vs->vs_completion_lock);
> +
> + vhost_work_queue(&vs->dev, &vs->vs_completion_work);
> +}
> +
> +static struct tcm_vhost_cmd *vhost_scsi_allocate_cmd(
> + struct tcm_vhost_tpg *tv_tpg,
> + struct virtio_scsi_cmd_req *v_req,
> + u32 exp_data_len,
> + int data_direction)
> +{
> + struct tcm_vhost_cmd *tv_cmd;
> + struct tcm_vhost_nexus *tv_nexus;
> + struct se_portal_group *se_tpg = &tv_tpg->se_tpg;
> + struct se_session *se_sess;
> + struct se_cmd *se_cmd;
> + int sam_task_attr;
> +
> + tv_nexus = tv_tpg->tpg_nexus;
> + if (!tv_nexus) {
> + pr_err("Unable to locate active struct tcm_vhost_nexus\n");
> + return ERR_PTR(-EIO);
> + }
> + se_sess = tv_nexus->tvn_se_sess;
> +
> + tv_cmd = kzalloc(sizeof(struct tcm_vhost_cmd), GFP_ATOMIC);
> + if (!tv_cmd) {
> + pr_err("Unable to allocate struct tcm_vhost_cmd\n");
> + return ERR_PTR(-ENOMEM);
> + }
> + INIT_LIST_HEAD(&tv_cmd->tvc_completion_list);
> + tv_cmd->tvc_tag = v_req->tag;
> +
> + se_cmd = &tv_cmd->tvc_se_cmd;
> + /*
> + * Locate the SAM Task Attr from virtio_scsi_cmd_req
> + */
> + sam_task_attr = v_req->task_attr;
> + /*
> + * Initialize struct se_cmd descriptor from target_core_mod infrastructure
> + */
> + transport_init_se_cmd(se_cmd, se_tpg->se_tpg_tfo, se_sess, exp_data_len,
> + data_direction, sam_task_attr,
> + &tv_cmd->tvc_sense_buf[0]);
> +
> +#if 0 /* FIXME: vhost_scsi_allocate_cmd() BIDI operation */
> + if (bidi)
> + se_cmd->se_cmd_flags |= SCF_BIDI;
> +#endif
> + return tv_cmd;
> +}
> +
> +/*
> + * Map a user memory range into a scatterlist
> + *
> + * Returns the number of scatterlist entries used or -errno on error.
> + */
> +static int vhost_scsi_map_to_sgl(struct scatterlist *sgl,
> + unsigned int sgl_count,
> + void __user *ptr, size_t len, int write)
> +{
> + struct scatterlist *sg = sgl;
> + unsigned int npages = 0;
> + int ret;
> +
> + while (len > 0) {
> + struct page *page;
> + unsigned int offset = (uintptr_t)ptr & ~PAGE_MASK;
> + unsigned int nbytes = min_t(unsigned int,
> + PAGE_SIZE - offset, len);
> +
> + if (npages == sgl_count) {
> + ret = -ENOBUFS;
> + goto err;
> + }
> +
> + ret = get_user_pages_fast((unsigned long)ptr, 1, write, &page);
> + BUG_ON(ret == 0); /* we should either get our page or fail */
> + if (ret < 0)
> + goto err;
> +
> + sg_set_page(sg, page, nbytes, offset);
> + ptr += nbytes;
> + len -= nbytes;
> + sg++;
> + npages++;
> + }
> + return npages;
> +
> +err:
> + /* Put pages that we hold */
> + for (sg = sgl; sg != &sgl[npages]; sg++)
> + put_page(sg_page(sg));
> + return ret;
> +}
> +
> +static int vhost_scsi_map_iov_to_sgl(struct tcm_vhost_cmd *tv_cmd,
> + struct iovec *iov, unsigned int niov,
> + int write)
> +{
> + int ret;
> + unsigned int i;
> + u32 sgl_count;
> + struct scatterlist *sg;
> +
> + /*
> + * Find out how long sglist needs to be
> + */
> + sgl_count = 0;
> + for (i = 0; i < niov; i++) {
> + sgl_count += (((uintptr_t)iov[i].iov_base + iov[i].iov_len +
> + PAGE_SIZE - 1) >> PAGE_SHIFT) -
> + ((uintptr_t)iov[i].iov_base >> PAGE_SHIFT);
> + }
> + /* TODO overflow checking */
> +
> + sg = kmalloc(sizeof(tv_cmd->tvc_sgl[0]) * sgl_count, GFP_ATOMIC);
> + if (!sg)
> + return -ENOMEM;
> + pr_debug("%s sg %p sgl_count %u is_err %ld\n", __func__,
> + sg, sgl_count, IS_ERR(sg));
> + sg_init_table(sg, sgl_count);
> +
> + tv_cmd->tvc_sgl = sg;
> + tv_cmd->tvc_sgl_count = sgl_count;
> +
> + pr_debug("Mapping %u iovecs for %u pages\n", niov, sgl_count);
> + for (i = 0; i < niov; i++) {
> + ret = vhost_scsi_map_to_sgl(sg, sgl_count, iov[i].iov_base,
> + iov[i].iov_len, write);
> + if (ret < 0) {
> + for (i = 0; i < tv_cmd->tvc_sgl_count; i++)
> + put_page(sg_page(&tv_cmd->tvc_sgl[i]));
> + kfree(tv_cmd->tvc_sgl);
> + tv_cmd->tvc_sgl = NULL;
> + tv_cmd->tvc_sgl_count = 0;
> + return ret;
> + }
> +
> + sg += ret;
> + sgl_count -= ret;
> + }
> + return 0;
> +}
> +
> +static void tcm_vhost_submission_work(struct work_struct *work)
> +{
> + struct tcm_vhost_cmd *tv_cmd =
> + container_of(work, struct tcm_vhost_cmd, work);
> + struct se_cmd *se_cmd = &tv_cmd->tvc_se_cmd;
> + struct scatterlist *sg_ptr, *sg_bidi_ptr = NULL;
> + int rc, sg_no_bidi = 0;
> + /*
> + * Locate the struct se_lun pointer based on v_req->lun, and
> + * attach it to struct se_cmd
> + */
> + rc = transport_lookup_cmd_lun(&tv_cmd->tvc_se_cmd, tv_cmd->tvc_lun);
> + if (rc < 0) {
> + pr_err("Failed to look up lun: %d\n", tv_cmd->tvc_lun);
> + transport_send_check_condition_and_sense(&tv_cmd->tvc_se_cmd,
> + tv_cmd->tvc_se_cmd.scsi_sense_reason, 0);
> + transport_generic_free_cmd(se_cmd, 0);
> + return;
> + }
> +
> + rc = target_setup_cmd_from_cdb(se_cmd, tv_cmd->tvc_cdb);
> + if (rc == -ENOMEM) {
> + transport_send_check_condition_and_sense(se_cmd,
> + TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE, 0);
> + transport_generic_free_cmd(se_cmd, 0);
> + return;
> + } else if (rc < 0) {
> + if (se_cmd->se_cmd_flags & SCF_SCSI_RESERVATION_CONFLICT)
> + tcm_vhost_queue_status(se_cmd);
> + else
> + transport_send_check_condition_and_sense(se_cmd,
> + se_cmd->scsi_sense_reason, 0);
> + transport_generic_free_cmd(se_cmd, 0);
> + return;
> + }
> +
> + if (tv_cmd->tvc_sgl_count) {
> + sg_ptr = tv_cmd->tvc_sgl;
> + /*
> + * For BIDI commands, pass in the extra READ buffer
> + * to transport_generic_map_mem_to_cmd() below..
> + */
> +/* FIXME: Fix BIDI operation in tcm_vhost_submission_work() */
> +#if 0
> + if (se_cmd->se_cmd_flags & SCF_BIDI) {
> + sg_bidi_ptr = NULL;
> + sg_no_bidi = 0;
> + }
> +#endif
> + } else {
> + sg_ptr = NULL;
> + }
> +
> + rc = transport_generic_map_mem_to_cmd(se_cmd, sg_ptr,
> + tv_cmd->tvc_sgl_count, sg_bidi_ptr,
> + sg_no_bidi);
> + if (rc < 0) {
> + transport_send_check_condition_and_sense(se_cmd,
> + se_cmd->scsi_sense_reason, 0);
> + transport_generic_free_cmd(se_cmd, 0);
> + return;
> + }
> + transport_handle_cdb_direct(se_cmd);
> +}
> +
> +static void vhost_scsi_handle_vq(struct vhost_scsi *vs)
> +{
> + struct vhost_virtqueue *vq = &vs->vqs[2];
> + struct virtio_scsi_cmd_req v_req;
> + struct tcm_vhost_tpg *tv_tpg;
> + struct tcm_vhost_cmd *tv_cmd;
> + u32 exp_data_len, data_first, data_num, data_direction;
> + unsigned out, in, i;
> + int head, ret;
> +
> + /* Must use ioctl VHOST_SCSI_SET_ENDPOINT */
> + tv_tpg = vs->vs_tpg;
> + if (unlikely(!tv_tpg)) {
> + pr_err("%s endpoint not set\n", __func__);
> + return;
> + }
> +
> + mutex_lock(&vq->mutex);
> + vhost_disable_notify(&vs->dev, vq);
> +
> + for (;;) {
> + head = vhost_get_vq_desc(&vs->dev, vq, vq->iov,
> + ARRAY_SIZE(vq->iov), &out, &in,
> + NULL, NULL);
> + pr_debug("vhost_get_vq_desc: head: %d, out: %u in: %u\n", head, out, in);
> + /* On error, stop handling until the next kick. */
> + if (unlikely(head < 0))
> + break;
> + /* Nothing new? Wait for eventfd to tell us they refilled. */
> + if (head == vq->num) {
> + if (unlikely(vhost_enable_notify(&vs->dev, vq))) {
> + vhost_disable_notify(&vs->dev, vq);
> + continue;
> + }
> + break;
> + }
> +
> +/* FIXME: BIDI operation */
> + if (out == 1 && in == 1) {
> + data_direction = DMA_NONE;
> + data_first = 0;
> + data_num = 0;
> + } else if (out == 1 && in > 1) {
> + data_direction = DMA_FROM_DEVICE;
> + data_first = out + 1;
> + data_num = in - 1;
> + } else if (out > 1 && in == 1) {
> + data_direction = DMA_TO_DEVICE;
> + data_first = 1;
> + data_num = out - 1;
> + } else {
> + pr_err("Invalid buffer layout out: %u in: %u\n", out, in);
> + break;
> + }
> +
> + /*
> + * Check for a sane resp buffer so we can report errors to
> + * the guest.
> + */
> + if (unlikely(vq->iov[out].iov_len !=
> + sizeof(struct virtio_scsi_cmd_resp))) {
> + pr_err("Expecting virtio_scsi_cmd_resp, got %zu bytes\n",
> + vq->iov[out].iov_len);
> + break;
> + }
> +
> + if (unlikely(vq->iov[0].iov_len != sizeof(v_req))) {
> + pr_err("Expecting virtio_scsi_cmd_req, got %zu bytes\n",
> + vq->iov[0].iov_len);
> + break;
> + }
> + pr_debug("Calling __copy_from_user: vq->iov[0].iov_base: %p, len: %zu\n",
> + vq->iov[0].iov_base, sizeof(v_req));
> + ret = __copy_from_user(&v_req, vq->iov[0].iov_base, sizeof(v_req));
> + if (unlikely(ret)) {
> + pr_err("Faulted on virtio_scsi_cmd_req\n");
> + break;
> + }
> +
> + exp_data_len = 0;
> + for (i = 0; i < data_num; i++) {
> + exp_data_len += vq->iov[data_first + i].iov_len;
> + }
> +
> + tv_cmd = vhost_scsi_allocate_cmd(tv_tpg, &v_req,
> + exp_data_len, data_direction);
> + if (IS_ERR(tv_cmd)) {
> + pr_err("vhost_scsi_allocate_cmd failed %ld\n", PTR_ERR(tv_cmd));
> + break;
> + }
> + pr_debug("Allocated tv_cmd: %p exp_data_len: %d, data_direction: %d\n",
> + tv_cmd, exp_data_len, data_direction);
> +
> + tv_cmd->tvc_vhost = vs;
> +
> + if (unlikely(vq->iov[out].iov_len !=
> + sizeof(struct virtio_scsi_cmd_resp))) {
> + pr_err("Expecting virtio_scsi_cmd_resp, "
> + " got %zu bytes, out: %d, in: %d\n", vq->iov[out].iov_len, out, in);
> + break;
> + }
> +
> + tv_cmd->tvc_resp = vq->iov[out].iov_base;
> +
> + /*
> + * Copy in the recieved CDB descriptor into tv_cmd->tvc_cdb
> + * that will be used by tcm_vhost_new_cmd_map() and down into
> + * target_setup_cmd_from_cdb()
> + */
> + memcpy(tv_cmd->tvc_cdb, v_req.cdb, TCM_VHOST_MAX_CDB_SIZE);
> + /*
> + * Check that the recieved CDB size does not exceeded our
> + * hardcoded max for tcm_vhost
> + */
> + /* TODO what if cdb was too small for varlen cdb header? */
> + if (unlikely(scsi_command_size(tv_cmd->tvc_cdb) > TCM_VHOST_MAX_CDB_SIZE)) {
> + pr_err("Received SCSI CDB with command_size: %d that exceeds"
> + " SCSI_MAX_VARLEN_CDB_SIZE: %d\n",
> + scsi_command_size(tv_cmd->tvc_cdb), TCM_VHOST_MAX_CDB_SIZE);
> + break; /* TODO */
> + }
> + tv_cmd->tvc_lun = ((v_req.lun[2] << 8) | v_req.lun[3]) & 0x3FFF;
> +
> + pr_debug("vhost_scsi got command opcode: %#02x, lun: %d\n",
> + tv_cmd->tvc_cdb[0], tv_cmd->tvc_lun);
> +
> + if (data_direction != DMA_NONE) {
> + ret = vhost_scsi_map_iov_to_sgl(tv_cmd, &vq->iov[data_first],
> + data_num, data_direction == DMA_TO_DEVICE);
> + if (unlikely(ret)) {
> + pr_err("Failed to map iov to sgl\n");
> + break; /* TODO */
> + }
> + }
> +
> + /*
> + * Save the descriptor from vhost_get_vq_desc() to be used to
> + * complete the virtio-scsi request in TCM callback context via
> + * tcm_vhost_queue_data_in() and tcm_vhost_queue_status()
> + */
> + tv_cmd->tvc_vq_desc = head;
> + /*
> + * Dispatch tv_cmd descriptor for cmwq execution in process
> + * context provided by tcm_vhost_workqueue. This also ensures
> + * tv_cmd is executed on the same kworker CPU as this vhost
> + * thread to gain positive L2 cache locality effects..
> + */
> + INIT_WORK(&tv_cmd->work, tcm_vhost_submission_work);
> + queue_work(tcm_vhost_workqueue, &tv_cmd->work);
> + }
> +
> + mutex_unlock(&vq->mutex);
> +}
> +
> +static void vhost_scsi_ctl_handle_kick(struct vhost_work *work)
> +{
> + pr_err("%s: The handling func for control queue.\n", __func__);
> +}
> +
> +static void vhost_scsi_evt_handle_kick(struct vhost_work *work)
> +{
> + pr_err("%s: The handling func for event queue.\n", __func__);
> +}
> +
> +static void vhost_scsi_handle_kick(struct vhost_work *work)
> +{
> + struct vhost_virtqueue *vq = container_of(work, struct vhost_virtqueue,
> + poll.work);
> + struct vhost_scsi *vs = container_of(vq->dev, struct vhost_scsi, dev);
> +
> + vhost_scsi_handle_vq(vs);
> +}
> +
> +/*
> + * Called from vhost_scsi_ioctl() context to walk the list of available tcm_vhost_tpg
> + * with an active struct tcm_vhost_nexus
> + */
> +static int vhost_scsi_set_endpoint(
> + struct vhost_scsi *vs,
> + struct vhost_scsi_target *t)
> +{
> + struct tcm_vhost_tport *tv_tport;
> + struct tcm_vhost_tpg *tv_tpg;
> + int index;
> +
> + mutex_lock(&vs->dev.mutex);
> + /* Verify that ring has been setup correctly. */
> + for (index = 0; index < vs->dev.nvqs; ++index) {
> + /* Verify that ring has been setup correctly. */
> + if (!vhost_vq_access_ok(&vs->vqs[index])) {
> + mutex_unlock(&vs->dev.mutex);
> + return -EFAULT;
> + }
> + }
> +
> + if (vs->vs_tpg) {
> + mutex_unlock(&vs->dev.mutex);
> + return -EEXIST;
> + }
> + mutex_unlock(&vs->dev.mutex);
> +
> + mutex_lock(&tcm_vhost_mutex);
> + list_for_each_entry(tv_tpg, &tcm_vhost_list, tv_tpg_list) {
> + mutex_lock(&tv_tpg->tv_tpg_mutex);
> + if (!tv_tpg->tpg_nexus) {
> + mutex_unlock(&tv_tpg->tv_tpg_mutex);
> + continue;
> + }
> + if (atomic_read(&tv_tpg->tv_tpg_vhost_count)) {
> + mutex_unlock(&tv_tpg->tv_tpg_mutex);
> + continue;
> + }
> + tv_tport = tv_tpg->tport;
> +
> + if (!strcmp(tv_tport->tport_name, t->vhost_wwpn) &&
> + (tv_tpg->tport_tpgt == t->vhost_tpgt)) {
> + atomic_inc(&tv_tpg->tv_tpg_vhost_count);
> + smp_mb__after_atomic_inc();
> + mutex_unlock(&tv_tpg->tv_tpg_mutex);
> + mutex_unlock(&tcm_vhost_mutex);
> +
> + mutex_lock(&vs->dev.mutex);
> + vs->vs_tpg = tv_tpg;
> + atomic_inc(&vs->vhost_ref_cnt);
> + smp_mb__after_atomic_inc();
> + mutex_unlock(&vs->dev.mutex);
> + return 0;
> + }
> + mutex_unlock(&tv_tpg->tv_tpg_mutex);
> + }
> + mutex_unlock(&tcm_vhost_mutex);
> + return -EINVAL;
> +}
> +
> +static int vhost_scsi_clear_endpoint(
> + struct vhost_scsi *vs,
> + struct vhost_scsi_target *t)
> +{
> + struct tcm_vhost_tport *tv_tport;
> + struct tcm_vhost_tpg *tv_tpg;
> + int index;
> +
> + mutex_lock(&vs->dev.mutex);
> + /* Verify that ring has been setup correctly. */
> + for (index = 0; index < vs->dev.nvqs; ++index) {
> + if (!vhost_vq_access_ok(&vs->vqs[index])) {
> + mutex_unlock(&vs->dev.mutex);
> + return -EFAULT;
> + }
> + }
> +
> + if (!vs->vs_tpg) {
> + mutex_unlock(&vs->dev.mutex);
> + return -ENODEV;
> + }
> + tv_tpg = vs->vs_tpg;
> + tv_tport = tv_tpg->tport;
> +
> + if (strcmp(tv_tport->tport_name, t->vhost_wwpn) ||
> + (tv_tpg->tport_tpgt != t->vhost_tpgt)) {
> + mutex_unlock(&vs->dev.mutex);
> + pr_warn("tv_tport->tport_name: %s, tv_tpg->tport_tpgt: %hu"
> + " does not match t->vhost_wwpn: %s, t->vhost_tpgt: %hu\n",
> + tv_tport->tport_name, tv_tpg->tport_tpgt,
> + t->vhost_wwpn, t->vhost_tpgt);
> + return -EINVAL;
> + }
> + atomic_dec(&tv_tpg->tv_tpg_vhost_count);
> + vs->vs_tpg = NULL;
> + mutex_unlock(&vs->dev.mutex);
> +
> + return 0;
> +}
> +
> +static int vhost_scsi_open(struct inode *inode, struct file *f)
> +{
> + struct vhost_scsi *s;
> + int r;
> +
> + s = kzalloc(sizeof(*s), GFP_KERNEL);
> + if (!s)
> + return -ENOMEM;
> +
> + vhost_work_init(&s->vs_completion_work, vhost_scsi_complete_cmd_work);
> + INIT_LIST_HEAD(&s->vs_completion_list);
> + spin_lock_init(&s->vs_completion_lock);
> +
> + s->vqs[0].handle_kick = vhost_scsi_ctl_handle_kick;
> + s->vqs[1].handle_kick = vhost_scsi_evt_handle_kick;
> + s->vqs[2].handle_kick = vhost_scsi_handle_kick;
> + r = vhost_dev_init(&s->dev, s->vqs, 3);
> + if (r < 0) {
> + kfree(s);
> + return r;
> + }
> +
> + f->private_data = s;
> + return 0;
> +}
> +
> +static int vhost_scsi_release(struct inode *inode, struct file *f)
> +{
> + struct vhost_scsi *s = f->private_data;
> +
> + if (s->vs_tpg && s->vs_tpg->tport) {
> + struct vhost_scsi_target backend;
> + memcpy(backend.vhost_wwpn, s->vs_tpg->tport->tport_name, sizeof(backend.vhost_wwpn));
> + backend.vhost_tpgt = s->vs_tpg->tport_tpgt;
> + vhost_scsi_clear_endpoint(s, &backend);
> + }
> +
> + vhost_dev_cleanup(&s->dev, false);
> + kfree(s);
> + return 0;
> +}
> +
> +static int vhost_scsi_set_features(struct vhost_scsi *vs, u64 features)
> +{
> + if (features & ~VHOST_FEATURES)
> + return -EOPNOTSUPP;
> +
> + mutex_lock(&vs->dev.mutex);
> + if ((features & (1 << VHOST_F_LOG_ALL)) &&
> + !vhost_log_access_ok(&vs->dev)) {
> + mutex_unlock(&vs->dev.mutex);
> + return -EFAULT;
> + }
> + vs->dev.acked_features = features;
> + /* TODO possibly smp_wmb() and flush vqs */
> + mutex_unlock(&vs->dev.mutex);
> + return 0;
> +}
> +
> +static long vhost_scsi_ioctl(struct file *f, unsigned int ioctl,
> + unsigned long arg)
> +{
> + struct vhost_scsi *vs = f->private_data;
> + struct vhost_scsi_target backend;
> + void __user *argp = (void __user *)arg;
> + u64 __user *featurep = argp;
> + u64 features;
> + int r;
> +
> + switch (ioctl) {
> + case VHOST_SCSI_SET_ENDPOINT:
> + if (copy_from_user(&backend, argp, sizeof backend))
> + return -EFAULT;
> +
> + return vhost_scsi_set_endpoint(vs, &backend);
> + case VHOST_SCSI_CLEAR_ENDPOINT:
> + if (copy_from_user(&backend, argp, sizeof backend))
> + return -EFAULT;
> +
> + return vhost_scsi_clear_endpoint(vs, &backend);
> + case VHOST_GET_FEATURES:
> + features = VHOST_FEATURES;
> + if (copy_to_user(featurep, &features, sizeof features))
> + return -EFAULT;
> + return 0;
> + case VHOST_SET_FEATURES:
> + if (copy_from_user(&features, featurep, sizeof features))
> + return -EFAULT;
> + return vhost_scsi_set_features(vs, features);
> + default:
> + mutex_lock(&vs->dev.mutex);
> + r = vhost_dev_ioctl(&vs->dev, ioctl, arg);
> + mutex_unlock(&vs->dev.mutex);
> + return r;
> + }
> +}
> +
> +static const struct file_operations vhost_scsi_fops = {
> + .owner = THIS_MODULE,
> + .release = vhost_scsi_release,
> + .unlocked_ioctl = vhost_scsi_ioctl,
> + /* TODO compat ioctl? */
> + .open = vhost_scsi_open,
> + .llseek = noop_llseek,
> +};
> +
> +static struct miscdevice vhost_scsi_misc = {
> + MISC_DYNAMIC_MINOR,
> + "vhost-scsi",
> + &vhost_scsi_fops,
> +};
> +
> +static int __init vhost_scsi_register(void)
> +{
> + return misc_register(&vhost_scsi_misc);
> +}
> +
> +static int vhost_scsi_deregister(void)
> +{
> + return misc_deregister(&vhost_scsi_misc);
> +}
> +
> +static char *tcm_vhost_dump_proto_id(struct tcm_vhost_tport *tport)
> +{
> + switch (tport->tport_proto_id) {
> + case SCSI_PROTOCOL_SAS:
> + return "SAS";
> + case SCSI_PROTOCOL_FCP:
> + return "FCP";
> + case SCSI_PROTOCOL_ISCSI:
> + return "iSCSI";
> + default:
> + break;
> + }
> +
> + return "Unknown";
> +}
> +
> +static int tcm_vhost_port_link(
> + struct se_portal_group *se_tpg,
> + struct se_lun *lun)
> +{
> + struct tcm_vhost_tpg *tv_tpg = container_of(se_tpg,
> + struct tcm_vhost_tpg, se_tpg);
> +
> + atomic_inc(&tv_tpg->tv_tpg_port_count);
> + smp_mb__after_atomic_inc();
> +
> + return 0;
> +}
> +
> +static void tcm_vhost_port_unlink(
> + struct se_portal_group *se_tpg,
> + struct se_lun *se_lun)
> +{
> + struct tcm_vhost_tpg *tv_tpg = container_of(se_tpg,
> + struct tcm_vhost_tpg, se_tpg);
> +
> + atomic_dec(&tv_tpg->tv_tpg_port_count);
> + smp_mb__after_atomic_dec();
> +}
> +
> +static struct se_node_acl *tcm_vhost_make_nodeacl(
> + struct se_portal_group *se_tpg,
> + struct config_group *group,
> + const char *name)
> +{
> + struct se_node_acl *se_nacl, *se_nacl_new;
> + struct tcm_vhost_nacl *nacl;
> + u64 wwpn = 0;
> + u32 nexus_depth;
> +
> + /* tcm_vhost_parse_wwn(name, &wwpn, 1) < 0)
> + return ERR_PTR(-EINVAL); */
> + se_nacl_new = tcm_vhost_alloc_fabric_acl(se_tpg);
> + if (!se_nacl_new)
> + return ERR_PTR(-ENOMEM);
> +//#warning FIXME: Hardcoded nexus depth in tcm_vhost_make_nodeacl()
> + nexus_depth = 1;
> + /*
> + * se_nacl_new may be released by core_tpg_add_initiator_node_acl()
> + * when converting a NodeACL from demo mode -> explict
> + */
> + se_nacl = core_tpg_add_initiator_node_acl(se_tpg, se_nacl_new,
> + name, nexus_depth);
> + if (IS_ERR(se_nacl)) {
> + tcm_vhost_release_fabric_acl(se_tpg, se_nacl_new);
> + return se_nacl;
> + }
> + /*
> + * Locate our struct tcm_vhost_nacl and set the FC Nport WWPN
> + */
> + nacl = container_of(se_nacl, struct tcm_vhost_nacl, se_node_acl);
> + nacl->iport_wwpn = wwpn;
> + /* tcm_vhost_format_wwn(&nacl->iport_name[0], TCM_VHOST_NAMELEN, wwpn); */
> +
> + return se_nacl;
> +}
> +
> +static void tcm_vhost_drop_nodeacl(struct se_node_acl *se_acl)
> +{
> + struct tcm_vhost_nacl *nacl = container_of(se_acl,
> + struct tcm_vhost_nacl, se_node_acl);
> + core_tpg_del_initiator_node_acl(se_acl->se_tpg, se_acl, 1);
> + kfree(nacl);
> +}
> +
> +static int tcm_vhost_make_nexus(
> + struct tcm_vhost_tpg *tv_tpg,
> + const char *name)
> +{
> + struct se_portal_group *se_tpg;
> + struct tcm_vhost_nexus *tv_nexus;
> +
> + mutex_lock(&tv_tpg->tv_tpg_mutex);
> + if (tv_tpg->tpg_nexus) {
> + mutex_unlock(&tv_tpg->tv_tpg_mutex);
> + pr_debug("tv_tpg->tpg_nexus already exists\n");
> + return -EEXIST;
> + }
> + se_tpg = &tv_tpg->se_tpg;
> +
> + tv_nexus = kzalloc(sizeof(struct tcm_vhost_nexus), GFP_KERNEL);
> + if (!tv_nexus) {
> + mutex_unlock(&tv_tpg->tv_tpg_mutex);
> + pr_err("Unable to allocate struct tcm_vhost_nexus\n");
> + return -ENOMEM;
> + }
> + /*
> + * Initialize the struct se_session pointer
> + */
> + tv_nexus->tvn_se_sess = transport_init_session();
> + if (IS_ERR(tv_nexus->tvn_se_sess)) {
> + mutex_unlock(&tv_tpg->tv_tpg_mutex);
> + kfree(tv_nexus);
> + return -ENOMEM;
> + }
> + /*
> + * Since we are running in 'demo mode' this call with generate a
> + * struct se_node_acl for the tcm_vhost struct se_portal_group with
> + * the SCSI Initiator port name of the passed configfs group 'name'.
> + */
> + tv_nexus->tvn_se_sess->se_node_acl = core_tpg_check_initiator_node_acl(
> + se_tpg, (unsigned char *)name);
> + if (!tv_nexus->tvn_se_sess->se_node_acl) {
> + mutex_unlock(&tv_tpg->tv_tpg_mutex);
> + pr_debug("core_tpg_check_initiator_node_acl() failed"
> + " for %s\n", name);
> + transport_free_session(tv_nexus->tvn_se_sess);
> + kfree(tv_nexus);
> + return -ENOMEM;
> + }
> + /*
> + * Now register the TCM vHost virtual I_T Nexus as active with the
> + * call to __transport_register_session()
> + */
> + __transport_register_session(se_tpg, tv_nexus->tvn_se_sess->se_node_acl,
> + tv_nexus->tvn_se_sess, tv_nexus);
> + tv_tpg->tpg_nexus = tv_nexus;
> +
> + mutex_unlock(&tv_tpg->tv_tpg_mutex);
> + return 0;
> +}
> +
> +static int tcm_vhost_drop_nexus(
> + struct tcm_vhost_tpg *tpg)
> +{
> + struct se_session *se_sess;
> + struct tcm_vhost_nexus *tv_nexus;
> +
> + mutex_lock(&tpg->tv_tpg_mutex);
> + tv_nexus = tpg->tpg_nexus;
> + if (!tv_nexus) {
> + mutex_unlock(&tpg->tv_tpg_mutex);
> + return -ENODEV;
> + }
> +
> + se_sess = tv_nexus->tvn_se_sess;
> + if (!se_sess) {
> + mutex_unlock(&tpg->tv_tpg_mutex);
> + return -ENODEV;
> + }
> +
> + if (atomic_read(&tpg->tv_tpg_port_count)) {
> + mutex_unlock(&tpg->tv_tpg_mutex);
> + pr_err("Unable to remove TCM_vHost I_T Nexus with"
> + " active TPG port count: %d\n",
> + atomic_read(&tpg->tv_tpg_port_count));
> + return -EPERM;
> + }
> +
> + if (atomic_read(&tpg->tv_tpg_vhost_count)) {
> + mutex_unlock(&tpg->tv_tpg_mutex);
> + pr_err("Unable to remove TCM_vHost I_T Nexus with"
> + " active TPG vhost count: %d\n",
> + atomic_read(&tpg->tv_tpg_vhost_count));
> + return -EPERM;
> + }
> +
> + pr_debug("TCM_vHost_ConfigFS: Removing I_T Nexus to emulated"
> + " %s Initiator Port: %s\n", tcm_vhost_dump_proto_id(tpg->tport),
> + tv_nexus->tvn_se_sess->se_node_acl->initiatorname);
> + /*
> + * Release the SCSI I_T Nexus to the emulated vHost Target Port
> + */
> + transport_deregister_session(tv_nexus->tvn_se_sess);
> + tpg->tpg_nexus = NULL;
> + mutex_unlock(&tpg->tv_tpg_mutex);
> +
> + kfree(tv_nexus);
> + return 0;
> +}
> +
> +static ssize_t tcm_vhost_tpg_show_nexus(
> + struct se_portal_group *se_tpg,
> + char *page)
> +{
> + struct tcm_vhost_tpg *tv_tpg = container_of(se_tpg,
> + struct tcm_vhost_tpg, se_tpg);
> + struct tcm_vhost_nexus *tv_nexus;
> + ssize_t ret;
> +
> + mutex_lock(&tv_tpg->tv_tpg_mutex);
> + tv_nexus = tv_tpg->tpg_nexus;
> + if (!tv_nexus) {
> + mutex_unlock(&tv_tpg->tv_tpg_mutex);
> + return -ENODEV;
> + }
> + ret = snprintf(page, PAGE_SIZE, "%s\n",
> + tv_nexus->tvn_se_sess->se_node_acl->initiatorname);
> + mutex_unlock(&tv_tpg->tv_tpg_mutex);
> +
> + return ret;
> +}
> +
> +static ssize_t tcm_vhost_tpg_store_nexus(
> + struct se_portal_group *se_tpg,
> + const char *page,
> + size_t count)
> +{
> + struct tcm_vhost_tpg *tv_tpg = container_of(se_tpg,
> + struct tcm_vhost_tpg, se_tpg);
> + struct tcm_vhost_tport *tport_wwn = tv_tpg->tport;
> + unsigned char i_port[TCM_VHOST_NAMELEN], *ptr, *port_ptr;
> + int ret;
> + /*
> + * Shutdown the active I_T nexus if 'NULL' is passed..
> + */
> + if (!strncmp(page, "NULL", 4)) {
> + ret = tcm_vhost_drop_nexus(tv_tpg);
> + return (!ret) ? count : ret;
> + }
> + /*
> + * Otherwise make sure the passed virtual Initiator port WWN matches
> + * the fabric protocol_id set in tcm_vhost_make_tport(), and call
> + * tcm_vhost_make_nexus().
> + */
> + if (strlen(page) >= TCM_VHOST_NAMELEN) {
> + pr_err("Emulated NAA Sas Address: %s, exceeds"
> + " max: %d\n", page, TCM_VHOST_NAMELEN);
> + return -EINVAL;
> + }
> + snprintf(&i_port[0], TCM_VHOST_NAMELEN, "%s", page);
> +
> + ptr = strstr(i_port, "naa.");
> + if (ptr) {
> + if (tport_wwn->tport_proto_id != SCSI_PROTOCOL_SAS) {
> + pr_err("Passed SAS Initiator Port %s does not"
> + " match target port protoid: %s\n", i_port,
> + tcm_vhost_dump_proto_id(tport_wwn));
> + return -EINVAL;
> + }
> + port_ptr = &i_port[0];
> + goto check_newline;
> + }
> + ptr = strstr(i_port, "fc.");
> + if (ptr) {
> + if (tport_wwn->tport_proto_id != SCSI_PROTOCOL_FCP) {
> + pr_err("Passed FCP Initiator Port %s does not"
> + " match target port protoid: %s\n", i_port,
> + tcm_vhost_dump_proto_id(tport_wwn));
> + return -EINVAL;
> + }
> + port_ptr = &i_port[3]; /* Skip over "fc." */
> + goto check_newline;
> + }
> + ptr = strstr(i_port, "iqn.");
> + if (ptr) {
> + if (tport_wwn->tport_proto_id != SCSI_PROTOCOL_ISCSI) {
> + pr_err("Passed iSCSI Initiator Port %s does not"
> + " match target port protoid: %s\n", i_port,
> + tcm_vhost_dump_proto_id(tport_wwn));
> + return -EINVAL;
> + }
> + port_ptr = &i_port[0];
> + goto check_newline;
> + }
> + pr_err("Unable to locate prefix for emulated Initiator Port:"
> + " %s\n", i_port);
> + return -EINVAL;
> + /*
> + * Clear any trailing newline for the NAA WWN
> + */
> +check_newline:
> + if (i_port[strlen(i_port)-1] == '\n')
> + i_port[strlen(i_port)-1] = '\0';
> +
> + ret = tcm_vhost_make_nexus(tv_tpg, port_ptr);
> + if (ret < 0)
> + return ret;
> +
> + return count;
> +}
> +
> +TF_TPG_BASE_ATTR(tcm_vhost, nexus, S_IRUGO | S_IWUSR);
> +
> +static struct configfs_attribute *tcm_vhost_tpg_attrs[] = {
> + &tcm_vhost_tpg_nexus.attr,
> + NULL,
> +};
> +
> +static struct se_portal_group *tcm_vhost_make_tpg(
> + struct se_wwn *wwn,
> + struct config_group *group,
> + const char *name)
> +{
> + struct tcm_vhost_tport*tport = container_of(wwn,
> + struct tcm_vhost_tport, tport_wwn);
> +
> + struct tcm_vhost_tpg *tpg;
> + unsigned long tpgt;
> + int ret;
> +
> + if (strstr(name, "tpgt_") != name)
> + return ERR_PTR(-EINVAL);
> + if (strict_strtoul(name + 5, 10, &tpgt) || tpgt > UINT_MAX)
> + return ERR_PTR(-EINVAL);
> +
> + tpg = kzalloc(sizeof(struct tcm_vhost_tpg), GFP_KERNEL);
> + if (!tpg) {
> + pr_err("Unable to allocate struct tcm_vhost_tpg");
> + return ERR_PTR(-ENOMEM);
> + }
> + mutex_init(&tpg->tv_tpg_mutex);
> + INIT_LIST_HEAD(&tpg->tv_tpg_list);
> + tpg->tport = tport;
> + tpg->tport_tpgt = tpgt;
> +
> + ret = core_tpg_register(&tcm_vhost_fabric_configfs->tf_ops, wwn,
> + &tpg->se_tpg, tpg, TRANSPORT_TPG_TYPE_NORMAL);
> + if (ret < 0) {
> + kfree(tpg);
> + return NULL;
> + }
> + mutex_lock(&tcm_vhost_mutex);
> + list_add_tail(&tpg->tv_tpg_list, &tcm_vhost_list);
> + mutex_unlock(&tcm_vhost_mutex);
> +
> + return &tpg->se_tpg;
> +}
> +
> +static void tcm_vhost_drop_tpg(struct se_portal_group *se_tpg)
> +{
> + struct tcm_vhost_tpg *tpg = container_of(se_tpg,
> + struct tcm_vhost_tpg, se_tpg);
> +
> + mutex_lock(&tcm_vhost_mutex);
> + list_del(&tpg->tv_tpg_list);
> + mutex_unlock(&tcm_vhost_mutex);
> + /*
> + * Release the virtual I_T Nexus for this vHost TPG
> + */
> + tcm_vhost_drop_nexus(tpg);
> + /*
> + * Deregister the se_tpg from TCM..
> + */
> + core_tpg_deregister(se_tpg);
> + kfree(tpg);
> +}
> +
> +static struct se_wwn *tcm_vhost_make_tport(
> + struct target_fabric_configfs *tf,
> + struct config_group *group,
> + const char *name)
> +{
> + struct tcm_vhost_tport *tport;
> + char *ptr;
> + u64 wwpn = 0;
> + int off = 0;
> +
> + /* if (tcm_vhost_parse_wwn(name, &wwpn, 1) < 0)
> + return ERR_PTR(-EINVAL); */
> +
> + tport = kzalloc(sizeof(struct tcm_vhost_tport), GFP_KERNEL);
> + if (!tport) {
> + pr_err("Unable to allocate struct tcm_vhost_tport");
> + return ERR_PTR(-ENOMEM);
> + }
> + tport->tport_wwpn = wwpn;
> + /* tcm_vhost_format_wwn(&tport->tport_name[0], TCM_VHOST__NAMELEN, wwpn); */
> + /*
> + * Determine the emulated Protocol Identifier and Target Port Name
> + * based on the incoming configfs directory name.
> + */
> + ptr = strstr(name, "naa.");
> + if (ptr) {
> + tport->tport_proto_id = SCSI_PROTOCOL_SAS;
> + goto check_len;
> + }
> + ptr = strstr(name, "fc.");
> + if (ptr) {
> + tport->tport_proto_id = SCSI_PROTOCOL_FCP;
> + off = 3; /* Skip over "fc." */
> + goto check_len;
> + }
> + ptr = strstr(name, "iqn.");
> + if (ptr) {
> + tport->tport_proto_id = SCSI_PROTOCOL_ISCSI;
> + goto check_len;
> + }
> +
> + pr_err("Unable to locate prefix for emulated Target Port:"
> + " %s\n", name);
> + kfree(tport);
> + return ERR_PTR(-EINVAL);
> +
> +check_len:
> + if (strlen(name) >= TCM_VHOST_NAMELEN) {
> + pr_err("Emulated %s Address: %s, exceeds"
> + " max: %d\n", name, tcm_vhost_dump_proto_id(tport),
> + TCM_VHOST_NAMELEN);
> + kfree(tport);
> + return ERR_PTR(-EINVAL);
> + }
> + snprintf(&tport->tport_name[0], TCM_VHOST_NAMELEN, "%s", &name[off]);
> +
> + pr_debug("TCM_VHost_ConfigFS: Allocated emulated Target"
> + " %s Address: %s\n", tcm_vhost_dump_proto_id(tport), name);
> +
> + return &tport->tport_wwn;
> +}
> +
> +static void tcm_vhost_drop_tport(struct se_wwn *wwn)
> +{
> + struct tcm_vhost_tport *tport = container_of(wwn,
> + struct tcm_vhost_tport, tport_wwn);
> +
> + pr_debug("TCM_VHost_ConfigFS: Deallocating emulated Target"
> + " %s Address: %s\n", tcm_vhost_dump_proto_id(tport),
> + tport->tport_name);;
> +
> + kfree(tport);
> +}
> +
> +static ssize_t tcm_vhost_wwn_show_attr_version(
> + struct target_fabric_configfs *tf,
> + char *page)
> +{
> + return sprintf(page, "TCM_VHOST fabric module %s on %s/%s"
> + "on "UTS_RELEASE"\n", TCM_VHOST_VERSION, utsname()->sysname,
> + utsname()->machine);
> +}
> +
> +TF_WWN_ATTR_RO(tcm_vhost, version);
> +
> +static struct configfs_attribute *tcm_vhost_wwn_attrs[] = {
> + &tcm_vhost_wwn_version.attr,
> + NULL,
> +};
> +
> +static struct target_core_fabric_ops tcm_vhost_ops = {
> + .get_fabric_name = tcm_vhost_get_fabric_name,
> + .get_fabric_proto_ident = tcm_vhost_get_fabric_proto_ident,
> + .tpg_get_wwn = tcm_vhost_get_fabric_wwn,
> + .tpg_get_tag = tcm_vhost_get_tag,
> + .tpg_get_default_depth = tcm_vhost_get_default_depth,
> + .tpg_get_pr_transport_id = tcm_vhost_get_pr_transport_id,
> + .tpg_get_pr_transport_id_len = tcm_vhost_get_pr_transport_id_len,
> + .tpg_parse_pr_out_transport_id = tcm_vhost_parse_pr_out_transport_id,
> + .tpg_check_demo_mode = tcm_vhost_check_true,
> + .tpg_check_demo_mode_cache = tcm_vhost_check_true,
> + .tpg_check_demo_mode_write_protect = tcm_vhost_check_false,
> + .tpg_check_prod_mode_write_protect = tcm_vhost_check_false,
> + .tpg_alloc_fabric_acl = tcm_vhost_alloc_fabric_acl,
> + .tpg_release_fabric_acl = tcm_vhost_release_fabric_acl,
> + .tpg_get_inst_index = tcm_vhost_tpg_get_inst_index,
> + .release_cmd = tcm_vhost_release_cmd,
> + .shutdown_session = tcm_vhost_shutdown_session,
> + .close_session = tcm_vhost_close_session,
> + .sess_get_index = tcm_vhost_sess_get_index,
> + .sess_get_initiator_sid = NULL,
> + .write_pending = tcm_vhost_write_pending,
> + .write_pending_status = tcm_vhost_write_pending_status,
> + .set_default_node_attributes = tcm_vhost_set_default_node_attrs,
> + .get_task_tag = tcm_vhost_get_task_tag,
> + .get_cmd_state = tcm_vhost_get_cmd_state,
> + .queue_data_in = tcm_vhost_queue_data_in,
> + .queue_status = tcm_vhost_queue_status,
> + .queue_tm_rsp = tcm_vhost_queue_tm_rsp,
> + .get_fabric_sense_len = tcm_vhost_get_fabric_sense_len,
> + .set_fabric_sense_len = tcm_vhost_set_fabric_sense_len,
> + /*
> + * Setup function pointers for generic logic in target_core_fabric_configfs.c
> + */
> + .fabric_make_wwn = tcm_vhost_make_tport,
> + .fabric_drop_wwn = tcm_vhost_drop_tport,
> + .fabric_make_tpg = tcm_vhost_make_tpg,
> + .fabric_drop_tpg = tcm_vhost_drop_tpg,
> + .fabric_post_link = tcm_vhost_port_link,
> + .fabric_pre_unlink = tcm_vhost_port_unlink,
> + .fabric_make_np = NULL,
> + .fabric_drop_np = NULL,
> + .fabric_make_nodeacl = tcm_vhost_make_nodeacl,
> + .fabric_drop_nodeacl = tcm_vhost_drop_nodeacl,
> +};
> +
> +static int tcm_vhost_register_configfs(void)
> +{
> + struct target_fabric_configfs *fabric;
> + int ret;
> +
> + pr_debug("TCM_VHOST fabric module %s on %s/%s"
> + " on "UTS_RELEASE"\n",TCM_VHOST_VERSION, utsname()->sysname,
> + utsname()->machine);
> + /*
> + * Register the top level struct config_item_type with TCM core
> + */
> + fabric = target_fabric_configfs_init(THIS_MODULE, "vhost");
> + if (IS_ERR(fabric)) {
> + pr_err("target_fabric_configfs_init() failed\n");
> + return PTR_ERR(fabric);
> + }
> + /*
> + * Setup fabric->tf_ops from our local tcm_vhost_ops
> + */
> + fabric->tf_ops = tcm_vhost_ops;
> + /*
> + * Setup default attribute lists for various fabric->tf_cit_tmpl
> + */
> + TF_CIT_TMPL(fabric)->tfc_wwn_cit.ct_attrs = tcm_vhost_wwn_attrs;
> + TF_CIT_TMPL(fabric)->tfc_tpg_base_cit.ct_attrs = tcm_vhost_tpg_attrs;
> + TF_CIT_TMPL(fabric)->tfc_tpg_attrib_cit.ct_attrs = NULL;
> + TF_CIT_TMPL(fabric)->tfc_tpg_param_cit.ct_attrs = NULL;
> + TF_CIT_TMPL(fabric)->tfc_tpg_np_base_cit.ct_attrs = NULL;
> + TF_CIT_TMPL(fabric)->tfc_tpg_nacl_base_cit.ct_attrs = NULL;
> + TF_CIT_TMPL(fabric)->tfc_tpg_nacl_attrib_cit.ct_attrs = NULL;
> + TF_CIT_TMPL(fabric)->tfc_tpg_nacl_auth_cit.ct_attrs = NULL;
> + TF_CIT_TMPL(fabric)->tfc_tpg_nacl_param_cit.ct_attrs = NULL;
> + /*
> + * Register the fabric for use within TCM
> + */
> + ret = target_fabric_configfs_register(fabric);
> + if (ret < 0) {
> + pr_err("target_fabric_configfs_register() failed"
> + " for TCM_VHOST\n");
> + return ret;
> + }
> + /*
> + * Setup our local pointer to *fabric
> + */
> + tcm_vhost_fabric_configfs = fabric;
> + pr_debug("TCM_VHOST[0] - Set fabric -> tcm_vhost_fabric_configfs\n");
> + return 0;
> +};
> +
> +static void tcm_vhost_deregister_configfs(void)
> +{
> + if (!tcm_vhost_fabric_configfs)
> + return;
> +
> + target_fabric_configfs_deregister(tcm_vhost_fabric_configfs);
> + tcm_vhost_fabric_configfs = NULL;
> + pr_debug("TCM_VHOST[0] - Cleared tcm_vhost_fabric_configfs\n");
> +};
> +
> +static int __init tcm_vhost_init(void)
> +{
> + int ret = -ENOMEM;
> +
> + tcm_vhost_workqueue = alloc_workqueue("tcm_vhost", 0, 0);
> + if (!tcm_vhost_workqueue)
> + goto out;
> +
> + ret = vhost_scsi_register();
> + if (ret < 0)
> + goto out_destroy_workqueue;
> +
> + ret = tcm_vhost_register_configfs();
> + if (ret < 0)
> + goto out_vhost_scsi_deregister;
> +
> + return 0;
> +
> +out_vhost_scsi_deregister:
> + vhost_scsi_deregister();
> +out_destroy_workqueue:
> + destroy_workqueue(tcm_vhost_workqueue);
> +out:
> + return ret;
> +};
> +
> +static void tcm_vhost_exit(void)
> +{
> + tcm_vhost_deregister_configfs();
> + vhost_scsi_deregister();
> + destroy_workqueue(tcm_vhost_workqueue);
> +};
> +
> +MODULE_DESCRIPTION("TCM_VHOST series fabric driver");
> +MODULE_LICENSE("GPL");
> +module_init(tcm_vhost_init);
> +module_exit(tcm_vhost_exit);
> diff --git a/drivers/vhost/tcm_vhost.h b/drivers/vhost/tcm_vhost.h
> new file mode 100644
> index 0000000..e942df9
> --- /dev/null
> +++ b/drivers/vhost/tcm_vhost.h
> @@ -0,0 +1,90 @@
> +#define TCM_VHOST_VERSION "v0.1"
> +#define TCM_VHOST_NAMELEN 256
> +#define TCM_VHOST_MAX_CDB_SIZE 32
> +
> +struct tcm_vhost_cmd {
> + /* Descriptor from vhost_get_vq_desc() for virt_queue segment */
> + int tvc_vq_desc;
> + /* The Tag from include/linux/virtio_scsi.h:struct virtio_scsi_cmd_req */
> + u64 tvc_tag;
> + /* The number of scatterlists associated with this cmd */
> + u32 tvc_sgl_count;
> + /* Saved unpacked SCSI LUN for tcm_vhost_submission_work() */
> + u32 tvc_lun;
> + /* Pointer to the SGL formatted memory from virtio-scsi */
> + struct scatterlist *tvc_sgl;
> + /* Pointer to response */
> + struct virtio_scsi_cmd_resp __user *tvc_resp;
> + /* Pointer to vhost_scsi for our device */
> + struct vhost_scsi *tvc_vhost;
> + /* The TCM I/O descriptor that is accessed via container_of() */
> + struct se_cmd tvc_se_cmd;
> + /* work item used for cmwq dispatch to tcm_vhost_submission_work() */
> + struct work_struct work;
> + /* Copy of the incoming SCSI command descriptor block (CDB) */
> + unsigned char tvc_cdb[TCM_VHOST_MAX_CDB_SIZE];
> + /* Sense buffer that will be mapped into outgoing status */
> + unsigned char tvc_sense_buf[TRANSPORT_SENSE_BUFFER];
> + /* Completed commands list, serviced from vhost worker thread */
> + struct list_head tvc_completion_list;
> +};
> +
> +struct tcm_vhost_nexus {
> + /* Pointer to TCM session for I_T Nexus */
> + struct se_session *tvn_se_sess;
> +};
> +
> +struct tcm_vhost_nacl {
> + /* Binary World Wide unique Port Name for Vhost Initiator port */
> + u64 iport_wwpn;
> + /* ASCII formatted WWPN for Sas Initiator port */
> + char iport_name[TCM_VHOST_NAMELEN];
> + /* Returned by tcm_vhost_make_nodeacl() */
> + struct se_node_acl se_node_acl;
> +};
> +
> +struct tcm_vhost_tpg {
> + /* Vhost port target portal group tag for TCM */
> + u16 tport_tpgt;
> + /* Used to track number of TPG Port/Lun Links wrt to explict I_T Nexus shutdown */
> + atomic_t tv_tpg_port_count;
> + /* Used for vhost_scsi device reference to tpg_nexus */
> + atomic_t tv_tpg_vhost_count;
> + /* list for tcm_vhost_list */
> + struct list_head tv_tpg_list;
> + /* Used to protect access for tpg_nexus */
> + struct mutex tv_tpg_mutex;
> + /* Pointer to the TCM VHost I_T Nexus for this TPG endpoint */
> + struct tcm_vhost_nexus *tpg_nexus;
> + /* Pointer back to tcm_vhost_tport */
> + struct tcm_vhost_tport *tport;
> + /* Returned by tcm_vhost_make_tpg() */
> + struct se_portal_group se_tpg;
> +};
> +
> +struct tcm_vhost_tport {
> + /* SCSI protocol the tport is providing */
> + u8 tport_proto_id;
> + /* Binary World Wide unique Port Name for Vhost Target port */
> + u64 tport_wwpn;
> + /* ASCII formatted WWPN for Vhost Target port */
> + char tport_name[TCM_VHOST_NAMELEN];
> + /* Returned by tcm_vhost_make_tport() */
> + struct se_wwn tport_wwn;
> +};
> +
> +/*
> + * As per request from MST, keep TCM_VHOST related ioctl defines out of
> + * linux/vhost.h (user-space) for now..
> + */
> +
> +#include <linux/vhost.h>
> +
> +struct vhost_scsi_target {
> + unsigned char vhost_wwpn[TRANSPORT_IQN_LEN];
> + unsigned short vhost_tpgt;
> +};
> +
> +/* VHOST_SCSI specific defines */
> +#define VHOST_SCSI_SET_ENDPOINT _IOW(VHOST_VIRTIO, 0x40, struct vhost_scsi_target)
> +#define VHOST_SCSI_CLEAR_ENDPOINT _IOW(VHOST_VIRTIO, 0x41, struct vhost_scsi_target)
> --
> 1.7.2.5
^ permalink raw reply
* Re: 0xB16B00B5? Really? (was Re: Move hyperv out of the drivers/staging/ directory)
From: valdis.kletnieks @ 2012-07-21 23:21 UTC (permalink / raw)
To: richard -rw- weinberger
Cc: KY Srinivasan, Bjørn Mork,
Greg KH (gregkh@linuxfoundation.org), Paolo Bonzini,
devel@linuxdriverproject.org, linux-kernel@vger.kernel.org,
virtualization@lists.osdl.org
In-Reply-To: <CAFLxGvwqKLhPWNfUJYm1KDXWEX1DNhPBGGApYFqMM0BsB87iig@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 1160 bytes --]
On Fri, 20 Jul 2012 17:03:59 +0200, richard -rw- weinberger said:
> On Fri, Jul 20, 2012 at 4:00 PM, KY Srinivasan <kys@microsoft.com> wrote:
> > Thank you for your interest in fixing this problem. When we decide to change this
> > ID, we will conform to the MSFT guidelines on constructing this guest ID.
> >
>
> I'm wondering why it hasn't been conform to the MSFT guidelines from
> the very beginning on?
It's a lot easier to sneak something cute into code when it isn't maintained in
git repositories and mailing lists readable by everybody and their pet llama.
I'll admit to not doing a *lot* of extensive review, but I try to at least read
Kconfig patches (mostly out of self-defense so when I do a 'make oldconfig' I
get presented with something useful ;) - and it's amazing how often I spot
issues in stuff that's presumably already had several pair of eyeballs look at
it already.
Probably what happened - some programmer had a 60 hour week, wrote the code on
a Friday and had a sudden attack of the sillies, and it went through a code
review meeting, but they had to cover 800 lines of code in a one-hour meeting
so nobody looked *too* close...
[-- Attachment #2: Type: application/pgp-signature, Size: 865 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox