[RFC] pv-scsi driver (scsiback/scsifront)

[RFC] pv-scsi driver (scsiback/scsifront)

[Tomonari Horikoshi]

[-- Attachment #1: Mail message body --]
[-- Type: text/plain, Size: 1105 bytes --]


Hi all.

 We developped a pv-scsi driver that we refered Fujita-san's scsi-driver
 and blkback.
 (see, http://www.xensource.com/files/xensummit_4/Xen_Summit_8_Matsumoto.pdf)

 The pv-scsi driver's feature is as follow:
  * Guest has dedicated SCSI-HBAs of Dom0.
  * Guest can send scsi_cdb to the HBAs.
  * Guest recognises the HBAs from hostno of xenstore.

 Currentlly, We are developping FC version based on this.

* Future work:
  * implement python code
  * performance tunning
  * attach, detach
  * suspend, resume

* We are wondering about:
 * We used "scsihost" as xenstore nodename. Is it suitable?
 * We consider about configfile format...  
   scsihost = ['fc,0', 'scsi,1', 'type,num']
               type = "fc" or "scsi"
               num  = scsi host number on Dom0
   Do you have any comment?
 * We have no idea how to implement suspend/resume feature.
   ex. Physical HBA mapping for resumed guest.
       Pending I/O.
       The WWN within FC mode for resumed guest.
       Influence of migration.
       ...

  Could you suggest to us about this?



Best regards,
Tomonari Horikoshi


[-- Attachment #2: scsiback.patch --]
[-- Type: application/octet-stream, Size: 27935 bytes --]

# HG changeset patch
# User t.horikoshi@jp.fujitsu.com
# Date 1179302059 -32400
# Node ID 6913be6b4f539aaf8a7ed6deb88247843eb88e2f
# Parent  d4a0706d6747cd54613328fe630c3bcb7cafbac5
add scsiback driver

Signed-off-by: Tomonari Horikoshi <t.horikoshi@jp.fujitsu.com>
Signed-off-by: Tsunehisa Doi <Doi.Tsunehisa@jp.fujitsu.com>
Signed-off-by: Jun Kamada <kama@jp.fujitsu.com>
Signed-off-by: Akira Hayakawa <hayakawa.akira@jp.fujitsu.com>

diff -r d4a0706d6747 -r 6913be6b4f53 linux-2.6-xen-sparse/drivers/xen/scsiback/Makefile
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/linux-2.6-xen-sparse/drivers/xen/scsiback/Makefile	Wed May 16 16:54:19 2007 +0900
@@ -0,0 +1,2 @@
+obj-$(CONFIG_XEN_SCSI_BACKEND)	+= scsibk.o
+scsibk-y			+= interface.o scsiback.o
diff -r d4a0706d6747 -r 6913be6b4f53 linux-2.6-xen-sparse/drivers/xen/scsiback/common.h
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/linux-2.6-xen-sparse/drivers/xen/scsiback/common.h	Wed May 16 16:54:19 2007 +0900
@@ -0,0 +1,103 @@
+/*
+ * Copyright (c) 2007, FUJITSU Limited
+ *
+ * Based on the blkback driver code.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version 2
+ * as published by the Free Software Foundation; or, when distributed
+ * separately from the Linux kernel or incorporated into other
+ * software packages, subject to the following license:
+ * 
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this source file (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use, copy, modify,
+ * merge, publish, distribute, sublicense, and/or sell copies of the Software,
+ * and to permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ * 
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ * 
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ */
+
+#ifndef __SCSIIF__BACKEND__COMMON_H__
+#define __SCSIIF__BACKEND__COMMON_H__
+
+#include <linux/version.h>
+#include <linux/module.h>
+#include <linux/interrupt.h>
+#include <linux/slab.h>
+#include <linux/vmalloc.h>
+#include <linux/wait.h>
+#include <asm/io.h>
+#include <asm/setup.h>
+#include <asm/pgalloc.h>
+#include <xen/evtchn.h>
+#include <asm/hypervisor.h>
+#include <xen/gnttab.h>
+#include <xen/driver_util.h>
+#include <xen/xenbus.h>
+#include <xen/interface/io/ring.h>
+#include <xen/interface/grant_table.h>
+#include <xen/interface/io/scsiif.h>
+
+#define DPRINTK(_f, _a...)			\
+	pr_debug("(file=%s, line=%d) " _f,	\
+		 __FILE__ , __LINE__ , ## _a )
+
+struct scsiback_info {
+	struct xenbus_device *dev;
+	struct Scsi_Host *host;
+	struct xenbus_watch backend_watch;
+
+	unsigned int evtchn;
+	unsigned int irq;
+
+	unsigned short host_no;
+
+	scsiif_back_ring_t ring;
+	struct vm_struct *ring_area;
+
+	grant_handle_t shmem_handle;
+	grant_ref_t shmem_ref;
+
+	struct work_struct scsiback_work;
+
+	spinlock_t ring_lock;
+	atomic_t refcnt;
+
+	struct task_struct *kthread;
+	wait_queue_head_t waiting_to_free;
+	wait_queue_head_t wq;
+	unsigned int waiting_reqs;
+
+	struct page **mmap_pages;
+};
+
+irqreturn_t scsiback_intr(int, void *, struct pt_regs *);
+int scsiback_init_sring(struct scsiback_info *,
+				unsigned long, unsigned int);
+int scsiback_schedule(void *data);
+
+
+#define scsiback_get(_b) (atomic_inc(&(_b)->refcnt))
+#define scsiback_put(_b)				\
+	do {						\
+		if (atomic_dec_and_test(&(_b)->refcnt))	\
+			wake_up(&(_b)->waiting_to_free);\
+	} while (0)
+
+struct scsiback_info *scsiinfo_alloc(domid_t domid);
+void scsiback_free(struct scsiback_info *info);
+void scsiback_disconnect(struct scsiback_info *info);
+void __init scsiback_interface_init(void);
+
+#endif /* __SCSIIF__BACKEND__COMMON_H__ */
diff -r d4a0706d6747 -r 6913be6b4f53 linux-2.6-xen-sparse/drivers/xen/scsiback/interface.c
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/linux-2.6-xen-sparse/drivers/xen/scsiback/interface.c	Wed May 16 16:54:19 2007 +0900
@@ -0,0 +1,165 @@
+/*
+ * interface management.
+ *
+ * Copyright (c) 2007, FUJITSU Limited
+ *
+ * Based on the blkback driver code.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version 2
+ * as published by the Free Software Foundation; or, when distributed
+ * separately from the Linux kernel or incorporated into other
+ * software packages, subject to the following license:
+ * 
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this source file (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use, copy, modify,
+ * merge, publish, distribute, sublicense, and/or sell copies of the Software,
+ * and to permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ * 
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ * 
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ */
+
+#include "common.h"
+#include <xen/evtchn.h>
+#include <linux/kthread.h>
+
+static kmem_cache_t *scsiback_cachep;
+
+struct scsiback_info *scsiinfo_alloc(domid_t domid)
+{
+	struct scsiback_info *info;
+
+	info = kmem_cache_alloc(scsiback_cachep, GFP_KERNEL);
+	if (!info)
+		return ERR_PTR(-ENOMEM);
+
+	memset(info, 0, sizeof(*info));
+	spin_lock_init(&info->ring_lock);
+	atomic_set(&info->refcnt, 1);
+	init_waitqueue_head(&info->wq);
+	init_waitqueue_head(&info->waiting_to_free);
+
+	return info;
+}
+
+static int map_frontend_page(
+	struct scsiback_info *info, unsigned long shared_page)
+{
+	struct gnttab_map_grant_ref op;
+	int err;
+
+	gnttab_set_map_op(&op, (unsigned long)info->ring_area->addr,
+				GNTMAP_host_map, shared_page,
+				info->dev->otherend_id);
+
+	err = HYPERVISOR_grant_table_op(GNTTABOP_map_grant_ref, &op, 1);
+	BUG_ON(err);
+
+	if (op.status) {
+		printk(" Grant table operation failure !\n");
+		return op.status;
+	}
+
+	info->shmem_ref = shared_page;
+	info->shmem_handle = op.handle;
+
+	return 0;
+}
+
+static void unmap_frontend_page(struct scsiback_info *info)
+{
+	struct gnttab_unmap_grant_ref op;
+	int err;
+
+	gnttab_set_unmap_op(&op, (unsigned long)info->ring_area->addr,
+				GNTMAP_host_map, info->shmem_handle);
+
+	err = HYPERVISOR_grant_table_op(GNTTABOP_unmap_grant_ref, &op, 1);
+	BUG_ON(err);
+}
+
+int scsiback_init_sring(struct scsiback_info *info,
+			 unsigned long shared_page, unsigned int evtchn)
+{
+	struct scsiif_sring *sring;
+	int err;
+
+	if (info->irq) {
+		printk("Already connected through?\n");
+		return 0;
+	}
+
+	info->ring_area = alloc_vm_area(PAGE_SIZE);
+	if (!info)
+		return -ENOMEM;
+
+	err = map_frontend_page(info, shared_page);
+	if (err)
+		goto free_vm;
+
+	sring = (struct scsiif_sring *) info->ring_area->addr;
+	BACK_RING_INIT(&info->ring, sring, PAGE_SIZE);
+
+	err = bind_interdomain_evtchn_to_irqhandler(
+			info->dev->otherend_id, evtchn,
+			scsiback_intr, 0, "scsiif-backend", info);
+
+	if (err < 0)
+		goto unmap_page;
+
+	info->irq = err;
+	return 0;
+
+unmap_page:
+	unmap_frontend_page(info);
+free_vm:
+	free_vm_area(info->ring_area);
+	return err;
+}
+
+void scsiback_disconnect(struct scsiback_info *info)
+{
+	if (info->kthread) {
+		kthread_stop(info->kthread);
+		info->kthread = NULL;
+	}
+
+	atomic_dec(&info->refcnt);
+	wait_event(info->waiting_to_free, atomic_read(&info->refcnt) == 0);
+	atomic_inc(&info->refcnt);
+
+	if (info->irq) {
+		unbind_from_irqhandler(info->irq, info);
+		info->irq = 0;
+	}
+
+	if (info->ring.sring) {
+		unmap_frontend_page(info);
+		free_vm_area(info->ring_area);
+		info->ring.sring = NULL;
+	}
+}
+
+void scsiback_free(struct scsiback_info *info)
+{
+	if (!atomic_dec_and_test(&info->refcnt))
+		BUG();
+	kmem_cache_free(scsiback_cachep, info);
+}
+
+void __init scsiback_interface_init(void)
+{
+	scsiback_cachep = kmem_cache_create("scsiif_cache",
+		sizeof(struct scsiback_info), 0, 0, NULL, NULL);
+}
diff -r d4a0706d6747 -r 6913be6b4f53 linux-2.6-xen-sparse/drivers/xen/scsiback/scsiback.c
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/linux-2.6-xen-sparse/drivers/xen/scsiback/scsiback.c	Wed May 16 16:54:19 2007 +0900
@@ -0,0 +1,716 @@
+/*
+ * Xen SCSI backend driver
+ *
+ * Copyright (c) 2007, FUJITSU Limited
+ *
+ * Based on the blkback driver code.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version 2
+ * as published by the Free Software Foundation; or, when distributed
+ * separately from the Linux kernel or incorporated into other
+ * software packages, subject to the following license:
+ * 
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this source file (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use, copy, modify,
+ * merge, publish, distribute, sublicense, and/or sell copies of the Software,
+ * and to permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ * 
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ * 
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ */
+
+#include <linux/list.h>
+#include <linux/blkdev.h>
+#include <linux/kthread.h>
+#include <xen/balloon.h>
+#include <asm/hypervisor.h>
+#include <scsi/scsi.h>
+#include <scsi/scsi_cmnd.h>
+#include <scsi/scsi_host.h>
+#include <scsi/scsi_device.h>
+#include <scsi/scsi_dbg.h>
+#include "common.h"
+
+
+#define INVALID_GRANT_HANDLE	0xFFFF
+
+typedef struct {
+	struct scsiback_info *info;
+	unsigned long rqid;
+	int use_sg;
+	atomic_t pendcnt;
+	struct request *rq;
+	struct list_head free_list;
+} pending_req_t;
+
+static pending_req_t *pending_reqs;
+static struct list_head pending_free;
+static DEFINE_SPINLOCK(pending_free_lock);
+static DECLARE_WAIT_QUEUE_HEAD(pending_free_wq);
+
+#define SCSIBACK_INVALID_HANDLE (~0)
+
+static struct page **pending_pages;
+static grant_handle_t *pending_grant_handles;
+
+
+static inline int vaddr_pagenr(pending_req_t *req, int seg)
+{
+	return (req - pending_reqs) * SG_TABLESIZE + seg;
+}
+
+static inline unsigned long vaddr(pending_req_t *req, int seg)
+{
+	unsigned long pfn = page_to_pfn(pending_pages[vaddr_pagenr(req, seg)]);
+	return (unsigned long)pfn_to_kaddr(pfn);
+}
+
+#define pending_handle(_req, _seg) \
+	(pending_grant_handles[vaddr_pagenr(_req, _seg)])
+
+
+
+/******************************************************************
+ * misc small helpers
+ */
+static pending_req_t* alloc_req(void)
+{
+	pending_req_t *req = NULL;
+	unsigned long flags;
+
+	spin_lock_irqsave(&pending_free_lock, flags);
+	if (!list_empty(&pending_free)) {
+		req = list_entry(pending_free.next, pending_req_t, free_list);
+		list_del(&req->free_list);
+	}
+	spin_unlock_irqrestore(&pending_free_lock, flags);
+	return req;
+}
+
+static void free_req(pending_req_t *req)
+{
+	unsigned long flags;
+	int was_empty;
+
+	spin_lock_irqsave(&pending_free_lock, flags);
+	was_empty = list_empty(&pending_free);
+	list_add(&req->free_list, &pending_free);
+	spin_unlock_irqrestore(&pending_free_lock, flags);
+	if (was_empty)
+		wake_up(&pending_free_wq);
+}
+
+
+static void fast_flush_area(pending_req_t *req)
+{
+	struct gnttab_unmap_grant_ref unmap[SG_TABLESIZE];
+	unsigned int i, invcount = 0;
+	grant_handle_t handle;
+	int err;
+
+	for (i = 0; i < req->use_sg; i++) {
+		handle = pending_handle(req, i);
+		if (handle == SCSIBACK_INVALID_HANDLE)
+			continue;
+		gnttab_set_unmap_op(&unmap[i], vaddr(req, i),
+					GNTMAP_host_map, handle);
+		pending_handle(req, i) = SCSIBACK_INVALID_HANDLE;
+		invcount++;
+	}
+
+	err = HYPERVISOR_grant_table_op(
+		GNTTABOP_unmap_grant_ref, unmap, invcount);
+	BUG_ON(err);
+}
+
+
+/******************************************************************************
+ * NOTIFICATION FROM GUEST OS.
+ */
+static void scsiback_notify_work(struct scsiback_info *info)
+{
+	info->waiting_reqs = 1;
+	wake_up(&info->wq);
+}
+
+irqreturn_t scsiback_intr(int irq, void *dev_id, struct pt_regs *regs)
+{
+	scsiback_notify_work(dev_id);
+	return IRQ_HANDLED;
+}
+
+static void make_sense(struct scsiback_info *info, struct request *req,
+			int32_t result, uint64_t rqid)
+{
+	struct scsiif_response *resp;
+	unsigned long flags;
+	struct scsiif_back_ring *ring = &info->ring;
+	int more_to_do = 0;
+	int notify;
+
+	spin_lock_irqsave(&info->ring_lock, flags);
+
+	resp = RING_GET_RESPONSE(ring, ring->rsp_prod_pvt);
+	memset(resp->sense_buffer, 0, SCSI_SENSE_BUFFERSIZE);
+
+	resp->result = result;
+	resp->rqid   = rqid;
+
+	if (req) {
+		memcpy(resp->sense_buffer, req->sense, req->sense_len);
+		resp->sense_len = req->sense_len;
+	} else
+		resp->sense_len = 0;
+
+	ring->rsp_prod_pvt++;
+
+	RING_PUSH_RESPONSES_AND_CHECK_NOTIFY(ring, notify);
+
+	if (ring->rsp_prod_pvt == ring->req_cons) {
+		RING_FINAL_CHECK_FOR_REQUESTS(ring, more_to_do);
+
+	} else if (RING_HAS_UNCONSUMED_REQUESTS(ring)) {
+		more_to_do = 1;
+	}
+
+	spin_unlock_irqrestore(&info->ring_lock, flags);
+
+	if (more_to_do)
+		scsiback_notify_work(info);
+	if (notify)
+		notify_remote_via_irq(info->irq);
+}
+
+
+static void scsiback_end_cmd_fn(struct request *req, int error)
+{
+	unsigned char sense_buffer[SCSI_SENSE_BUFFERSIZE];
+	pending_req_t *pending_req = req->end_io_data;
+	pending_req->rq = req;
+
+	if (req->errors != 0) {
+		printk("Buffer not up-to-date at end of operation, "
+			"req->errors=0x%08x\n", req->errors);
+		memcpy(sense_buffer, req->sense, req->sense_len);
+		__scsi_print_sense("scsiback_end_cmd_fn", sense_buffer,
+				   req->sense_len);
+	}
+
+	if (atomic_dec_and_test(&pending_req->pendcnt)) {
+		fast_flush_area(pending_req);
+		make_sense(pending_req->info, pending_req->rq, 
+				req->errors, pending_req->rqid);
+		scsiback_put(pending_req->info);
+		free_req(pending_req);
+	}
+	
+	__blk_put_request(req->q, req);
+}
+
+
+/* quoted scsi_lib.c/scsi_merge_bio */
+static int scsiback_merge_bio(struct request *rq, struct bio *bio)
+{
+	struct request_queue *q = rq->q;
+
+	bio->bi_flags &= ~(1 << BIO_SEG_VALID);
+	if (rq_data_dir(rq) == WRITE)
+		bio->bi_rw |= (1 << BIO_RW);
+
+	blk_queue_bounce(q, &bio);
+
+	if (!rq->bio)
+		blk_rq_bio_prep(q, rq, bio);
+	else if (!q->back_merge_fn(q, rq, bio))
+		return -EINVAL;
+	else {
+		rq->biotail->bi_next = bio;
+		rq->biotail          = bio;
+		rq->hard_nr_sectors += bio_sectors(bio);
+		rq->nr_sectors       = rq->hard_nr_sectors;
+	}
+
+	return 0;
+}
+
+
+/* quoted scsi_lib.c/scsi_bi_endio */
+static int scsiback_bi_endio(struct bio *bio, unsigned int bytes_done, int error)
+{
+	if (bio->bi_size)
+		return 1;
+
+	bio_put(bio);
+	return 0;
+}
+
+
+/* quoted scsi_lib.c/scsi_req_map_sg . */
+static int requset_map_sg(struct scsiif_request *ring_req,
+				pending_req_t *pending_req)
+{
+	struct request *rq = pending_req->rq;
+	struct request_queue *q = pending_req->rq->q;
+	int nr_pages;
+	int nsegs = pending_req->use_sg;
+	unsigned int data_len = 0, len, bytes, off;
+	struct page *page;
+	struct bio *bio = NULL;
+	int i, err, nr_vecs = 0;
+
+	for (i = 0; i < nsegs; i++) {
+		page = virt_to_page(vaddr(pending_req, i));
+		off = ring_req->seg[i].offset;
+		len = ring_req->seg[i].length;
+		data_len += len;
+
+		nr_pages = (len + off + PAGE_SIZE - 1) >> PAGE_SHIFT;
+
+		while (len > 0) {
+			bytes = min_t(unsigned int, len, PAGE_SIZE - off);
+
+			if (!bio) {
+				nr_vecs = min_t(int, BIO_MAX_PAGES, nr_pages);
+				nr_pages -= nr_vecs;
+				bio = bio_alloc(GFP_KERNEL, nr_vecs);
+				if (!bio) {
+					err = -ENOMEM;
+					goto free_bios;
+				}
+				bio->bi_end_io = scsiback_bi_endio;
+			}
+
+			if (bio_add_pc_page(q, bio, page, bytes, off) !=
+						bytes) {
+				bio_put(bio);
+				err = -EINVAL;
+				goto free_bios;
+			}
+
+			if (bio->bi_vcnt >= nr_vecs) {
+				err = scsiback_merge_bio(rq, bio);
+				if (err) {
+					bio_endio(bio, bio->bi_size, 0);
+					goto free_bios;
+				}
+				bio = NULL;
+			}
+
+			page++;
+			len -= bytes;
+			off = 0;
+		}
+	}
+
+	rq->buffer   = rq->data = NULL;
+	rq->data_len = data_len;
+	return 0;
+
+free_bios:
+	while ((bio = rq->bio) != NULL) {
+		rq->bio = bio->bi_next;
+		/*
+		 * call endio instead of bio_put incase it was bounced
+		 */
+		bio_endio(bio, bio->bi_size, 0);
+	}
+	return err;
+}
+
+
+static void scsiback_cmd_exec(struct scsiback_info *info,
+		struct scsiif_request *ring_req, pending_req_t *pending_req)
+{
+
+	struct gnttab_map_grant_ref map[SG_TABLESIZE];
+	struct scsi_device *sdev;
+	char sense[SCSI_SENSE_BUFFERSIZE];
+
+	int i, err = 0;
+	int write = (ring_req->sc_data_direction == DMA_TO_DEVICE);
+	u32 flags, use_sg;
+
+	use_sg = ring_req->use_sg;
+	sdev   = scsi_device_lookup(info->host, ring_req->channel,
+					ring_req->id, ring_req->lun);
+
+	if (sdev == NULL) {
+		printk("Scsi Device Not Found\n");
+		goto fail_response;
+	}
+
+	pending_req->rq     = blk_get_request(sdev->request_queue,
+						write, GFP_KERNEL);
+	pending_req->info   = info;
+	pending_req->rqid   = ring_req->rqid;
+	pending_req->use_sg = use_sg;
+
+
+	pending_req->rq->flags  |= REQ_BLOCK_PC;
+	pending_req->rq->cmd_len = ring_req->cmd_len;
+	memcpy(pending_req->rq->cmd, ring_req->cmnd, ring_req->cmd_len);
+
+	memset(sense, 0, sizeof(sense)); /*FIXME*/
+	pending_req->rq->sense     = sense;
+	pending_req->rq->sense_len = 0;
+
+	pending_req->rq->retries   = ring_req->retries;
+	pending_req->rq->timeout   = ring_req->timeout_per_command;
+
+	pending_req->rq->end_io_data = pending_req;
+
+	for (i = 0; i < use_sg; i++) {
+		flags = GNTMAP_host_map;
+		if (write)
+			flags |= GNTMAP_readonly;
+
+		gnttab_set_map_op(&map[i], vaddr(pending_req, i), flags,
+					ring_req->seg[i].gref,
+					info->dev->otherend_id);
+	}
+
+	if (use_sg) { 
+		err = HYPERVISOR_grant_table_op(GNTTABOP_map_grant_ref,
+							map, use_sg);
+		BUG_ON(err);
+	}
+
+	for (i = 0; i < use_sg; i++) {		
+		if (unlikely(map[i].status != 0)) {
+			printk("invalid buffer -- could not remap it\n");
+			map[i].handle = SCSIBACK_INVALID_HANDLE;
+			err |= 1;
+		}
+
+		pending_handle(pending_req, i) = map[i].handle;
+
+		if (err)
+			continue;
+
+		set_phys_to_machine(__pa(vaddr(
+			pending_req, i)) >> PAGE_SHIFT,
+			FOREIGN_FRAME(map[i].dev_bus_addr >> PAGE_SHIFT));
+	}
+
+	if (err)
+		goto fail_flush;
+
+	if (use_sg) { 
+		if (requset_map_sg(ring_req, pending_req)) {
+			printk("SG Request Map Error\n");
+			goto fail_map;
+		}
+	}
+
+	atomic_set(&pending_req->pendcnt, 1);
+	scsiback_get(info);
+
+	blk_execute_rq_nowait(pending_req->rq->q, NULL, pending_req->rq,
+				1, scsiback_end_cmd_fn);
+
+	return ;
+
+fail_map:
+fail_flush:
+	fast_flush_area(pending_req);
+fail_response:
+	make_sense(info, NULL, 1, ring_req->rqid);
+	free_req(pending_req);
+	printk("%s error\n",__FUNCTION__);
+}
+
+static int do_scsiback_cmd_fn(struct scsiback_info *info)
+{
+	struct scsiif_back_ring *ring = &info->ring;
+	struct scsiif_request *ring_req;
+
+	pending_req_t *pending_req;
+	RING_IDX rc, rp;
+	int more_to_do = 0;
+
+	rc = ring->req_cons;
+	rp = ring->sring->req_prod;
+	rmb();
+
+	while ((rc != rp) && !RING_REQUEST_CONS_OVERFLOW(ring, rc)) {
+
+		pending_req = alloc_req();
+		if (NULL == pending_req) {
+			more_to_do = 1;
+			break;
+		}
+
+		ring_req = RING_GET_REQUEST(ring, rc);
+		ring->req_cons = ++rc;
+
+		scsiback_cmd_exec(info, ring_req, pending_req);
+	}
+
+	return more_to_do;
+}
+
+int scsiback_schedule(void *data)
+{
+	struct scsiback_info *info = data;
+
+	scsiback_get(info);
+
+	while (!kthread_should_stop()) {
+		wait_event_interruptible(
+			info->wq,
+			info->waiting_reqs || kthread_should_stop());
+		wait_event_interruptible(
+			pending_free_wq,
+			!list_empty(&pending_free) || kthread_should_stop());
+
+		info->waiting_reqs = 0;
+		smp_mb();
+
+		if (do_scsiback_cmd_fn(info))
+			info->waiting_reqs = 1;
+	}
+
+	info->kthread = NULL;
+	scsiback_put(info);
+
+	return 0;
+}
+
+
+static int scsiback_remove(struct xenbus_device *dev)
+{
+	struct scsiback_info *info = dev->dev.driver_data;
+
+	if (info->backend_watch.node) {
+		unregister_xenbus_watch(&info->backend_watch);
+		kfree(info->backend_watch.node);
+		info->backend_watch.node = NULL;
+	}
+
+	if (info) {
+		scsiback_disconnect(info);
+		scsiback_free(info);
+		info = NULL;
+	}
+
+	kfree(info);
+
+	return 0;
+}
+
+
+static int scsiback_connect(struct scsiback_info *info)
+{
+	struct xenbus_device *dev = info->dev;
+	unsigned long ring_ref;
+	unsigned int evtchn;
+	int err;
+
+	err = xenbus_gather(XBT_NIL, dev->otherend, "ring-ref", "%lu",
+			&ring_ref, "event-channel", "%u", &evtchn, NULL);
+	if (err) {
+		xenbus_dev_fatal(dev, err, "reading %s ring", dev->otherend);
+		return err;
+	}
+
+	err = scsiback_init_sring(info, ring_ref, evtchn);
+	if (err) {
+		return err;
+	}
+
+	info->kthread = kthread_run(scsiback_schedule, info, "scsiif");
+	if (IS_ERR(info->kthread)) {
+		err = PTR_ERR(info->kthread);
+		info->kthread = NULL;
+		xenbus_dev_error(info->dev, err, "start scsiif");
+	}
+
+	return 0;
+}
+
+static void scsiback_frontend_changed(struct xenbus_device *dev,
+					enum xenbus_state frontend_state)
+{
+	struct scsiback_info *info = dev->dev.driver_data;
+	int err;
+
+	switch (frontend_state) {
+	case XenbusStateInitialising:
+		break;
+
+	case XenbusStateInitialised:
+	case XenbusStateConnected:
+		if (dev->state == XenbusStateConnected)
+			break;
+
+		err = scsiback_connect(info);
+		if (err)
+			break;
+
+		err = xenbus_switch_state(dev, XenbusStateConnected);
+		if (err)
+			xenbus_dev_fatal(dev, err, "switching to Connected state",
+					dev->nodename);
+		break;
+
+	case XenbusStateClosing:
+		scsiback_disconnect(info);
+		xenbus_switch_state(dev, XenbusStateClosing);
+		break;
+
+	case XenbusStateClosed:
+		xenbus_switch_state(dev, XenbusStateClosed);
+		if (xenbus_dev_is_online(dev))
+			break;
+
+	case XenbusStateUnknown:
+		/*
+		 * workaround.
+		 */
+		if (info->host->host_no)
+			device_unregister(&dev->dev);
+		break;
+	default:
+		xenbus_dev_fatal(dev, -EINVAL, "saw state %d at frontend",
+					frontend_state);
+		break;
+	}
+}
+
+
+static void scsiback_backend_changed(struct xenbus_watch *watch,
+					const char **vec, unsigned int len)
+{
+	struct scsiback_info *info =
+		container_of(watch, struct scsiback_info, backend_watch);
+
+	printk("%s %p %u\n", __FUNCTION__, info->dev, info->dev->state);
+
+	/*FIXME*/
+}
+
+
+static int scsiback_probe(struct xenbus_device *dev,
+			   const struct xenbus_device_id *id)
+{
+	int err, hostno;
+	struct scsiback_info *info;
+
+
+	DPRINTK("%p %d\n", dev, dev->otherend_id);
+
+	err = xenbus_scanf(XBT_NIL, dev->nodename, "hostno", "%d", &hostno);
+	if (err == -ENOENT ) {
+		printk("error xenbus_scanf hostno=%d\n",hostno);
+		goto fail;
+	}
+
+	info = scsiinfo_alloc(dev->otherend_id);
+	if (IS_ERR(info)) {
+		err = PTR_ERR(info);
+		info = NULL;
+		xenbus_dev_fatal(dev, err, "creating block interface");
+		goto fail;
+	}
+
+	info->host = scsi_host_lookup(hostno);
+
+	dev->dev.driver_data = info;
+
+	info->dev = dev;
+	info->irq = 0;
+
+	err = xenbus_watch_path2(dev, dev->nodename,
+				"scsi-host",
+				&info->backend_watch,
+				scsiback_backend_changed);
+	if (err)
+		goto fail;
+
+	err = xenbus_switch_state(dev, XenbusStateInitWait);
+	if (err)
+		goto fail;
+
+	return 0;
+
+fail:
+	printk("%s failed\n",__FUNCTION__);
+	scsiback_remove(dev);
+
+	return err;
+}
+
+
+static struct xenbus_device_id scsiback_ids[] = {
+	{ "scsihost" },
+	{ "" }
+};
+
+static struct xenbus_driver scsiback = {
+	.name			= "scsihost",
+	.owner			= THIS_MODULE,
+	.ids			= scsiback_ids,
+	.probe			= scsiback_probe,
+	.remove			= scsiback_remove,
+	.otherend_changed	= scsiback_frontend_changed
+};
+
+static int __init scsiback_init(void)
+{
+	int err = -ENOMEM;
+	int i, mmap_pages;
+
+	if (!is_running_on_xen())
+		return -ENODEV;
+
+	mmap_pages = CAN_QUEUE * SG_TABLESIZE;
+
+	pending_reqs          = kmalloc(sizeof(pending_reqs[0]) *
+					CAN_QUEUE, GFP_KERNEL);
+	pending_grant_handles = kmalloc(sizeof(pending_grant_handles[0]) *
+					mmap_pages, GFP_KERNEL);
+	pending_pages         = alloc_empty_pages_and_pagevec(mmap_pages);
+
+	if (!pending_reqs || !pending_grant_handles || !pending_pages)
+		goto out_of_memory;
+
+	for (i = 0; i < mmap_pages; i++)
+		pending_grant_handles[i] = SCSIBACK_INVALID_HANDLE;
+
+	scsiback_interface_init();
+
+	memset(pending_reqs, 0, sizeof(pending_reqs));
+	INIT_LIST_HEAD(&pending_free);
+
+	for (i = 0; i < CAN_QUEUE; i++)
+		list_add_tail(&pending_reqs[i].free_list, &pending_free);
+
+	err = xenbus_register_backend(&scsiback);
+	if (err)
+		goto out_of_memory;
+
+	return 0;
+
+ out_of_memory:
+	kfree(pending_reqs);
+	kfree(pending_grant_handles);
+	free_empty_pages_and_pagevec(pending_pages, mmap_pages);
+	printk("%s: out of memory\n", __FUNCTION__);
+	return -ENOMEM;
+}
+
+module_init(scsiback_init);
+
+MODULE_DESCRIPTION("Xen SCSI backend driver");
+MODULE_LICENSE("Dual BSD/GPL");
diff -r d4a0706d6747 -r 6913be6b4f53 xen/include/public/io/scsiif.h
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/xen/include/public/io/scsiif.h	Wed May 16 16:54:19 2007 +0900
@@ -0,0 +1,49 @@
+/*
+ * Copyright(c) FUJITSU Limited 2007.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the Free
+ * Software Foundation; either version 2 of the License, or (at your option)
+ * any later version.
+ *
+ * Based on the blkback/blkfront driver code.
+ *
+ */
+#ifndef __XEN__PUBLIC_IO_SCSI_H__
+#define __XEN__PUBLIC_IO_SCSI_H__
+
+#include "ring.h"
+#include "../grant_table.h"
+
+#define CAN_QUEUE	8
+#define SG_TABLESIZE	11
+
+struct scsiif_request {
+#define MAX_COMMAND_SIZE	16
+	uint8_t cmnd[MAX_COMMAND_SIZE];
+	uint8_t cmd_len;
+	uint32_t id, lun, channel;
+	uint16_t sc_data_direction;
+	uint16_t use_sg;
+	uint16_t request_bufflen;
+	int32_t retries;
+	int32_t timeout_per_command;
+	uint64_t rqid;
+	struct scsiif_request_segment{
+		grant_ref_t gref;
+		uint32_t offset;
+		uint32_t length;
+	} seg[SG_TABLESIZE];
+};
+
+struct scsiif_response {
+	uint64_t rqid;
+	int32_t result;
+	uint64_t sense_len;
+#define SCSI_SENSE_BUFFERSIZE 	96
+	uint8_t sense_buffer[SCSI_SENSE_BUFFERSIZE];
+};
+
+DEFINE_RING_TYPES(scsiif, struct scsiif_request, struct scsiif_response);
+
+#endif

[-- Attachment #3: scsifront.patch --]
[-- Type: application/octet-stream, Size: 14247 bytes --]

# HG changeset patch
# User t.horikoshi@jp.fujitsu.com
# Date 1179302161 -32400
# Node ID 38b476ccc82814fb55e3a36f88c995e353cf63dc
# Parent  6913be6b4f539aaf8a7ed6deb88247843eb88e2f
add scsifront driver

Signed-off-by: Tomonari Horikoshi <t.horikoshi@jp.fujitsu.com>
Signed-off-by: Tsunehisa Doi <Doi.Tsunehisa@jp.fujitsu.com>
Signed-off-by: Jun Kamada <kama@jp.fujitsu.com>
Signed-off-by: Akira Hayakawa <hayakawa.akira@jp.fujitsu.com>

diff -r 6913be6b4f53 -r 38b476ccc828 linux-2.6-xen-sparse/drivers/xen/scsifront/Makefile
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/linux-2.6-xen-sparse/drivers/xen/scsifront/Makefile	Wed May 16 16:56:01 2007 +0900
@@ -0,0 +1,4 @@
+obj-$(CONFIG_XEN_SCSI_FRONTEND)	:= xenscsi.o
+
+xenscsi-objs := scsifront.o
+
diff -r 6913be6b4f53 -r 38b476ccc828 linux-2.6-xen-sparse/drivers/xen/scsifront/scsifront.c
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/linux-2.6-xen-sparse/drivers/xen/scsifront/scsifront.c	Wed May 16 16:56:01 2007 +0900
@@ -0,0 +1,524 @@
+/*
+ * Xen SCSI frontend driver
+ *
+ * Copyright (c) 2007, FUJITSU Limited
+ *
+ * Based on the scsifront driver code by FUJITA Tomonori
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License version 2
+ * as published by the Free Software Foundation; or, when distributed
+ * separately from the Linux kernel or incorporated into other
+ * software packages, subject to the following license:
+ * 
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this source file (the "Software"), to deal in the Software without
+ * restriction, including without limitation the rights to use, copy, modify,
+ * merge, publish, distribute, sublicense, and/or sell copies of the Software,
+ * and to permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ * 
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ * 
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ */
+
+#include <linux/version.h>
+#include <linux/sched.h>
+#include <linux/interrupt.h>
+#include <linux/scatterlist.h>
+#include <linux/uio.h>
+#include <scsi/scsi.h>
+#include <scsi/scsi_cmnd.h>
+#include <scsi/scsi_host.h>
+#include <scsi/scsi_device.h>
+#include <xen/evtchn.h>
+#include <xen/xenbus.h>
+#include <xen/interface/xen.h>
+#include <xen/interface/io/scsiif.h>
+#include <xen/interface/io/ring.h>
+#include <xen/gnttab.h>
+#include <asm/hypervisor.h>
+
+#ifdef HAVE_XEN_PLATFORM_COMPAT_H
+#include <xen/platform-compat.h>
+#endif
+
+#define GRANT_INVALID_REF 0
+
+
+#define DPRINTK(_f, _a...)			\
+	pr_debug("(file=%s, line=%d) " _f,	\
+		 __FILE__ , __LINE__ , ## _a )
+
+
+struct scsi_shadow {
+	struct scsiif_request ring_req;
+	unsigned long req_scsi_cmnd;
+};
+
+#define SCSI_RING_SIZE __RING_SIZE((struct scsiif_sring *)0, PAGE_SIZE)
+
+struct scsifront_info {
+	struct xenbus_device *dev;
+	struct Scsi_Host *host;
+	unsigned int evtchn;
+	unsigned int irq;
+	unsigned long ring_ref;
+	struct scsiif_front_ring ring;
+	struct scsi_shadow shadow[SCSI_RING_SIZE];
+	unsigned long shadow_free;
+};
+
+
+static inline int GET_ID_FROM_FREELIST(
+	struct scsifront_info *info)
+{
+	unsigned long free = info->shadow_free;
+	BUG_ON(free > SCSI_RING_SIZE);
+	info->shadow_free = info->shadow[free].ring_req.rqid;
+	info->shadow[free].ring_req.rqid = 0x0fffffee; /* debug */
+	return free;
+}
+
+static inline void ADD_ID_TO_FREELIST(
+	struct scsifront_info *info, unsigned long id)
+{
+	info->shadow[id].ring_req.rqid  = info->shadow_free;
+	info->shadow[id].req_scsi_cmnd = 0;
+	info->shadow_free = id;
+}
+
+static void scsifront_free(struct scsifront_info *info)
+{
+	
+	scsi_remove_host(info->host);
+	scsi_host_put(info->host);
+
+	flush_scheduled_work();
+
+	if (info->ring_ref != GRANT_INVALID_REF) {
+		gnttab_end_foreign_access(info->ring_ref, 0,
+					(unsigned long)info->ring.sring);
+		info->ring_ref = GRANT_INVALID_REF;
+		info->ring.sring = NULL;
+	}
+
+	if (info->irq)
+		unbind_from_irqhandler(info->irq, info);
+	info->irq = 0;
+}
+
+static int map_data_for_request(struct scsifront_info *info,
+		struct scsi_cmnd *sc, struct scsiif_request *ring_req)
+{
+	struct scatterlist *sg = sc->request_buffer;
+
+	grant_ref_t gref_head;
+	int err, i, ref;
+	int write = (sc->sc_data_direction == DMA_TO_DEVICE);
+
+	if (!sg || sc->sc_data_direction == DMA_NONE)
+		return 0;
+
+	err = gnttab_alloc_grant_references(SG_TABLESIZE, &gref_head);
+	if (err)
+		return -ENOMEM;
+
+	for (i = 0; i < sc->use_sg; i++, sg++) {
+		ref = gnttab_claim_grant_reference(&gref_head);
+		BUG_ON(ref == -ENOSPC);  /*FIXME*/
+
+		gnttab_grant_foreign_access_ref(ref, info->dev->otherend_id,
+			(page_to_phys(sg->page) >> PAGE_SHIFT), write);
+		ring_req->seg[i].gref   = ref;
+		ring_req->seg[i].offset = sg->offset;
+		ring_req->seg[i].length = sg->length;
+	}
+
+	gnttab_free_grant_references(gref_head);
+
+	return 0;
+}
+
+static int scsifront_queuecommand(struct scsi_cmnd *sc,
+					void (*done)(struct scsi_cmnd *))
+{
+	struct Scsi_Host *host = sc->device->host;
+	struct scsifront_info *info = (struct scsifront_info *) host->hostdata;
+	struct scsiif_request *ring_req;
+	struct scsiif_front_ring *ring = &info->ring;
+	int err, notify;
+	unsigned long id;
+
+	if (info->dev->state != XenbusStateConnected || RING_FULL(ring)) {
+		printk("busy %u!\n", info->dev->state);
+		return SCSI_MLQUEUE_HOST_BUSY;
+	}
+	sc->scsi_done = done;
+	sc->result    = 0;
+
+	ring_req = RING_GET_REQUEST(&info->ring, ring->req_prod_pvt);
+
+	ring_req->id      = sc->device->id;
+	ring_req->lun     = sc->device->lun;
+	ring_req->channel = sc->device->channel;
+	ring_req->cmd_len = sc->cmd_len;
+
+	BUG_ON(sc->cmd_len > MAX_COMMAND_SIZE);
+
+	if ( sc->cmd_len )
+		memcpy(ring_req->cmnd, sc->cmnd, sc->cmd_len);
+	else
+		memset(ring_req->cmnd, 0, MAX_COMMAND_SIZE);
+
+	ring_req->use_sg              = sc->use_sg;
+	ring_req->sc_data_direction   = sc->sc_data_direction;
+	ring_req->request_bufflen     = sc->request_bufflen;
+	ring_req->retries             = sc->retries;
+	ring_req->timeout_per_command = sc->timeout_per_command;
+
+	id = GET_ID_FROM_FREELIST(info);	/* use id by response */
+	info->shadow[id].req_scsi_cmnd = (unsigned long)sc;
+	ring_req->rqid                 = id;
+	err = map_data_for_request(info, sc, ring_req);
+	if (err) {
+		printk("%s error\n",__FUNCTION__);
+		return SCSI_MLQUEUE_HOST_BUSY;
+	}
+
+	ring->req_prod_pvt++;
+
+	info->shadow[id].ring_req = *ring_req;
+
+	RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(ring, notify);
+
+	if (notify)
+		notify_remote_via_irq(info->irq);
+
+	return 0;
+}
+
+static int scsifront_eh_abort_handler(struct scsi_cmnd *sc)
+{
+	/* not implemented */
+	BUG();
+	return 0;
+}
+
+static void scsifront_cmd_done(struct scsi_shadow *s)
+{
+	int i;
+
+	if (!s->ring_req.request_bufflen ||
+		(s->ring_req.sc_data_direction != DMA_TO_DEVICE &&
+			s->ring_req.sc_data_direction != DMA_FROM_DEVICE)) {
+		return;
+	}
+
+	if (!s->ring_req.use_sg)
+		return;
+
+	for (i = 0; i < s->ring_req.use_sg; i++ )
+		gnttab_end_foreign_access(s->ring_req.seg[i].gref, 0, 0UL);
+}
+
+static irqreturn_t scsifront_intr(int irq, void *dev_id,
+					struct pt_regs *ptregs)
+{
+	struct scsifront_info *info = (struct scsifront_info *) dev_id;
+	struct scsiif_front_ring *ring = &info->ring;
+	struct scsiif_response *ring_res;
+
+	struct scsi_cmnd *sc;
+	int i, rp;
+	unsigned long id;
+
+	if (info->dev->state != XenbusStateConnected)
+		return IRQ_HANDLED;
+
+again:
+	rp = info->ring.sring->rsp_prod;
+	rmb();
+
+	for (i = info->ring.rsp_cons; i != rp; i++) {
+		ring_res = RING_GET_RESPONSE(ring, i);
+
+		id = ring_res->rqid;
+		sc = (struct scsi_cmnd *)info->shadow[id].req_scsi_cmnd;
+		scsifront_cmd_done(&info->shadow[id]);
+
+		ADD_ID_TO_FREELIST(info, id);
+
+		sc->result = ring_res->result;
+		sc->resid  = 0;
+
+		BUG_ON(ring_res->sense_len > SCSI_SENSE_BUFFERSIZE);
+
+		if (ring_res->sense_len)
+			memcpy(sc->sense_buffer, ring_res->sense_buffer,
+				ring_res->sense_len);
+
+		sc->scsi_done(sc);
+	}
+
+	info->ring.rsp_cons = i;
+	if (i != info->ring.req_prod_pvt) {
+		int more_to_do;
+		RING_FINAL_CHECK_FOR_RESPONSES(ring, more_to_do);
+		if (more_to_do)
+			goto again;
+	} else
+		ring->sring->rsp_event = i + 1;
+
+	return IRQ_HANDLED;
+}
+
+static int scsifront_alloc_ring(struct scsifront_info *info)
+{
+	struct xenbus_device *dev = info->dev;
+	struct scsiif_sring *sring;
+	int err = -ENOMEM;
+
+	info->ring_ref = GRANT_INVALID_REF;
+
+	sring = (struct scsiif_sring *) __get_free_page(GFP_KERNEL);
+	if (!sring) {
+		xenbus_dev_fatal(dev, err, "fail to allocate shared ring");
+		return err;
+	}
+
+	SHARED_RING_INIT(sring);
+	FRONT_RING_INIT(&info->ring, sring, PAGE_SIZE);
+	DPRINTK("0x%x\n", RING_SIZE(&info->ring));
+
+	err = xenbus_grant_ring(dev, virt_to_mfn(info->ring.sring));
+	if (err < 0) {
+		xenbus_dev_fatal(dev, err, "fail to grant shared ring");
+		goto free_sring;
+	}
+	info->ring_ref = err;
+	err = bind_listening_port_to_irqhandler(
+			dev->otherend_id, scsifront_intr,
+			SA_SAMPLE_RANDOM, "scsifront", info);
+
+	if (err <= 0) {
+		xenbus_dev_fatal(dev, err, "bind_listening_port_to_irqhandler");
+		goto fail;
+	}
+	info->irq = err;
+
+	return 0;
+fail:
+	/* free resource */
+free_sring:
+	free_page((unsigned long) sring);
+	scsifront_free(info);
+
+	return err;
+}
+
+static int scsifront_init_ring(struct scsifront_info *info)
+{
+	struct xenbus_device *dev = info->dev;
+	struct xenbus_transaction xbt;
+	int err;
+
+	DPRINTK("");
+
+	err = scsifront_alloc_ring(info);
+	if (err)
+		return err;
+	DPRINTK("%lu %u\n", info->ring_ref, info->evtchn);
+
+again:
+	err = xenbus_transaction_start(&xbt);
+	if (err) {
+		xenbus_dev_fatal(dev, err, "starting transaction");
+	}
+
+	err = xenbus_printf(xbt, dev->nodename, "ring-ref", "%lu",
+				info->ring_ref);
+	if (err) {
+		xenbus_dev_fatal(dev, err, "%s", "writing ring-ref");
+		goto fail;
+	}
+
+#ifdef XEN303 /*FJVMIO xen3.0.5*/
+	err = xenbus_printf(xbt, dev->nodename, "event-channel", "%u",
+				info->evtchn);
+#else
+	err = xenbus_printf(xbt, dev->nodename, "event-channel", "%u",
+				irq_to_evtchn_port(info->irq));
+#endif
+	if (err) {
+		xenbus_dev_fatal(dev, err, "%s", "writing event-channel");
+		goto fail;
+	}
+
+	err = xenbus_transaction_end(xbt, 0);
+	if (err) {
+		if (err == -EAGAIN)
+			goto again;
+		xenbus_dev_fatal(dev, err, "completing transaction");
+	} else
+		xenbus_switch_state(dev, XenbusStateInitialised);
+
+	return 0;
+fail:
+	xenbus_transaction_end(xbt, 1);
+	/* free resource */
+	scsifront_free(info);
+	
+	return err;
+}
+
+static struct scsi_host_template scsifront_sht = {
+	.module			= THIS_MODULE,
+	.name			= "Xen SCSI frontend driver",
+	.queuecommand		= scsifront_queuecommand,
+	.eh_abort_handler	= scsifront_eh_abort_handler,
+	.cmd_per_lun		= CAN_QUEUE,
+	.can_queue		= CAN_QUEUE,
+	.this_id 		= -1,
+	.sg_tablesize		= SG_TABLESIZE,
+	.use_clustering		= DISABLE_CLUSTERING,
+	.proc_name		= "scsifront",
+};
+
+static int scsifront_connect(struct scsifront_info *info)
+{
+	struct xenbus_device *dev = info->dev;
+	struct Scsi_Host *host = info->host;
+	int err = -ENOMEM;
+
+	DPRINTK("%u\n", dev->state);
+	if (dev->state == XenbusStateConnected)
+		return 0;
+
+	xenbus_switch_state(dev, XenbusStateConnected);
+
+	/* FIXME */
+	host->max_id      = 1;
+	host->max_channel = 0;
+
+	err = scsi_add_host(host, &dev->dev);
+	if (err) {
+		printk("fail to add scsi host %d\n", err);
+		return err;
+	}
+
+	scsi_scan_host(host);
+
+	return 0;
+}
+
+static int scsifront_probe(struct xenbus_device *dev,
+				const struct xenbus_device_id *id)
+{
+	struct Scsi_Host *host;
+	struct scsifront_info *info;
+	int i, err = -ENOMEM;
+
+	host = scsi_host_alloc(&scsifront_sht, sizeof(*info));
+	if (!host) {
+		xenbus_dev_fatal(dev, err, "fail to allocate scsi host");
+		return err;
+	}
+	info = (struct scsifront_info *) host->hostdata;
+	dev->dev.driver_data = info;
+	info->dev  = dev;
+	info->host = host;
+
+	for (i = 0; i < SCSI_RING_SIZE; i++)
+		info->shadow[i].ring_req.rqid = i + 1;
+	info->shadow[SCSI_RING_SIZE - 1].ring_req.rqid = 0x0fffffff;
+
+	err = scsifront_init_ring(info);
+	if (err) {
+		scsi_host_put(host);
+		return err;
+	}
+
+	return 0;
+}
+
+
+static int scsifront_remove(struct xenbus_device *dev)
+{
+	struct scsifront_info *info = dev->dev.driver_data;
+
+	scsifront_free(info);
+	
+	return 0;
+}
+
+static void scsifront_backend_changed(struct xenbus_device *dev,
+					XenbusState backend_state)
+{
+	struct scsifront_info *info = dev->dev.driver_data;
+
+	DPRINTK("%p %u %u\n", dev, dev->state, backend_state);
+
+	switch (backend_state) {
+	case XenbusStateUnknown:
+	case XenbusStateInitialising:
+	case XenbusStateInitWait:
+	case XenbusStateInitialised:
+	case XenbusStateClosed:
+		break;
+
+	case XenbusStateConnected:
+		scsifront_connect(info);
+		break;
+
+	case XenbusStateClosing:
+		break;
+	}
+}
+
+static struct xenbus_device_id scsifront_ids[] = {
+	{ "scsihost" },
+	{ "" }
+};
+
+
+static struct xenbus_driver scsifront_driver = {
+	.name			= "scsihost",
+	.owner			= THIS_MODULE,
+	.ids			= scsifront_ids,
+	.probe			= scsifront_probe,
+	.remove			= scsifront_remove,
+/* 	.resume			= scsifront_resume, */
+	.otherend_changed	= scsifront_backend_changed,
+};
+
+static int __init scsifront_init(void)
+{
+	int err;
+
+	if (!is_running_on_xen())
+		return -ENODEV;
+
+	err = xenbus_register_frontend(&scsifront_driver);
+
+	return err;
+}
+
+static void scsifront_exit(void)
+{
+	xenbus_unregister_driver(&scsifront_driver);
+}
+
+module_init(scsifront_init);
+module_exit(scsifront_exit);
+
+MODULE_DESCRIPTION("Xen SCSI frontend driver");
+MODULE_LICENSE("GPL");

[-- Attachment #4: makeconfig.patch --]
[-- Type: application/octet-stream, Size: 4236 bytes --]

# HG changeset patch
# User t.horikoshi@jp.fujitsu.com
# Date 1179302318 -32400
# Node ID 7a02a4017780fc2d97232572d9e7d88a02ca9b9c
# Parent  38b476ccc82814fb55e3a36f88c995e353cf63dc
modiy config

Signed-off-by: Tomonari Horikoshi <t.horikoshi@jp.fujitsu.com>
Signed-off-by: Tsunehisa Doi <Doi.Tsunehisa@jp.fujitsu.com>
Signed-off-by: Jun Kamada <kama@jp.fujitsu.com>
Signed-off-by: Akira Hayakawa <hayakawa.akira@jp.fujitsu.com>

diff -r 38b476ccc828 -r 7a02a4017780 buildconfigs/linux-defconfig_xen_ia64
--- a/buildconfigs/linux-defconfig_xen_ia64	Wed May 16 16:56:01 2007 +0900
+++ b/buildconfigs/linux-defconfig_xen_ia64	Wed May 16 16:58:38 2007 +0900
@@ -1630,6 +1630,7 @@ CONFIG_XEN_XENBUS_DEV=y
 CONFIG_XEN_XENBUS_DEV=y
 CONFIG_XEN_BACKEND=y
 CONFIG_XEN_BLKDEV_BACKEND=y
+CONFIG_XEN_SCSI_BACKEND=y
 CONFIG_XEN_BLKDEV_TAP=y
 CONFIG_XEN_NETDEV_BACKEND=y
 # CONFIG_XEN_NETDEV_PIPELINED_TRANSMITTER is not set
@@ -1641,6 +1642,7 @@ CONFIG_XEN_PCIDEV_BACKEND_SLOT=y
 # CONFIG_XEN_PCIDEV_BE_DEBUG is not set
 CONFIG_XEN_TPMDEV_BACKEND=m
 CONFIG_XEN_BLKDEV_FRONTEND=y
+CONFIG_XEN_SCSI_FRONTEND=m
 CONFIG_XEN_NETDEV_FRONTEND=y
 CONFIG_XEN_FRAMEBUFFER=y
 CONFIG_XEN_KEYBOARD=y
diff -r 38b476ccc828 -r 7a02a4017780 buildconfigs/linux-defconfig_xen_x86_32
--- a/buildconfigs/linux-defconfig_xen_x86_32	Wed May 16 16:56:01 2007 +0900
+++ b/buildconfigs/linux-defconfig_xen_x86_32	Wed May 16 16:58:38 2007 +0900
@@ -1083,13 +1083,13 @@ CONFIG_IDEDMA_AUTO=y
 # SCSI device support
 #
 CONFIG_RAID_ATTRS=m
-CONFIG_SCSI=m
+CONFIG_SCSI=y
 CONFIG_SCSI_PROC_FS=y
 
 #
 # SCSI support type (disk, tape, CD-ROM)
 #
-CONFIG_BLK_DEV_SD=m
+CONFIG_BLK_DEV_SD=y
 CONFIG_CHR_DEV_ST=m
 CONFIG_CHR_DEV_OSST=m
 CONFIG_BLK_DEV_SR=m
@@ -3253,6 +3253,7 @@ CONFIG_XEN_XENBUS_DEV=y
 CONFIG_XEN_XENBUS_DEV=y
 CONFIG_XEN_BACKEND=y
 CONFIG_XEN_BLKDEV_BACKEND=y
+CONFIG_XEN_SCSI_BACKEND=y
 CONFIG_XEN_BLKDEV_TAP=y
 CONFIG_XEN_NETDEV_BACKEND=y
 # CONFIG_XEN_NETDEV_PIPELINED_TRANSMITTER is not set
@@ -3264,6 +3265,7 @@ CONFIG_XEN_PCIDEV_BACKEND_VPCI=y
 # CONFIG_XEN_PCIDEV_BE_DEBUG is not set
 CONFIG_XEN_TPMDEV_BACKEND=m
 CONFIG_XEN_BLKDEV_FRONTEND=y
+CONFIG_XEN_SCSI_FRONTEND=m
 CONFIG_XEN_NETDEV_FRONTEND=y
 CONFIG_XEN_FRAMEBUFFER=y
 CONFIG_XEN_KEYBOARD=y
diff -r 38b476ccc828 -r 7a02a4017780 linux-2.6-xen-sparse/drivers/xen/Kconfig
--- a/linux-2.6-xen-sparse/drivers/xen/Kconfig	Wed May 16 16:56:01 2007 +0900
+++ b/linux-2.6-xen-sparse/drivers/xen/Kconfig	Wed May 16 16:58:38 2007 +0900
@@ -53,6 +53,14 @@ config XEN_BLKDEV_BACKEND
 	  The block-device backend driver allows the kernel to export its
 	  block devices to other guests via a high-performance shared-memory
 	  interface.
+
+config XEN_SCSI_BACKEND
+       tristate "SCSI backend driver"
+       depends on XEN_BACKEND
+       default y
+       help
+         The SCSI backend driver allows the kernel to export its SCSI HBAs
+         to other guests via a high-performance shared-memory interface.
 
 config XEN_BLKDEV_TAP
 	tristate "Block-device tap backend driver"
@@ -162,6 +170,14 @@ config XEN_BLKDEV_FRONTEND
 	  dedicated device-driver domain, or your master control domain
 	  (domain 0), then you almost certainly want to say Y here.
 
+config XEN_SCSI_FRONTEND
+	tristate "SCSI frontend driver"
+	depends on XEN && SCSI
+	default y
+	help
+	  The SCSI frontend driver allows the kernel to access SCSI HBAs
+	  within another guest OS.
+
 config XEN_NETDEV_FRONTEND
 	tristate "Network-device frontend driver"
 	depends on XEN && NET
diff -r 38b476ccc828 -r 7a02a4017780 linux-2.6-xen-sparse/drivers/xen/Makefile
--- a/linux-2.6-xen-sparse/drivers/xen/Makefile	Wed May 16 16:56:01 2007 +0900
+++ b/linux-2.6-xen-sparse/drivers/xen/Makefile	Wed May 16 16:58:38 2007 +0900
@@ -9,10 +9,12 @@ obj-y	+= char/
 
 obj-y	+= util.o
 obj-$(CONFIG_XEN_BLKDEV_BACKEND)	+= blkback/
+obj-$(CONFIG_XEN_SCSI_BACKEND)		+= scsiback/
 obj-$(CONFIG_XEN_BLKDEV_TAP)		+= blktap/
 obj-$(CONFIG_XEN_NETDEV_BACKEND)	+= netback/
 obj-$(CONFIG_XEN_TPMDEV_BACKEND)	+= tpmback/
 obj-$(CONFIG_XEN_BLKDEV_FRONTEND)	+= blkfront/
+obj-$(CONFIG_XEN_SCSI_FRONTEND)		+= scsifront/
 obj-$(CONFIG_XEN_NETDEV_FRONTEND)	+= netfront/
 obj-$(CONFIG_XEN_PCIDEV_BACKEND)	+= pciback/
 obj-$(CONFIG_XEN_PCIDEV_FRONTEND)	+= pcifront/

[-- Attachment #5: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [RFC] pv-scsi driver (scsiback/scsifront)

[James Smart]

Tomonari Horikoshi wrote:
> Hi all.
> 
>  We developped a pv-scsi driver that we refered Fujita-san's scsi-driver
>  and blkback.
>  (see, http://www.xensource.com/files/xensummit_4/Xen_Summit_8_Matsumoto.pdf)
> 
>  The pv-scsi driver's feature is as follow:
>   * Guest has dedicated SCSI-HBAs of Dom0.
>   * Guest can send scsi_cdb to the HBAs.
>   * Guest recognises the HBAs from hostno of xenstore.
> 
>  Currentlly, We are developping FC version based on this.
> 
> * Future work:
>   * implement python code
>   * performance tunning
>   * attach, detach
>   * suspend, resume
> 
> * We are wondering about:
>  * We used "scsihost" as xenstore nodename. Is it suitable?
>  * We consider about configfile format...  
>    scsihost = ['fc,0', 'scsi,1', 'type,num']
>                type = "fc" or "scsi"
>                num  = scsi host number on Dom0
I would expect "fc" does not need to be specified, unless there
is FC-isms exposed to the guest.

>    Do you have any comment?
>  * We have no idea how to implement suspend/resume feature.
>    ex. Physical HBA mapping for resumed guest.
>        Pending I/O.
>        The WWN within FC mode for resumed guest.
The WWN is a whole different issue - and I'm going to want to make
sure that whatever you do here is consistent with FC NPIV virtual
ports instantiated in Dom 0. See:
http://marc.info/?l=linux-scsi&m=117768770720886&w=2

>        Influence of migration.
>        ...
> 
>   Could you suggest to us about this?
> 
> 
> 
> Best regards,
> Tomonari Horikoshi
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: [RFC] pv-scsi driver (scsiback/scsifront)

[Ian Pratt]

>  We developped a pv-scsi driver that we refered Fujita-san's
scsi-driver
>  and blkback.
>  (see,
> http://www.xensource.com/files/xensummit_4/Xen_Summit_8_Matsumoto.pdf)

This is good work, and we'd certainly like to get it polished and in
mainline -- thanks! 

>  The pv-scsi driver's feature is as follow:
>   * Guest has dedicated SCSI-HBAs of Dom0.

Is it really the case that you must dedicate a HBA to the guest? Surely
we can extend it to enable an individual LUN to be mapped through to a
guest, translating the host:bus:id etc accordingly?

>  * We consider about configfile format...
>    scsihost = ['fc,0', 'scsi,1', 'type,num']
>                type = "fc" or "scsi"

Why do you need to select between fc and scsi?

>  * We have no idea how to implement suspend/resume feature.
>    ex. Physical HBA mapping for resumed guest.
>        Pending I/O.
>        The WWN within FC mode for resumed guest.
>        Influence of migration.
>
>   Could you suggest to us about this?

The blkfront/back code is obviously a good crib for this. Basically, the
front end driver needs to store enough information to be able to reissue
any uncompleted requests across a migration. This is accomplished with a
'shadow ring'. The frontend needs to be capable of reconnecting if the
backend goes out of state connected, and then reissue the requests.  For
migration write-after-write safety, the backend shouldn't close until
all outstanding requests have either been completed or aborted.

One other thing I notice is that you've kept the blk ring protocol's 11
page limit per request. The blkring kinda gets away with this because
(at least in principle) we could merge consecutive requests in blkback
to create larger IOs. I guess we could do that with scsi requests too,
but I'd feel more comfortable if we didn't mess with the request stream
that the guest is generating. We probably need to make the number of
pages in an SG list variable, and hence have variable sized requests
across the ring. We should defintiely make the ring multi-page too.

What do you think?

Thanks,
Ian

Re: [RFC] pv-scsi driver (scsiback/scsifront)

[akira hayakawa]

Thank you for your comment.

I work in same group Tomonari Horikoshi works.

>Tomonari Horikoshi wrote:
>> Hi all.
>> 
>>  We developped a pv-scsi driver that we refered Fujita-san's scsi-driver
>>  and blkback.
>>  (see, http://www.xensource.com/files/xensummit_4/Xen_Summit_8_Matsumoto.
>> pdf)
>> 
>>  The pv-scsi driver's feature is as follow:
>>   * Guest has dedicated SCSI-HBAs of Dom0.
>>   * Guest can send scsi_cdb to the HBAs.
>>   * Guest recognises the HBAs from hostno of xenstore.
>> 
>>  Currentlly, We are developping FC version based on this.
>> 
>> * Future work:
>>   * implement python code
>>   * performance tunning
>>   * attach, detach
>>   * suspend, resume
>> 
>> * We are wondering about:
>>  * We used "scsihost" as xenstore nodename. Is it suitable?
>>  * We consider about configfile format...  
>>    scsihost = ['fc,0', 'scsi,1', 'type,num']
>>                type = "fc" or "scsi"
>>                num  = scsi host number on Dom0
>I would expect "fc" does not need to be specified, unless there
>is FC-isms exposed to the guest.

We want to use SAN management software on guest OS. The software
works on native(no VM) linux. So we think it is necesarry to 
have guest OS shown whether HBA card is FC or SCSI in the same
way of native linux.

>>    Do you have any comment?
>>  * We have no idea how to implement suspend/resume feature.
>>    ex. Physical HBA mapping for resumed guest.
>>        Pending I/O.
>>        The WWN within FC mode for resumed guest.
>The WWN is a whole different issue - and I'm going to want to make
>sure that whatever you do here is consistent with FC NPIV virtual
>ports instantiated in Dom 0. See:
>http://marc.info/?l=linux-scsi&m=117768770720886&w=2

We think whether WWN is same value or not when a guest resumes again is
unknown because the WWN may be already used by another guest. 

Best Regards,
Akira Hayakawa

Re: [RFC] pv-scsi driver (scsiback/scsifront)

[James Smart]

>> I would expect "fc" does not need to be specified, unless there
>> is FC-isms exposed to the guest.
> 
> We want to use SAN management software on guest OS. The software
> works on native(no VM) linux. So we think it is necesarry to 
> have guest OS shown whether HBA card is FC or SCSI in the same
> way of native linux.

Well - depends on what/how your san mgmt works. If it's straight scsi,
then it would be fine - but you can't talk to anything non-scsi and
not enumerated by the hba.  If it's layered on hbaapi, it does mean
you want to talk FC, not just scsi, and now things change significantly.

> 
>>>    Do you have any comment?
>>>  * We have no idea how to implement suspend/resume feature.
>>>    ex. Physical HBA mapping for resumed guest.
>>>        Pending I/O.
>>>        The WWN within FC mode for resumed guest.
>> The WWN is a whole different issue - and I'm going to want to make
>> sure that whatever you do here is consistent with FC NPIV virtual
>> ports instantiated in Dom 0. See:
>> http://marc.info/?l=linux-scsi&m=117768770720886&w=2
> 
> We think whether WWN is same value or not when a guest resumes again is
> unknown because the WWN may be already used by another guest. 

This confuses me greatly. WWN's are how FC ports are known - which controls
their SAN visibility and device access.  If it's changing for the VM, unless
you have everything seeing everything (i.e. no SAN zoning or lun masking,
which is very very rare in a production environment) then whether you see
your storage is questionable. For this reason, regular NPIV will be adding
the WWNs as a resource of the guest, much like the ethernet MAC addresses.
And, if it is bound to the guest, it matches the model needed for
suspend/resume, although there are challenges for discovery and enumeration.

Additionally, I certainly hope you are keeping far more control on how WWN's
are allocated and used. There is that small part about uniqueness that has
to be maintained or the fabric will show very nasty issues.

-- james

RE: [RFC] pv-scsi driver (scsiback/scsifront)

[akira hayakawa]

Thank you for your comment.
I'm sorry for delaying response.

I work in same group Tomonari Horikoshi works.

>>  We developped a pv-scsi driver that we refered Fujita-san's
>scsi-driver
>>  and blkback.
>>  (see,
>> http://www.xensource.com/files/xensummit_4/Xen_Summit_8_Matsumoto.pdf)
>
>This is good work, and we'd certainly like to get it polished and in
>mainline -- thanks! 
>
>>  The pv-scsi driver's feature is as follow:
>>   * Guest has dedicated SCSI-HBAs of Dom0.
>
>Is it really the case that you must dedicate a HBA to the guest? 

Yes, and we are planning to use NPIV function so that more than one
guest can use a HBA.

>Surely we can extend it to enable an individual LUN to be mapped
>through to a guest, translating the host:bus:id etc accordingly?
>
>>  * We consider about configfile format...
>>    scsihost = ['fc,0', 'scsi,1', 'type,num']
>>                type = "fc" or "scsi"
>
>Why do you need to select between fc and scsi?

Please refer "Re: [RFC] pv-scsi driver(scsiback/scsifront)"(respose
to James.Smart@Emulex.Com.).

>>  * We have no idea how to implement suspend/resume feature.
>>    ex. Physical HBA mapping for resumed guest.
>>        Pending I/O.
>>        The WWN within FC mode for resumed guest.
>>        Influence of migration.
>>
>>   Could you suggest to us about this?
>
>The blkfront/back code is obviously a good crib for this. Basically, the
>front end driver needs to store enough information to be able to reissue
>any uncompleted requests across a migration. This is accomplished with a
>'shadow ring'. The frontend needs to be capable of reconnecting if the
>backend goes out of state connected, and then reissue the requests.  For
>migration write-after-write safety, the backend shouldn't close until
>all outstanding requests have either been completed or aborted.

Thank you so much, your advise is very helpful to us.

>One other thing I notice is that you've kept the blk ring protocol's 11
>page limit per request. The blkring kinda gets away with this because
>(at least in principle) we could merge consecutive requests in blkback
>to create larger IOs. I guess we could do that with scsi requests too,
>but I'd feel more comfortable if we didn't mess with the request stream
>that the guest is generating. We probably need to make the number of
>pages in an SG list variable, and hence have variable sized requests
>across the ring. We should defintiely make the ring multi-page too.
>
>What do you think?

I agree with you. We try to defintiely make the ring multi-page.

>Thanks,
>Ian

Best Regards,
Aki