From mboxrd@z Thu Jan 1 00:00:00 1970 From: mlin@kernel.org (Ming Lin) Date: Thu, 10 Sep 2015 10:28:18 -0700 Subject: [RFC PATCH 0/2] virtio nvme In-Reply-To: References: <1441864112-12765-1-git-send-email-mlin@kernel.org> Message-ID: <1441906098.18716.21.camel@ssi> On Thu, 2015-09-10@15:38 +0100, Stefan Hajnoczi wrote: > On Thu, Sep 10, 2015@6:48 AM, Ming Lin wrote: > > These 2 patches added virtio-nvme to kernel and qemu, > > basically modified from virtio-blk and nvme code. > > > > As title said, request for your comments. > > > > Play it in Qemu with: > > -drive file=disk.img,format=raw,if=none,id=D22 \ > > -device virtio-nvme-pci,drive=D22,serial=1234,num_queues=4 > > > > The goal is to have a full NVMe stack from VM guest(virtio-nvme) > > to host(vhost_nvme) to LIO NVMe-over-fabrics target. > > Why is a virtio-nvme guest device needed? I guess there must either > be NVMe-only features that you want to pass through, or you think the > performance will be significantly better than virtio-blk/virtio-scsi? It simply passes through NVMe commands. Right now performance is poor. Performance tunning is on my todo list. It should be as good as virtio-blk/virtio-scsi. > > At first glance it seems like the virtio_nvme guest driver is just > another block driver like virtio_blk, so I'm not clear why a > virtio-nvme device makes sense. I think the future "LIO NVMe target" only speaks NVMe protocol. Nick(CCed), could you correct me if I'm wrong? For SCSI stack, we have: virtio-scsi(guest) tcm_vhost(or vhost_scsi, host) LIO-scsi-target For NVMe stack, we'll have similar components: virtio-nvme(guest) vhost_nvme(host) LIO-NVMe-target > > > Now there are lots of duplicated code with linux/nvme-core.c and qemu/nvme.c. > > The ideal result is to have a multi level NVMe stack(similar as SCSI). > > So we can re-use the nvme code, for example > > > > .-------------------------. > > | NVMe device register | > > Upper level | NVMe protocol process | > > | | > > '-------------------------' > > > > > > > > .-----------. .-----------. .------------------. > > Lower level | PCIe | | VIRTIO | |NVMe over Fabrics | > > | | | | |initiator | > > '-----------' '-----------' '------------------' > > You mentioned LIO and SCSI. How will NVMe over Fabrics be integrated > into LIO? If it is mapped to SCSI then using virtio_scsi in the guest > and tcm_vhost should work. I think it's not mapped to SCSI. Nick, would you share more here? > > Please also post virtio draft specifications documenting the virtio device. I'll do this later. > > Stefan From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ming Lin Subject: Re: [RFC PATCH 0/2] virtio nvme Date: Thu, 10 Sep 2015 10:28:18 -0700 Message-ID: <1441906098.18716.21.camel@ssi> References: <1441864112-12765-1-git-send-email-mlin@kernel.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org To: Stefan Hajnoczi Cc: Christoph Hellwig , linux-nvme@lists.infradead.org, Linux Virtualization List-Id: virtualization@lists.linuxfoundation.org On Thu, 2015-09-10 at 15:38 +0100, Stefan Hajnoczi wrote: > On Thu, Sep 10, 2015 at 6:48 AM, Ming Lin wrote: > > These 2 patches added virtio-nvme to kernel and qemu, > > basically modified from virtio-blk and nvme code. > > > > As title said, request for your comments. > > > > Play it in Qemu with: > > -drive file=disk.img,format=raw,if=none,id=D22 \ > > -device virtio-nvme-pci,drive=D22,serial=1234,num_queues=4 > > > > The goal is to have a full NVMe stack from VM guest(virtio-nvme) > > to host(vhost_nvme) to LIO NVMe-over-fabrics target. > > Why is a virtio-nvme guest device needed? I guess there must either > be NVMe-only features that you want to pass through, or you think the > performance will be significantly better than virtio-blk/virtio-scsi? It simply passes through NVMe commands. Right now performance is poor. Performance tunning is on my todo list. It should be as good as virtio-blk/virtio-scsi. > > At first glance it seems like the virtio_nvme guest driver is just > another block driver like virtio_blk, so I'm not clear why a > virtio-nvme device makes sense. I think the future "LIO NVMe target" only speaks NVMe protocol. Nick(CCed), could you correct me if I'm wrong? For SCSI stack, we have: virtio-scsi(guest) tcm_vhost(or vhost_scsi, host) LIO-scsi-target For NVMe stack, we'll have similar components: virtio-nvme(guest) vhost_nvme(host) LIO-NVMe-target > > > Now there are lots of duplicated code with linux/nvme-core.c and qemu/nvme.c. > > The ideal result is to have a multi level NVMe stack(similar as SCSI). > > So we can re-use the nvme code, for example > > > > .-------------------------. > > | NVMe device register | > > Upper level | NVMe protocol process | > > | | > > '-------------------------' > > > > > > > > .-----------. .-----------. .------------------. > > Lower level | PCIe | | VIRTIO | |NVMe over Fabrics | > > | | | | |initiator | > > '-----------' '-----------' '------------------' > > You mentioned LIO and SCSI. How will NVMe over Fabrics be integrated > into LIO? If it is mapped to SCSI then using virtio_scsi in the guest > and tcm_vhost should work. I think it's not mapped to SCSI. Nick, would you share more here? > > Please also post virtio draft specifications documenting the virtio device. I'll do this later. > > Stefan