All of lore.kernel.org
 help / color / mirror / Atom feed
From: Johannes Thumshirn <jthumshirn@suse.de>
To: Jack Wang <jinpu.wang@profitbricks.com>
Cc: linux-block@vger.kernel.org, linux-rdma@vger.kernel.org,
	dledford@redhat.com, axboe@kernel.dk, hch@lst.de,
	mail@fholler.de, Milind.dumbare@gmail.com,
	yun.wang@profitbricks.com
Subject: Re: [RFC PATCH 00/28] INFINIBAND NETWORK BLOCK DEVICE (IBNBD)
Date: Fri, 24 Mar 2017 13:15:26 +0100	[thread overview]
Message-ID: <20170324121526.GF3571@linux-x5ow.site> (raw)
In-Reply-To: <1490352343-20075-1-git-send-email-jinpu.wangl@profitbricks.com>

On Fri, Mar 24, 2017 at 11:45:15AM +0100, Jack Wang wrote:
> From: Jack Wang <jinpu.wang@profitbricks.com>
> 
> This series introduces IBNBD/IBTRS kernel modules.
> 
> IBNBD (InfiniBand network block device) allows for an RDMA transfer of block IO
> over InfiniBand network. The driver presents itself as a block device on client
> side and transmits the block requests in a zero-copy fashion to the server-side
> via InfiniBand. The server part of the driver converts the incoming buffers back
> into BIOs and hands them down to the underlying block device. As soon as IO
> responses come back from the drive, they are being transmitted back to the
> client.
> 
> We design and implement this solution based on our need for Cloud Computing,
> the key features are:
> - High throughput and low latency due to:
> 1) Only two rdma messages per IO
> 2) Simplified client side server memory management
> 3) Eliminated SCSI sublayer
> - Simple configuration and handling
> 1) Server side is completely passive: volumes do not need to be
> explicitly exported
> 2) Only IB port GID and device path needed on client side to map
> a block device
> 3) A device can be remapped automatically i.e. after storage
> reboot
> - Pinning of IO-related processing to the CPU of the producer
> 
> For usage please refer to Documentation/IBNBD.txt in later patch.
> My colleague Danil Kpnis presents IBNBD in Vault-2017 about our design/feature/
> tradeoff/performance:
> 
> http://events.linuxfoundation.org/sites/events/files/slides/IBNBD-Vault-2017.pdf
> 

Hi Jack,

Sorry to ask (I haven't attented the Vault presentation) but why can't you use
NVMe over Fabrics in your environment? From what I see in your presentation
and cover letter, it provides all you need and is in fact a standard Linux and
Windows already have implemented.

Thanks,
	Johannes
-- 
Johannes Thumshirn                                          Storage
jthumshirn@suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N�rnberg
GF: Felix Imend�rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N�rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850

WARNING: multiple messages have this Message-ID (diff)
From: Johannes Thumshirn <jthumshirn-l3A5Bk7waGM@public.gmane.org>
To: Jack Wang <jinpu.wang-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
Cc: linux-block-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org,
	axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org,
	hch-jcswGhMUV9g@public.gmane.org,
	mail-99BIx50xQYGELgA04lAiVw@public.gmane.org,
	Milind.dumbare-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org,
	yun.wang-EIkl63zCoXaH+58JC4qpiA@public.gmane.org
Subject: Re: [RFC PATCH 00/28] INFINIBAND NETWORK BLOCK DEVICE (IBNBD)
Date: Fri, 24 Mar 2017 13:15:26 +0100	[thread overview]
Message-ID: <20170324121526.GF3571@linux-x5ow.site> (raw)
In-Reply-To: <1490352343-20075-1-git-send-email-jinpu.wangl-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>

On Fri, Mar 24, 2017 at 11:45:15AM +0100, Jack Wang wrote:
> From: Jack Wang <jinpu.wang-EIkl63zCoXaH+58JC4qpiA@public.gmane.org>
> 
> This series introduces IBNBD/IBTRS kernel modules.
> 
> IBNBD (InfiniBand network block device) allows for an RDMA transfer of block IO
> over InfiniBand network. The driver presents itself as a block device on client
> side and transmits the block requests in a zero-copy fashion to the server-side
> via InfiniBand. The server part of the driver converts the incoming buffers back
> into BIOs and hands them down to the underlying block device. As soon as IO
> responses come back from the drive, they are being transmitted back to the
> client.
> 
> We design and implement this solution based on our need for Cloud Computing,
> the key features are:
> - High throughput and low latency due to:
> 1) Only two rdma messages per IO
> 2) Simplified client side server memory management
> 3) Eliminated SCSI sublayer
> - Simple configuration and handling
> 1) Server side is completely passive: volumes do not need to be
> explicitly exported
> 2) Only IB port GID and device path needed on client side to map
> a block device
> 3) A device can be remapped automatically i.e. after storage
> reboot
> - Pinning of IO-related processing to the CPU of the producer
> 
> For usage please refer to Documentation/IBNBD.txt in later patch.
> My colleague Danil Kpnis presents IBNBD in Vault-2017 about our design/feature/
> tradeoff/performance:
> 
> http://events.linuxfoundation.org/sites/events/files/slides/IBNBD-Vault-2017.pdf
> 

Hi Jack,

Sorry to ask (I haven't attented the Vault presentation) but why can't you use
NVMe over Fabrics in your environment? From what I see in your presentation
and cover letter, it provides all you need and is in fact a standard Linux and
Windows already have implemented.

Thanks,
	Johannes
-- 
Johannes Thumshirn                                          Storage
jthumshirn-l3A5Bk7waGM@public.gmane.org                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

  parent reply	other threads:[~2017-03-24 12:15 UTC|newest]

Thread overview: 87+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-03-24 10:45 [RFC PATCH 00/28] INFINIBAND NETWORK BLOCK DEVICE (IBNBD) Jack Wang
2017-03-24 10:45 ` Jack Wang
2017-03-24 10:45 ` [PATCH 01/28] ibtrs: add header shared between ibtrs_client and ibtrs_server Jack Wang
2017-03-24 10:45   ` Jack Wang
2017-03-24 12:35   ` Johannes Thumshirn
2017-03-24 12:35     ` Johannes Thumshirn
2017-03-24 12:54     ` Jinpu Wang
2017-03-24 12:54       ` Jinpu Wang
2017-03-24 14:31       ` Johannes Thumshirn
2017-03-24 14:31         ` Johannes Thumshirn
2017-03-24 14:35         ` Jinpu Wang
2017-03-24 14:35           ` Jinpu Wang
2017-03-24 10:45 ` [PATCH 02/28] ibtrs: add header for log MICROs " Jack Wang
2017-03-24 10:45 ` [PATCH 03/28] ibtrs_lib: add common functions shared by client and server Jack Wang
2017-03-24 10:45 ` [PATCH 04/28] ibtrs_clt: add header file for exported interface Jack Wang
2017-03-24 10:45 ` [PATCH 05/28] ibtrs_clt: main functionality of ibtrs_client Jack Wang
2017-03-24 10:45   ` Jack Wang
2017-03-24 10:45 ` [PATCH 06/28] ibtrs_clt: add header file shared only in ibtrs_client Jack Wang
2017-03-24 10:45 ` [PATCH 07/28] ibtrs_clt: add files for sysfs interface Jack Wang
2017-03-24 10:45 ` [PATCH 08/28] ibtrs_clt: add Makefile and Kconfig Jack Wang
2017-03-25  5:51   ` kbuild test robot
2017-03-25  5:51     ` kbuild test robot
2017-03-25  6:55   ` kbuild test robot
2017-03-25  6:55     ` kbuild test robot
2017-03-24 10:45 ` [PATCH 09/28] ibtrs_srv: add header file for exported interface Jack Wang
2017-03-24 10:45   ` Jack Wang
2017-03-24 10:45 ` [PATCH 10/28] ibtrs_srv: add main functionality for ibtrs_server Jack Wang
2017-03-24 10:45 ` [PATCH 11/28] ibtrs_srv: add header shared in ibtrs_server Jack Wang
2017-03-24 10:45   ` Jack Wang
2017-03-24 10:45 ` [PATCH 12/28] ibtrs_srv: add sysfs interface Jack Wang
2017-03-24 10:45 ` [PATCH 13/28] ibtrs_srv: add Makefile and Kconfig Jack Wang
2017-03-24 10:45   ` Jack Wang
2017-03-25  7:55   ` kbuild test robot
2017-03-25  7:55     ` kbuild test robot
2017-03-25 10:54   ` kbuild test robot
2017-03-25 10:54     ` kbuild test robot
2017-03-24 10:45 ` [PATCH 14/28] ibnbd: add headers shared by ibnbd_client and ibnbd_server Jack Wang
2017-03-24 10:45 ` [PATCH 15/28] ibnbd: add shared library functions Jack Wang
2017-03-24 10:45   ` Jack Wang
2017-03-24 10:45 ` [PATCH 16/28] ibnbd_clt: add main functionality of ibnbd_client Jack Wang
2017-03-24 10:45   ` Jack Wang
2017-03-24 10:45 ` [PATCH 17/28] ibnbd_clt: add header shared in ibnbd_client Jack Wang
2017-03-24 10:45   ` Jack Wang
2017-03-24 10:45 ` [PATCH 18/28] ibnbd_clt: add sysfs interface Jack Wang
2017-03-24 10:45   ` Jack Wang
2017-03-24 10:45 ` [PATCH 19/28] ibnbd_clt: add log helpers Jack Wang
2017-03-24 10:45 ` [PATCH 20/28] ibnbd_clt: add Makefile and Kconfig Jack Wang
2017-03-24 10:45   ` Jack Wang
2017-03-25  8:38   ` kbuild test robot
2017-03-25  8:38     ` kbuild test robot
2017-03-25 11:17   ` kbuild test robot
2017-03-25 11:17     ` kbuild test robot
2017-03-24 10:45 ` [PATCH 21/28] ibnbd_srv: add header shared in ibnbd_server Jack Wang
2017-03-24 10:45   ` Jack Wang
2017-03-24 10:45 ` [PATCH 22/28] ibnbd_srv: add main functionality Jack Wang
2017-03-24 10:45 ` [PATCH 23/28] ibnbd_srv: add abstraction for submit IO to file or block device Jack Wang
2017-03-24 10:45   ` Jack Wang
2017-03-24 10:45 ` [PATCH 24/28] ibnbd_srv: add log helpers Jack Wang
2017-03-24 10:45 ` [PATCH 25/28] ibnbd_srv: add sysfs interface Jack Wang
2017-03-24 10:45   ` Jack Wang
2017-03-24 10:45 ` [PATCH 26/28] ibnbd_srv: add Makefile and Kconfig Jack Wang
2017-03-25  9:27   ` kbuild test robot
2017-03-25  9:27     ` kbuild test robot
2017-03-24 10:45 ` [PATCH 27/28] ibnbd: add doc for how to use ibnbd and sysfs interface Jack Wang
2017-03-25  7:44   ` kbuild test robot
2017-03-25  7:44     ` kbuild test robot
2017-03-24 10:45 ` [PATCH 28/28] MAINTRAINERS: Add maintainer for IBNBD/IBTRS Jack Wang
2017-03-24 12:15 ` Johannes Thumshirn [this message]
2017-03-24 12:15   ` [RFC PATCH 00/28] INFINIBAND NETWORK BLOCK DEVICE (IBNBD) Johannes Thumshirn
2017-03-24 12:46   ` Jinpu Wang
2017-03-24 12:46     ` Jinpu Wang
2017-03-24 12:48     ` Johannes Thumshirn
2017-03-24 12:48       ` Johannes Thumshirn
2017-03-24 13:31     ` Bart Van Assche
2017-03-24 13:31       ` Bart Van Assche
2017-03-24 14:24       ` Jinpu Wang
2017-03-24 14:24         ` Jinpu Wang
2017-03-24 14:20 ` Steve Wise
2017-03-24 14:20   ` Steve Wise
2017-03-24 14:37   ` Jinpu Wang
2017-03-24 14:37     ` Jinpu Wang
2017-03-27  2:20 ` Sagi Grimberg
2017-03-27  2:20   ` Sagi Grimberg
2017-03-27 10:21   ` Jinpu Wang
2017-03-27 10:21     ` Jinpu Wang
2017-03-28 14:17     ` Roman Penyaev
2017-03-28 14:17       ` Roman Penyaev

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170324121526.GF3571@linux-x5ow.site \
    --to=jthumshirn@suse.de \
    --cc=Milind.dumbare@gmail.com \
    --cc=axboe@kernel.dk \
    --cc=dledford@redhat.com \
    --cc=hch@lst.de \
    --cc=jinpu.wang@profitbricks.com \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=mail@fholler.de \
    --cc=yun.wang@profitbricks.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.