From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============5027344726278236170==" MIME-Version: 1.0 From: Walker, Benjamin Subject: Re: [SPDK] Bdev claim and single writer semantics Date: Thu, 26 Apr 2018 23:26:33 +0000 Message-ID: <1524785192.2609.14.camel@intel.com> In-Reply-To: CANvN+e=8pD0y1m=WoJwHPCp0K0Anh2+-X79t0U1xpXTDTV2BKA@mail.gmail.com List-ID: To: spdk@lists.01.org --===============5027344726278236170== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Thu, 2018-04-26 at 15:25 +0300, Andrey Kuzmin wrote: > Hi all, > = > while looking at the block device subsystem code, I noticed bdev > claims being used for two purposes at the same time. First (and a > primary one, as far as I understood it) is a perfectly reasonable idea > to provide the bdev subsystem with dependency information so that > block devices could be brought up and shut down in the proper order > (which doesn't seem to be in place yet, though). > = > On the second hand, claims are somewhat counter-intuitively also used > to ensure single writer semantics where only a module that has claimed > the block device has write access. While it may be considered a safety > measure of sorts, it doesn't go well with the case where a > higher-level application is interested in shared access to the > underlying block device(s), in particular when there are > application-level means to ensure access consistency under multiple > writers scenario. > = > While adding support for the shared access semantics seems to be > pretty straightforward, I thought it reasonable to bring the issue up > here first, looking for insights, comments, and objections. Please let > me know if there are any or I can give a shot to a shared access > patch. Your analysis is all correct. There are two access control mechanisms in the bdev layer for different purposes; claiming, which is for stacking bdevs together to make I/O pipelines, and descriptors, which is standard read/wri= te access control for consumers of bdevs. We thought it made the most sense to allow write access to a bdev from only a single module for safety reasons. That's actually a fairly narrow limitation and I'm not sure it is precludin= g you from doing what you are looking to do. The rule, as it is written today, is that if a bdev isn't claimed by a bdev module (i.e. it doesn't have a bdev stacked on top of it), then multiple wr= ite descriptors can be used. If it is claimed by a bdev module (i.e. another bd= ev is stacked on top of it), then other consumers may only open it as read only. Is your use case that you have some stack of bdevs, and then at some point = your application wants to write to one of the bdevs that's not at the top of the stack? As a concrete example, this would be like having a RAID volume that = you normally write to, but at some point your application bypasses the RAID log= ic and writes directly to one of the disks that back it. That sounds fishy to = me, but I'm open to legitimate use cases here. If you application wants to use a bdev that is at the top of a stack from multiple places, then you should be able to open multiple write descriptors today. Just don't have your application claim the bdev. We've tried to make this as clear as possible, but it could really use some better documentation at a minimum. And discussion about how best to present these concepts to users and make it easier to use is always very welcome. L= et me know what your thoughts are. > = > Best regards, > Andrey > _______________________________________________ > SPDK mailing list > SPDK(a)lists.01.org > https://lists.01.org/mailman/listinfo/spdk --===============5027344726278236170==--