* Re: [PATCH V4 00/13] MD: a caching layer for raid5/6
From: Shaohua Li @ 2015-07-10 5:18 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid, songliubraving, hch, dan.j.williams, Kernel-team
In-Reply-To: <20150710151044.396f9645@noble>
On Fri, Jul 10, 2015 at 03:10:44PM +1000, NeilBrown wrote:
> On Thu, 9 Jul 2015 21:52:43 -0700 Shaohua Li <shli@fb.com> wrote:
>
> > On Fri, Jul 10, 2015 at 02:36:56PM +1000, NeilBrown wrote:
> > > On Thu, 9 Jul 2015 21:08:49 -0700 Shaohua Li <shli@fb.com> wrote:
> > >
>
> > > There is also the issue of what action commits a previous transaction.
> > > I'm not sure what you had. I'm suggesting that each metadata block
> > > commits previous transactions. Is that a close-enough match to what
> > > you had?
> >
> > What did you mean about a transaction? In my implementation, metadata
> > block and followed stripe data/parity consist of an io unit. io units can
> > be finished out of order. but if io unit has flush request (the data has
> > flush/flush bio or metadata is a flush block), the io unit can only
> > start after all previous io units and disk cache flush finish. Such io
> > unit is strictly ordered. The log patch describes this behavior. Does it
> > match?
>
> Yes, a "transaction" is an "io unit". The flushing is the same.
> I just couldn't remember how, when reading the log on restart, you
> determined if a given "io unit" was reliably consistent, or whether it
> should be ignored (having possibly only partially been written).
The metadata block has a checksum for data of the block. data/parity has
checksum stored in metadata block. This way we can know if metadata and
data is consistent.
Thanks,
Shaohua
^ permalink raw reply
* Re: [PATCH V4 00/13] MD: a caching layer for raid5/6
From: NeilBrown @ 2015-07-10 5:10 UTC (permalink / raw)
To: Shaohua Li; +Cc: linux-raid, songliubraving, hch, dan.j.williams, Kernel-team
In-Reply-To: <20150710045225.GA1746743@devbig257.prn2.facebook.com>
On Thu, 9 Jul 2015 21:52:43 -0700 Shaohua Li <shli@fb.com> wrote:
> On Fri, Jul 10, 2015 at 02:36:56PM +1000, NeilBrown wrote:
> > On Thu, 9 Jul 2015 21:08:49 -0700 Shaohua Li <shli@fb.com> wrote:
> >
> > There is also the issue of what action commits a previous transaction.
> > I'm not sure what you had. I'm suggesting that each metadata block
> > commits previous transactions. Is that a close-enough match to what
> > you had?
>
> What did you mean about a transaction? In my implementation, metadata
> block and followed stripe data/parity consist of an io unit. io units can
> be finished out of order. but if io unit has flush request (the data has
> flush/flush bio or metadata is a flush block), the io unit can only
> start after all previous io units and disk cache flush finish. Such io
> unit is strictly ordered. The log patch describes this behavior. Does it
> match?
Yes, a "transaction" is an "io unit". The flushing is the same.
I just couldn't remember how, when reading the log on restart, you
determined if a given "io unit" was reliably consistent, or whether it
should be ignored (having possibly only partially been written).
Thanks,
NeilBrown
^ permalink raw reply
* Re: [PATCH V4 00/13] MD: a caching layer for raid5/6
From: Shaohua Li @ 2015-07-10 4:52 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid, songliubraving, hch, dan.j.williams, Kernel-team
In-Reply-To: <20150710143656.4ee7e647@noble>
On Fri, Jul 10, 2015 at 02:36:56PM +1000, NeilBrown wrote:
> On Thu, 9 Jul 2015 21:08:49 -0700 Shaohua Li <shli@fb.com> wrote:
>
> > On Fri, Jul 10, 2015 at 09:21:19AM +1000, NeilBrown wrote:
> > > On Tue, 7 Jul 2015 22:44:02 -0700 Shaohua Li <shli@fb.com> wrote:
> > >
> > > > On Wed, Jul 08, 2015 at 11:56:36AM +1000, NeilBrown wrote:
> > > > >
> > > > > Hi,
> > > > > I made some time to look at these this morning - sorry for the delay.
> > > > >
> > > > > Having it all broken down with more complete comments helps a lot -
> > > > > thanks.
> > > > >
> > > > > Unfortunately ... it helps confirm that I really don't like this. It
> > > > > seems much more complex that it should be. There certainly are a lot
> > > > > of details that need to be carefully considered, but the result needs
> > > > > to be simpler.
> > > > >
> > > > > A big part of my concern is that you choose to avoid making proper use
> > > > > of the current stripe cache. Your argument is that it is already
> > > > > too complex. That is probably true but I don't think you decrease
> > > > > total complexity by adding extra bits on the side that themselves are
> > > > > complex in completely different ways.
> > > > >
> > > > > I would like to start small and keep it as simple as possible. I
> > > > > think the minimum useful functionality is closing the write-hole.
> > > > > Anything else could be done outside raid5 (e.g. bcache), but the
> > > > > write-hole needs integration. Once that is closed, it may make sense
> > > > > to add more functionality.
> > > > >
> > > > > To do that we need a log, and it was quite sensible for you to put
> > > > > that first in the list of patches. So let's start with that.
> > > > >
> > > > > I would like to propose a single metadata block format. This block
> > > > > includes the general RAID5 parameters (both 'old' and 'new' for when a
> > > > > reshape is happening), it includes linkage to find nearby metadata.
> > > > > It includes a list of the data written after the metadata. And it
> > > > > includes a list of device-addresses of parity blocks which are now safe
> > > > > on the RAID and so any log content is now ignored.
> > > > >
> > > > > The general mechanism for writing to the log is:
> > > > > - collect a list of bios. As each arrives update the metadata
> > > > > block with index information.
> > > > > - when ready (there are various triggers) write the metadata block
> > > > > with FLUSH/FUA and write all the data blocks normally. This
> > > > > metadata block plus data/parity blocks is a 'transaction'.
> > > > >
> > > > > We never trigger a log transaction until all writes for the previous
> > > > > transaction have completed. This means that once the metadata block is
> > > > > safe, all previous data must also be safe. If this imposes too much
> > > > > waiting we could allow a "double-buffer" approach were each metadata
> > > > > block completes the previous-but-one transaction.
> > > > >
> > > > > When all of the previous writes complete we trigger a new log write
> > > > > if there is an outstanding SYNC write, or maybe an outstanding
> > > > > FLUSH/FUA, but otherwise we wait until the metadata block is full, or
> > > > > some amount of time has passed.
> > > > >
> > > > > This one metadata block serves all of the purposes that you
> > > > > identified. It records where data is, it commits previous writes, and
> > > > > records which stripes are being written to RAID and which have been
> > > > > fully committed.
> > > > >
> > > > > With that log in place, we add the code to "hijack ops_run_io" to
> > > > > write all dirty blocks from the stripe_head to the cache. This
> > > > > includes the parity of course. Immediately that data is safe the
> > > > > write requests can be returned and the data is written the the RAID
> > > > > devices.
> > > > >
> > > > > "Recovery" at boot time simply involves:
> > > > > 1. find and load first metadata block
> > > > > 2. load the "next" metadata block. If it doesn't exist, stop.
> > > > > 3. For each parity/data block listed in current block, get the
> > > > > stripe_head and read the block in.
> > > > > 4. For each completed stripe listed, find the stripe_head and
> > > > > invalidate it.
> > > > > 5. Make the "next" metadata block the current block and goto 2
> > > > > 6. Flush out all dirty data in stripe cache.
> > > > >
> > > > >
> > > > > I think that all of this *needs* to use the stripe_cache.
> > > > > Intercepting normal processing between party computation and writing
> > > > > to the RAID devices must involve the stripe_cache.
> > > > >
> > > > > So this is a "simple" start. Once this is written and agreed and
> > > > > tested and debugged, then we can look at the latency-hiding side of
> > > > > the cache. I think this should still use the stripe_cache - just
> > > > > schedule the writes to the log a bit earlier.
> > > > > If a case can be made for a separate cache then I'm not completely
> > > > > closed to the idea but I don't really want to think about it until the
> > > > > write-hole closing code is done.
> > > > >
> > > > > So:
> > > > > [PATCH V4 01/13] MD: add a new disk role to present cache device
> > > > > probably OK
> > > > > [PATCH V4 02/13] raid5: directly use mddev->queue
> > > > > OK
> > > > > [PATCH V4 03/13] raid5: cache log handling
> > > > > strip this down to a bare minimum. It needs:
> > > > > -on-disk metadata block format
> > > > > -two or three active transactions which identify a list of bios
> > > > > that are currently in the stripe cache. As the bio is added,
> > > > > its address info is added to the metadata block.
> > > > > -interface to "add a bio to a transaction" and "record a
> > > > > completed stripe".
> > > > > -timer flush the current transaction if previous one was not
> > > > > empty.
> > > > >
> > > > > I don't think there needs to be a separate superblock in the log.
> > > > > Each device in the array already has an 'md' superblock. Use e.g
> > > > > the 'recovery_offset' field in there (which is per-device and
> > > > > meaningless for a log) to store the address of the most recent
> > > > > metadata blocks at the time the superblock was written. Search
> > > > > forward and backward from there to find whole log. Each metadata
> > > > > block holds everything else that might be useful.
> > > > >
> > > > > [PATCH V4 05/13] raid5: cache reclaim support
> > > > > Just want the "hijack ops_run_io" part of this so that when normal
> > > > > RAID5 processing wants to write a stripe, blocks get diverted to
> > > > > the log first and further processing of the stripe is delayed until
> > > > > those writes complete and the transaction is safe.
> > > > >
> > > > > [PATCH V4 08/13] raid5: cache recovery support
> > > > > Just load all the log into the stripe cache.
> > > > >
> > > > >
> > > > > That should be much more manageable and would be a good start towards
> > > > > getting all the functionality that you want.
> > > > >
> > > > > Feel free to ask questions if I haven't explained things, or if you
> > > > > want me to look over the metadata format or whatever. I tend to answer
> > > > > direct questions more quickly than I answer 100kB patchsets :-)
> > > > Hi,
> > > >
> > > > So we are back to the original point (eg, only fix the write hole issue) after
> > > > 3 months development, which is really disappointed. I don't object just fixing
> > > > the write hole issue as a start, it's simple which is my arguement of first
> > > > post too, and probably a slight change of patch 3 meets the goal. The cache
> > > > does significantly increase complexity, which is well known when we move from
> > > > 'just fix write hole' to 'fix write hole and do cache'. But the code is right
> > > > there, I'm wondering how bad it is. Just because of stripe_cache? I don't see
> > > > any significant technical merit stripe_cache is a must-have here. You said the
> > > > code is complex. Yes, it is, but it's not because of the new data cache. It's a
> > > > simple radix tree and takes less than 20% of the total code. The remaining code
> > > > does what any log device should do (write to log, reclaim when log is full,
> > > > recovery when log crash). Similar things must be done even stripe_cache is
> > > > used. And if we don't use stripe_cache, we don't deeply couple with current
> > > > state machine, the new data cache actually brings flexibility (for example, we
> > > > can choose not using cache, or we can easily put the cache to NVRAM).
> > >
> > > I don't think we are completely back to the start. The product of
> > > recent months isn't just code - it is also your understanding and
> > > expertise. You've explored the problems and understand them. You
> > > could quickly see issues because you have struggled with them already.
> > >
> > > I think the principal of "write one to throw away" is a good one - and
> > > probably one we don't apply often enough. The first time you implement
> > > something you don't really know what you are doing, so the result is
> > > bound to be imperfect. The second time you have a much better idea of
> > > the big picture.
> > >
> > > The real big problem that I have with the current proposal is that I
> > > just don't understand it. Maybe that is my failing. But all the
> > > evidence suggests that I'll end up needing to maintain it, so I *have*
> > > to understand it.
> > >
> > > But let's not assume we are throwing everything away (but equally not
> > > promise that we won't) and start with just the first (non trivial)
> > > patch. The log.
> > >
> > > I think we do want a log. I think you have identified the key
> > > functionality that the log must provide. But I think the current
> > > layout is too complex. This is something that will be persistent on
> > > storage devices, so we want to get it as "right" as we can, early.
> > > It is fairly cheap to throw away a log and start a new one, so fixing
> > > design mistakes isn't too hard. But we do have to support any format
> > > we create indefinitely.
> > > So simple is best. Simple is most extensible.
> > >
> > > As I said: A single metadata block format, with:
> > > - forward/backward linkage - and probably a link to the "first" block
> > > in the log at the time of writing
> > > - array state data - brieifly
> > > - list of data/parity block addresses and maybe checksums
> > > - list of "completed" stripes
> > >
> > > The log is simply these metadata blocks mixed with the data/parity they
> > > they describe.
> > > Each metadata block commits all previous transactions (or maybe all but
> > > the most recent).
> > > The current md superblock contains the address of one of these metadata
> > > blocks.
> > >
> > > Does that seem reasonable? Does that seem simpler than what you had?
> > > Will that meet your perceived need? It meets mine.
> > > If we can do that, I'll apply it and we can move on to the next step.
> >
> > My original layout has 3 types of blocks: super block, metadata block for data/parity,
> > flush block to list completed stripes.
> >
> > Compared to your single metadata block format, your format
> > - delete super block. I'm fine with this. The only useful bits in my
> > super block is log tail and seq. we can overload MD superblock
> > recovery_offset/resync_offset to record seq/log tail.
> > - merge metadata block and flush block to one metadata block. That is
> > the new metadata block can record a mix of data/parity/'completed'
> > stripes. I don't object to this, it's doable, but don't think this
> > makes things simpler.
> >
> > If you insist, we can take this one. The new metadata block can record a
> > mix of data/parity/completed stripes, but it can also just record data
> > or completed stripes. So I'll change the disk format to cater this, but
> > I'll always put 'completed stripes' info into a separate metadata block
> > (that is not mixing with data/parity, but we still just have one
> > metadata type). That way I only need slight change of current code and
> > it works for the new format. We can, if necessary, let the new metadata
> > records any mix of data/parity/completed stripes later. Sounds good?
>
> Yes it does. Having a single sort of metadata block is an important
> part of the goal. How the code actually chooses to use these is a
> separate issue that can change harmlessly.
>
> There is also the issue of what action commits a previous transaction.
> I'm not sure what you had. I'm suggesting that each metadata block
> commits previous transactions. Is that a close-enough match to what
> you had?
What did you mean about a transaction? In my implementation, metadata
block and followed stripe data/parity consist of an io unit. io units can
be finished out of order. but if io unit has flush request (the data has
flush/flush bio or metadata is a flush block), the io unit can only
start after all previous io units and disk cache flush finish. Such io
unit is strictly ordered. The log patch describes this behavior. Does it
match?
Thanks,
Shaohua
^ permalink raw reply
* Re: [PATCH V4 00/13] MD: a caching layer for raid5/6
From: NeilBrown @ 2015-07-10 4:36 UTC (permalink / raw)
To: Shaohua Li; +Cc: linux-raid, songliubraving, hch, dan.j.williams, Kernel-team
In-Reply-To: <20150710040847.GA1408097@devbig257.prn2.facebook.com>
On Thu, 9 Jul 2015 21:08:49 -0700 Shaohua Li <shli@fb.com> wrote:
> On Fri, Jul 10, 2015 at 09:21:19AM +1000, NeilBrown wrote:
> > On Tue, 7 Jul 2015 22:44:02 -0700 Shaohua Li <shli@fb.com> wrote:
> >
> > > On Wed, Jul 08, 2015 at 11:56:36AM +1000, NeilBrown wrote:
> > > >
> > > > Hi,
> > > > I made some time to look at these this morning - sorry for the delay.
> > > >
> > > > Having it all broken down with more complete comments helps a lot -
> > > > thanks.
> > > >
> > > > Unfortunately ... it helps confirm that I really don't like this. It
> > > > seems much more complex that it should be. There certainly are a lot
> > > > of details that need to be carefully considered, but the result needs
> > > > to be simpler.
> > > >
> > > > A big part of my concern is that you choose to avoid making proper use
> > > > of the current stripe cache. Your argument is that it is already
> > > > too complex. That is probably true but I don't think you decrease
> > > > total complexity by adding extra bits on the side that themselves are
> > > > complex in completely different ways.
> > > >
> > > > I would like to start small and keep it as simple as possible. I
> > > > think the minimum useful functionality is closing the write-hole.
> > > > Anything else could be done outside raid5 (e.g. bcache), but the
> > > > write-hole needs integration. Once that is closed, it may make sense
> > > > to add more functionality.
> > > >
> > > > To do that we need a log, and it was quite sensible for you to put
> > > > that first in the list of patches. So let's start with that.
> > > >
> > > > I would like to propose a single metadata block format. This block
> > > > includes the general RAID5 parameters (both 'old' and 'new' for when a
> > > > reshape is happening), it includes linkage to find nearby metadata.
> > > > It includes a list of the data written after the metadata. And it
> > > > includes a list of device-addresses of parity blocks which are now safe
> > > > on the RAID and so any log content is now ignored.
> > > >
> > > > The general mechanism for writing to the log is:
> > > > - collect a list of bios. As each arrives update the metadata
> > > > block with index information.
> > > > - when ready (there are various triggers) write the metadata block
> > > > with FLUSH/FUA and write all the data blocks normally. This
> > > > metadata block plus data/parity blocks is a 'transaction'.
> > > >
> > > > We never trigger a log transaction until all writes for the previous
> > > > transaction have completed. This means that once the metadata block is
> > > > safe, all previous data must also be safe. If this imposes too much
> > > > waiting we could allow a "double-buffer" approach were each metadata
> > > > block completes the previous-but-one transaction.
> > > >
> > > > When all of the previous writes complete we trigger a new log write
> > > > if there is an outstanding SYNC write, or maybe an outstanding
> > > > FLUSH/FUA, but otherwise we wait until the metadata block is full, or
> > > > some amount of time has passed.
> > > >
> > > > This one metadata block serves all of the purposes that you
> > > > identified. It records where data is, it commits previous writes, and
> > > > records which stripes are being written to RAID and which have been
> > > > fully committed.
> > > >
> > > > With that log in place, we add the code to "hijack ops_run_io" to
> > > > write all dirty blocks from the stripe_head to the cache. This
> > > > includes the parity of course. Immediately that data is safe the
> > > > write requests can be returned and the data is written the the RAID
> > > > devices.
> > > >
> > > > "Recovery" at boot time simply involves:
> > > > 1. find and load first metadata block
> > > > 2. load the "next" metadata block. If it doesn't exist, stop.
> > > > 3. For each parity/data block listed in current block, get the
> > > > stripe_head and read the block in.
> > > > 4. For each completed stripe listed, find the stripe_head and
> > > > invalidate it.
> > > > 5. Make the "next" metadata block the current block and goto 2
> > > > 6. Flush out all dirty data in stripe cache.
> > > >
> > > >
> > > > I think that all of this *needs* to use the stripe_cache.
> > > > Intercepting normal processing between party computation and writing
> > > > to the RAID devices must involve the stripe_cache.
> > > >
> > > > So this is a "simple" start. Once this is written and agreed and
> > > > tested and debugged, then we can look at the latency-hiding side of
> > > > the cache. I think this should still use the stripe_cache - just
> > > > schedule the writes to the log a bit earlier.
> > > > If a case can be made for a separate cache then I'm not completely
> > > > closed to the idea but I don't really want to think about it until the
> > > > write-hole closing code is done.
> > > >
> > > > So:
> > > > [PATCH V4 01/13] MD: add a new disk role to present cache device
> > > > probably OK
> > > > [PATCH V4 02/13] raid5: directly use mddev->queue
> > > > OK
> > > > [PATCH V4 03/13] raid5: cache log handling
> > > > strip this down to a bare minimum. It needs:
> > > > -on-disk metadata block format
> > > > -two or three active transactions which identify a list of bios
> > > > that are currently in the stripe cache. As the bio is added,
> > > > its address info is added to the metadata block.
> > > > -interface to "add a bio to a transaction" and "record a
> > > > completed stripe".
> > > > -timer flush the current transaction if previous one was not
> > > > empty.
> > > >
> > > > I don't think there needs to be a separate superblock in the log.
> > > > Each device in the array already has an 'md' superblock. Use e.g
> > > > the 'recovery_offset' field in there (which is per-device and
> > > > meaningless for a log) to store the address of the most recent
> > > > metadata blocks at the time the superblock was written. Search
> > > > forward and backward from there to find whole log. Each metadata
> > > > block holds everything else that might be useful.
> > > >
> > > > [PATCH V4 05/13] raid5: cache reclaim support
> > > > Just want the "hijack ops_run_io" part of this so that when normal
> > > > RAID5 processing wants to write a stripe, blocks get diverted to
> > > > the log first and further processing of the stripe is delayed until
> > > > those writes complete and the transaction is safe.
> > > >
> > > > [PATCH V4 08/13] raid5: cache recovery support
> > > > Just load all the log into the stripe cache.
> > > >
> > > >
> > > > That should be much more manageable and would be a good start towards
> > > > getting all the functionality that you want.
> > > >
> > > > Feel free to ask questions if I haven't explained things, or if you
> > > > want me to look over the metadata format or whatever. I tend to answer
> > > > direct questions more quickly than I answer 100kB patchsets :-)
> > > Hi,
> > >
> > > So we are back to the original point (eg, only fix the write hole issue) after
> > > 3 months development, which is really disappointed. I don't object just fixing
> > > the write hole issue as a start, it's simple which is my arguement of first
> > > post too, and probably a slight change of patch 3 meets the goal. The cache
> > > does significantly increase complexity, which is well known when we move from
> > > 'just fix write hole' to 'fix write hole and do cache'. But the code is right
> > > there, I'm wondering how bad it is. Just because of stripe_cache? I don't see
> > > any significant technical merit stripe_cache is a must-have here. You said the
> > > code is complex. Yes, it is, but it's not because of the new data cache. It's a
> > > simple radix tree and takes less than 20% of the total code. The remaining code
> > > does what any log device should do (write to log, reclaim when log is full,
> > > recovery when log crash). Similar things must be done even stripe_cache is
> > > used. And if we don't use stripe_cache, we don't deeply couple with current
> > > state machine, the new data cache actually brings flexibility (for example, we
> > > can choose not using cache, or we can easily put the cache to NVRAM).
> >
> > I don't think we are completely back to the start. The product of
> > recent months isn't just code - it is also your understanding and
> > expertise. You've explored the problems and understand them. You
> > could quickly see issues because you have struggled with them already.
> >
> > I think the principal of "write one to throw away" is a good one - and
> > probably one we don't apply often enough. The first time you implement
> > something you don't really know what you are doing, so the result is
> > bound to be imperfect. The second time you have a much better idea of
> > the big picture.
> >
> > The real big problem that I have with the current proposal is that I
> > just don't understand it. Maybe that is my failing. But all the
> > evidence suggests that I'll end up needing to maintain it, so I *have*
> > to understand it.
> >
> > But let's not assume we are throwing everything away (but equally not
> > promise that we won't) and start with just the first (non trivial)
> > patch. The log.
> >
> > I think we do want a log. I think you have identified the key
> > functionality that the log must provide. But I think the current
> > layout is too complex. This is something that will be persistent on
> > storage devices, so we want to get it as "right" as we can, early.
> > It is fairly cheap to throw away a log and start a new one, so fixing
> > design mistakes isn't too hard. But we do have to support any format
> > we create indefinitely.
> > So simple is best. Simple is most extensible.
> >
> > As I said: A single metadata block format, with:
> > - forward/backward linkage - and probably a link to the "first" block
> > in the log at the time of writing
> > - array state data - brieifly
> > - list of data/parity block addresses and maybe checksums
> > - list of "completed" stripes
> >
> > The log is simply these metadata blocks mixed with the data/parity they
> > they describe.
> > Each metadata block commits all previous transactions (or maybe all but
> > the most recent).
> > The current md superblock contains the address of one of these metadata
> > blocks.
> >
> > Does that seem reasonable? Does that seem simpler than what you had?
> > Will that meet your perceived need? It meets mine.
> > If we can do that, I'll apply it and we can move on to the next step.
>
> My original layout has 3 types of blocks: super block, metadata block for data/parity,
> flush block to list completed stripes.
>
> Compared to your single metadata block format, your format
> - delete super block. I'm fine with this. The only useful bits in my
> super block is log tail and seq. we can overload MD superblock
> recovery_offset/resync_offset to record seq/log tail.
> - merge metadata block and flush block to one metadata block. That is
> the new metadata block can record a mix of data/parity/'completed'
> stripes. I don't object to this, it's doable, but don't think this
> makes things simpler.
>
> If you insist, we can take this one. The new metadata block can record a
> mix of data/parity/completed stripes, but it can also just record data
> or completed stripes. So I'll change the disk format to cater this, but
> I'll always put 'completed stripes' info into a separate metadata block
> (that is not mixing with data/parity, but we still just have one
> metadata type). That way I only need slight change of current code and
> it works for the new format. We can, if necessary, let the new metadata
> records any mix of data/parity/completed stripes later. Sounds good?
Yes it does. Having a single sort of metadata block is an important
part of the goal. How the code actually chooses to use these is a
separate issue that can change harmlessly.
There is also the issue of what action commits a previous transaction.
I'm not sure what you had. I'm suggesting that each metadata block
commits previous transactions. Is that a close-enough match to what
you had?
Thanks,
NeilBrown
^ permalink raw reply
* Re: [PATCH V4 00/13] MD: a caching layer for raid5/6
From: Shaohua Li @ 2015-07-10 4:08 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid, songliubraving, hch, dan.j.williams, Kernel-team
In-Reply-To: <20150710092119.297fd9e1@noble>
On Fri, Jul 10, 2015 at 09:21:19AM +1000, NeilBrown wrote:
> On Tue, 7 Jul 2015 22:44:02 -0700 Shaohua Li <shli@fb.com> wrote:
>
> > On Wed, Jul 08, 2015 at 11:56:36AM +1000, NeilBrown wrote:
> > >
> > > Hi,
> > > I made some time to look at these this morning - sorry for the delay.
> > >
> > > Having it all broken down with more complete comments helps a lot -
> > > thanks.
> > >
> > > Unfortunately ... it helps confirm that I really don't like this. It
> > > seems much more complex that it should be. There certainly are a lot
> > > of details that need to be carefully considered, but the result needs
> > > to be simpler.
> > >
> > > A big part of my concern is that you choose to avoid making proper use
> > > of the current stripe cache. Your argument is that it is already
> > > too complex. That is probably true but I don't think you decrease
> > > total complexity by adding extra bits on the side that themselves are
> > > complex in completely different ways.
> > >
> > > I would like to start small and keep it as simple as possible. I
> > > think the minimum useful functionality is closing the write-hole.
> > > Anything else could be done outside raid5 (e.g. bcache), but the
> > > write-hole needs integration. Once that is closed, it may make sense
> > > to add more functionality.
> > >
> > > To do that we need a log, and it was quite sensible for you to put
> > > that first in the list of patches. So let's start with that.
> > >
> > > I would like to propose a single metadata block format. This block
> > > includes the general RAID5 parameters (both 'old' and 'new' for when a
> > > reshape is happening), it includes linkage to find nearby metadata.
> > > It includes a list of the data written after the metadata. And it
> > > includes a list of device-addresses of parity blocks which are now safe
> > > on the RAID and so any log content is now ignored.
> > >
> > > The general mechanism for writing to the log is:
> > > - collect a list of bios. As each arrives update the metadata
> > > block with index information.
> > > - when ready (there are various triggers) write the metadata block
> > > with FLUSH/FUA and write all the data blocks normally. This
> > > metadata block plus data/parity blocks is a 'transaction'.
> > >
> > > We never trigger a log transaction until all writes for the previous
> > > transaction have completed. This means that once the metadata block is
> > > safe, all previous data must also be safe. If this imposes too much
> > > waiting we could allow a "double-buffer" approach were each metadata
> > > block completes the previous-but-one transaction.
> > >
> > > When all of the previous writes complete we trigger a new log write
> > > if there is an outstanding SYNC write, or maybe an outstanding
> > > FLUSH/FUA, but otherwise we wait until the metadata block is full, or
> > > some amount of time has passed.
> > >
> > > This one metadata block serves all of the purposes that you
> > > identified. It records where data is, it commits previous writes, and
> > > records which stripes are being written to RAID and which have been
> > > fully committed.
> > >
> > > With that log in place, we add the code to "hijack ops_run_io" to
> > > write all dirty blocks from the stripe_head to the cache. This
> > > includes the parity of course. Immediately that data is safe the
> > > write requests can be returned and the data is written the the RAID
> > > devices.
> > >
> > > "Recovery" at boot time simply involves:
> > > 1. find and load first metadata block
> > > 2. load the "next" metadata block. If it doesn't exist, stop.
> > > 3. For each parity/data block listed in current block, get the
> > > stripe_head and read the block in.
> > > 4. For each completed stripe listed, find the stripe_head and
> > > invalidate it.
> > > 5. Make the "next" metadata block the current block and goto 2
> > > 6. Flush out all dirty data in stripe cache.
> > >
> > >
> > > I think that all of this *needs* to use the stripe_cache.
> > > Intercepting normal processing between party computation and writing
> > > to the RAID devices must involve the stripe_cache.
> > >
> > > So this is a "simple" start. Once this is written and agreed and
> > > tested and debugged, then we can look at the latency-hiding side of
> > > the cache. I think this should still use the stripe_cache - just
> > > schedule the writes to the log a bit earlier.
> > > If a case can be made for a separate cache then I'm not completely
> > > closed to the idea but I don't really want to think about it until the
> > > write-hole closing code is done.
> > >
> > > So:
> > > [PATCH V4 01/13] MD: add a new disk role to present cache device
> > > probably OK
> > > [PATCH V4 02/13] raid5: directly use mddev->queue
> > > OK
> > > [PATCH V4 03/13] raid5: cache log handling
> > > strip this down to a bare minimum. It needs:
> > > -on-disk metadata block format
> > > -two or three active transactions which identify a list of bios
> > > that are currently in the stripe cache. As the bio is added,
> > > its address info is added to the metadata block.
> > > -interface to "add a bio to a transaction" and "record a
> > > completed stripe".
> > > -timer flush the current transaction if previous one was not
> > > empty.
> > >
> > > I don't think there needs to be a separate superblock in the log.
> > > Each device in the array already has an 'md' superblock. Use e.g
> > > the 'recovery_offset' field in there (which is per-device and
> > > meaningless for a log) to store the address of the most recent
> > > metadata blocks at the time the superblock was written. Search
> > > forward and backward from there to find whole log. Each metadata
> > > block holds everything else that might be useful.
> > >
> > > [PATCH V4 05/13] raid5: cache reclaim support
> > > Just want the "hijack ops_run_io" part of this so that when normal
> > > RAID5 processing wants to write a stripe, blocks get diverted to
> > > the log first and further processing of the stripe is delayed until
> > > those writes complete and the transaction is safe.
> > >
> > > [PATCH V4 08/13] raid5: cache recovery support
> > > Just load all the log into the stripe cache.
> > >
> > >
> > > That should be much more manageable and would be a good start towards
> > > getting all the functionality that you want.
> > >
> > > Feel free to ask questions if I haven't explained things, or if you
> > > want me to look over the metadata format or whatever. I tend to answer
> > > direct questions more quickly than I answer 100kB patchsets :-)
> > Hi,
> >
> > So we are back to the original point (eg, only fix the write hole issue) after
> > 3 months development, which is really disappointed. I don't object just fixing
> > the write hole issue as a start, it's simple which is my arguement of first
> > post too, and probably a slight change of patch 3 meets the goal. The cache
> > does significantly increase complexity, which is well known when we move from
> > 'just fix write hole' to 'fix write hole and do cache'. But the code is right
> > there, I'm wondering how bad it is. Just because of stripe_cache? I don't see
> > any significant technical merit stripe_cache is a must-have here. You said the
> > code is complex. Yes, it is, but it's not because of the new data cache. It's a
> > simple radix tree and takes less than 20% of the total code. The remaining code
> > does what any log device should do (write to log, reclaim when log is full,
> > recovery when log crash). Similar things must be done even stripe_cache is
> > used. And if we don't use stripe_cache, we don't deeply couple with current
> > state machine, the new data cache actually brings flexibility (for example, we
> > can choose not using cache, or we can easily put the cache to NVRAM).
>
> I don't think we are completely back to the start. The product of
> recent months isn't just code - it is also your understanding and
> expertise. You've explored the problems and understand them. You
> could quickly see issues because you have struggled with them already.
>
> I think the principal of "write one to throw away" is a good one - and
> probably one we don't apply often enough. The first time you implement
> something you don't really know what you are doing, so the result is
> bound to be imperfect. The second time you have a much better idea of
> the big picture.
>
> The real big problem that I have with the current proposal is that I
> just don't understand it. Maybe that is my failing. But all the
> evidence suggests that I'll end up needing to maintain it, so I *have*
> to understand it.
>
> But let's not assume we are throwing everything away (but equally not
> promise that we won't) and start with just the first (non trivial)
> patch. The log.
>
> I think we do want a log. I think you have identified the key
> functionality that the log must provide. But I think the current
> layout is too complex. This is something that will be persistent on
> storage devices, so we want to get it as "right" as we can, early.
> It is fairly cheap to throw away a log and start a new one, so fixing
> design mistakes isn't too hard. But we do have to support any format
> we create indefinitely.
> So simple is best. Simple is most extensible.
>
> As I said: A single metadata block format, with:
> - forward/backward linkage - and probably a link to the "first" block
> in the log at the time of writing
> - array state data - brieifly
> - list of data/parity block addresses and maybe checksums
> - list of "completed" stripes
>
> The log is simply these metadata blocks mixed with the data/parity they
> they describe.
> Each metadata block commits all previous transactions (or maybe all but
> the most recent).
> The current md superblock contains the address of one of these metadata
> blocks.
>
> Does that seem reasonable? Does that seem simpler than what you had?
> Will that meet your perceived need? It meets mine.
> If we can do that, I'll apply it and we can move on to the next step.
My original layout has 3 types of blocks: super block, metadata block for data/parity,
flush block to list completed stripes.
Compared to your single metadata block format, your format
- delete super block. I'm fine with this. The only useful bits in my
super block is log tail and seq. we can overload MD superblock
recovery_offset/resync_offset to record seq/log tail.
- merge metadata block and flush block to one metadata block. That is
the new metadata block can record a mix of data/parity/'completed'
stripes. I don't object to this, it's doable, but don't think this
makes things simpler.
If you insist, we can take this one. The new metadata block can record a
mix of data/parity/completed stripes, but it can also just record data
or completed stripes. So I'll change the disk format to cater this, but
I'll always put 'completed stripes' info into a separate metadata block
(that is not mixing with data/parity, but we still just have one
metadata type). That way I only need slight change of current code and
it works for the new format. We can, if necessary, let the new metadata
records any mix of data/parity/completed stripes later. Sounds good?
Thanks,
Shaohua
^ permalink raw reply
* Re: [PATCH V4 00/13] MD: a caching layer for raid5/6
From: NeilBrown @ 2015-07-09 23:21 UTC (permalink / raw)
To: Shaohua Li; +Cc: linux-raid, songliubraving, hch, dan.j.williams, Kernel-team
In-Reply-To: <20150708054344.GA2709238@devbig257.prn2.facebook.com>
On Tue, 7 Jul 2015 22:44:02 -0700 Shaohua Li <shli@fb.com> wrote:
> On Wed, Jul 08, 2015 at 11:56:36AM +1000, NeilBrown wrote:
> >
> > Hi,
> > I made some time to look at these this morning - sorry for the delay.
> >
> > Having it all broken down with more complete comments helps a lot -
> > thanks.
> >
> > Unfortunately ... it helps confirm that I really don't like this. It
> > seems much more complex that it should be. There certainly are a lot
> > of details that need to be carefully considered, but the result needs
> > to be simpler.
> >
> > A big part of my concern is that you choose to avoid making proper use
> > of the current stripe cache. Your argument is that it is already
> > too complex. That is probably true but I don't think you decrease
> > total complexity by adding extra bits on the side that themselves are
> > complex in completely different ways.
> >
> > I would like to start small and keep it as simple as possible. I
> > think the minimum useful functionality is closing the write-hole.
> > Anything else could be done outside raid5 (e.g. bcache), but the
> > write-hole needs integration. Once that is closed, it may make sense
> > to add more functionality.
> >
> > To do that we need a log, and it was quite sensible for you to put
> > that first in the list of patches. So let's start with that.
> >
> > I would like to propose a single metadata block format. This block
> > includes the general RAID5 parameters (both 'old' and 'new' for when a
> > reshape is happening), it includes linkage to find nearby metadata.
> > It includes a list of the data written after the metadata. And it
> > includes a list of device-addresses of parity blocks which are now safe
> > on the RAID and so any log content is now ignored.
> >
> > The general mechanism for writing to the log is:
> > - collect a list of bios. As each arrives update the metadata
> > block with index information.
> > - when ready (there are various triggers) write the metadata block
> > with FLUSH/FUA and write all the data blocks normally. This
> > metadata block plus data/parity blocks is a 'transaction'.
> >
> > We never trigger a log transaction until all writes for the previous
> > transaction have completed. This means that once the metadata block is
> > safe, all previous data must also be safe. If this imposes too much
> > waiting we could allow a "double-buffer" approach were each metadata
> > block completes the previous-but-one transaction.
> >
> > When all of the previous writes complete we trigger a new log write
> > if there is an outstanding SYNC write, or maybe an outstanding
> > FLUSH/FUA, but otherwise we wait until the metadata block is full, or
> > some amount of time has passed.
> >
> > This one metadata block serves all of the purposes that you
> > identified. It records where data is, it commits previous writes, and
> > records which stripes are being written to RAID and which have been
> > fully committed.
> >
> > With that log in place, we add the code to "hijack ops_run_io" to
> > write all dirty blocks from the stripe_head to the cache. This
> > includes the parity of course. Immediately that data is safe the
> > write requests can be returned and the data is written the the RAID
> > devices.
> >
> > "Recovery" at boot time simply involves:
> > 1. find and load first metadata block
> > 2. load the "next" metadata block. If it doesn't exist, stop.
> > 3. For each parity/data block listed in current block, get the
> > stripe_head and read the block in.
> > 4. For each completed stripe listed, find the stripe_head and
> > invalidate it.
> > 5. Make the "next" metadata block the current block and goto 2
> > 6. Flush out all dirty data in stripe cache.
> >
> >
> > I think that all of this *needs* to use the stripe_cache.
> > Intercepting normal processing between party computation and writing
> > to the RAID devices must involve the stripe_cache.
> >
> > So this is a "simple" start. Once this is written and agreed and
> > tested and debugged, then we can look at the latency-hiding side of
> > the cache. I think this should still use the stripe_cache - just
> > schedule the writes to the log a bit earlier.
> > If a case can be made for a separate cache then I'm not completely
> > closed to the idea but I don't really want to think about it until the
> > write-hole closing code is done.
> >
> > So:
> > [PATCH V4 01/13] MD: add a new disk role to present cache device
> > probably OK
> > [PATCH V4 02/13] raid5: directly use mddev->queue
> > OK
> > [PATCH V4 03/13] raid5: cache log handling
> > strip this down to a bare minimum. It needs:
> > -on-disk metadata block format
> > -two or three active transactions which identify a list of bios
> > that are currently in the stripe cache. As the bio is added,
> > its address info is added to the metadata block.
> > -interface to "add a bio to a transaction" and "record a
> > completed stripe".
> > -timer flush the current transaction if previous one was not
> > empty.
> >
> > I don't think there needs to be a separate superblock in the log.
> > Each device in the array already has an 'md' superblock. Use e.g
> > the 'recovery_offset' field in there (which is per-device and
> > meaningless for a log) to store the address of the most recent
> > metadata blocks at the time the superblock was written. Search
> > forward and backward from there to find whole log. Each metadata
> > block holds everything else that might be useful.
> >
> > [PATCH V4 05/13] raid5: cache reclaim support
> > Just want the "hijack ops_run_io" part of this so that when normal
> > RAID5 processing wants to write a stripe, blocks get diverted to
> > the log first and further processing of the stripe is delayed until
> > those writes complete and the transaction is safe.
> >
> > [PATCH V4 08/13] raid5: cache recovery support
> > Just load all the log into the stripe cache.
> >
> >
> > That should be much more manageable and would be a good start towards
> > getting all the functionality that you want.
> >
> > Feel free to ask questions if I haven't explained things, or if you
> > want me to look over the metadata format or whatever. I tend to answer
> > direct questions more quickly than I answer 100kB patchsets :-)
> Hi,
>
> So we are back to the original point (eg, only fix the write hole issue) after
> 3 months development, which is really disappointed. I don't object just fixing
> the write hole issue as a start, it's simple which is my arguement of first
> post too, and probably a slight change of patch 3 meets the goal. The cache
> does significantly increase complexity, which is well known when we move from
> 'just fix write hole' to 'fix write hole and do cache'. But the code is right
> there, I'm wondering how bad it is. Just because of stripe_cache? I don't see
> any significant technical merit stripe_cache is a must-have here. You said the
> code is complex. Yes, it is, but it's not because of the new data cache. It's a
> simple radix tree and takes less than 20% of the total code. The remaining code
> does what any log device should do (write to log, reclaim when log is full,
> recovery when log crash). Similar things must be done even stripe_cache is
> used. And if we don't use stripe_cache, we don't deeply couple with current
> state machine, the new data cache actually brings flexibility (for example, we
> can choose not using cache, or we can easily put the cache to NVRAM).
I don't think we are completely back to the start. The product of
recent months isn't just code - it is also your understanding and
expertise. You've explored the problems and understand them. You
could quickly see issues because you have struggled with them already.
I think the principal of "write one to throw away" is a good one - and
probably one we don't apply often enough. The first time you implement
something you don't really know what you are doing, so the result is
bound to be imperfect. The second time you have a much better idea of
the big picture.
The real big problem that I have with the current proposal is that I
just don't understand it. Maybe that is my failing. But all the
evidence suggests that I'll end up needing to maintain it, so I *have*
to understand it.
But let's not assume we are throwing everything away (but equally not
promise that we won't) and start with just the first (non trivial)
patch. The log.
I think we do want a log. I think you have identified the key
functionality that the log must provide. But I think the current
layout is too complex. This is something that will be persistent on
storage devices, so we want to get it as "right" as we can, early.
It is fairly cheap to throw away a log and start a new one, so fixing
design mistakes isn't too hard. But we do have to support any format
we create indefinitely.
So simple is best. Simple is most extensible.
As I said: A single metadata block format, with:
- forward/backward linkage - and probably a link to the "first" block
in the log at the time of writing
- array state data - brieifly
- list of data/parity block addresses and maybe checksums
- list of "completed" stripes
The log is simply these metadata blocks mixed with the data/parity they
they describe.
Each metadata block commits all previous transactions (or maybe all but
the most recent).
The current md superblock contains the address of one of these metadata
blocks.
Does that seem reasonable? Does that seem simpler than what you had?
Will that meet your perceived need? It meets mine.
If we can do that, I'll apply it and we can move on to the next step.
(and just BTW, if we wanted to use NVRAM to help with this, I would
start out by just putting the log in NVRAM, and then see what needs to
be tweeked to make use of the easier addressing of NVRAM over a block
device).
NeilBrown
^ permalink raw reply
* Re: your protracted and stalled funds transfer from the U.K.
From: adamwitkover56 @ 2015-07-09 9:38 UTC (permalink / raw)
To: Recipients
Attention Pls,
I am writing you with regards to your protracted and stalled funds transfer from the U.K. and would urge you to accord this correspondence your full indulgence and attention, as the authenticity and difference would be evident if you do so.
As a freelance, independent External Audit expert assisting a private financial institution here in Britain in all aspects of their work, your transfer file came to my attention just this morning, and I was compelled to take special interest in it owing to its peculiar history of procedural inconsistencies. Obviously all the persons you assigned to process your transfer have proved to be incompetent. Simply put it that they have given room to some management of the Bank to try and smuggle your fund through a non-existent back door.
My intention is to quickly reactivate and sort out your transfer without any distraction through a simple, transparent and authoritative means where you will not be required to pay any unauthorized fees. You are forbidden to preempt anyone of this contact and I await your response at your earliest convenience.
Thank you.
Sincerely,
Adam Witkover
^ permalink raw reply
* Re: Resurrecting a Dirty RAID-5
From: jtroan @ 2015-07-09 2:57 UTC (permalink / raw)
To: Phil Turmel; +Cc: linux-raid
In-Reply-To: <559D2374.7040208@turmel.org>
Many, many thanks for the solution. I was able to run to a pair of mdadm
-A commands to stitch both RAID devices back together and got it to
completely boot up on the two drives.
As expected, it did note the degraded md1 since the third drive was still
missing. I'll install it when I have a few more spare moments, and have
left the system powered down for now.
I'm also thinking about adding a fourth drive and try to config it as a
hot-spare, giving me some extra margin for failure.
=======================================================================
John M. Troan <jtroan@jt-sw.com>
Maintainer: Football Site @ JT-SW.com
http://www.jt-sw.com/football
Chief of Computer Operations
U.S.S. Kitty Hawk / NCC-1659
=======================================================================
Phil Turmel <philip@turmel.org> wrote on 07/08/2015 09:19:48:
>
> To: jtroan@jt-sw.com, linux-raid@vger.kernel.org,
> Subject: Re: Resurrecting a Dirty RAID-5
>
> Good morning John,
>
> On 07/08/2015 08:14 AM, jtroan@jt-sw.com wrote:
>
> [trim /]
>
> > I booted from the RHEL 6.5 DVD and entered rescue mode. Using
information
> > from the Linux RAID wiki [1], I was able to confirm that my sd0 and sd1
> > drives are alive and have all three partitions I had originally
deployed to
> > them -- /boot, /, and swap space.
> >
> > Following the advice on the wiki [1], I'm asking for some help in using
> > mdadm via rescue mode in trying to get this dirty RAID-5 back on its
feet
> > with the surviving two drives. (I've got a third drive already on hand
> > that's I'll then use to bring the RAID back to full robustness.)
>
> From this report, you should only need to do a forced assembly from the
> rescue environment with the good devices. Like so (substituting actual
> names):
>
> mdadm -Afv /dev/md2 /dev/sda2 /dev/sdb2
>
> If that fails, paste the verbose output in your reply here.
>
> If the above succeeds, you may shut down, plug in your new drive, and
> boot into your normal environment (still degraded, but should be
> bootable). Then add the new drive's partitions to each array.
>
> Don't do *anything* else.
>
> HTH,
>
> Phil
^ permalink raw reply
* Re: [PATCH] bcache: Deletion of an unnecessary check before the function call "kobject_put"
From: SF Markus Elfring @ 2015-07-08 17:48 UTC (permalink / raw)
To: Kent Overstreet, Neil Brown, linux-bcache
Cc: linux-raid, LKML, kernel-janitors, Julia Lawall
In-Reply-To: <546CFECC.8020309@users.sourceforge.net>
> From: Markus Elfring <elfring@users.sourceforge.net>
> Date: Wed, 19 Nov 2014 21:25:39 +0100
>
> The kobject_put() function tests whether its argument is NULL and then
> returns immediately. Thus the test around the call is not needed.
>
> This issue was detected by using the Coccinelle software.
>
> Signed-off-by: Markus Elfring <elfring@users.sourceforge.net>
> ---
> drivers/md/bcache/super.c | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
> index d4713d0..1443c55 100644
> --- a/drivers/md/bcache/super.c
> +++ b/drivers/md/bcache/super.c
> @@ -2076,8 +2076,7 @@ static void bcache_exit(void)
> {
> bch_debug_exit();
> bch_request_exit();
> - if (bcache_kobj)
> - kobject_put(bcache_kobj);
> + kobject_put(bcache_kobj);
> if (bcache_wq)
> destroy_workqueue(bcache_wq);
> if (bcache_major)
>
Would you like to integrate this update suggestion
into another source code repository?
Regards,
Markus
^ permalink raw reply
* Re: Resurrecting a Dirty RAID-5
From: Phil Turmel @ 2015-07-08 13:19 UTC (permalink / raw)
To: jtroan, linux-raid
In-Reply-To: <OFC233BA12.DDED55A2-ON85257E7C.00420655-85257E7C.0043331F@jt-sw.com>
Good morning John,
On 07/08/2015 08:14 AM, jtroan@jt-sw.com wrote:
[trim /]
> I booted from the RHEL 6.5 DVD and entered rescue mode. Using information
> from the Linux RAID wiki [1], I was able to confirm that my sd0 and sd1
> drives are alive and have all three partitions I had originally deployed to
> them -- /boot, /, and swap space.
>
> Following the advice on the wiki [1], I'm asking for some help in using
> mdadm via rescue mode in trying to get this dirty RAID-5 back on its feet
> with the surviving two drives. (I've got a third drive already on hand
> that's I'll then use to bring the RAID back to full robustness.)
From this report, you should only need to do a forced assembly from the
rescue environment with the good devices. Like so (substituting actual
names):
mdadm -Afv /dev/md2 /dev/sda2 /dev/sdb2
If that fails, paste the verbose output in your reply here.
If the above succeeds, you may shut down, plug in your new drive, and
boot into your normal environment (still degraded, but should be
bootable). Then add the new drive's partitions to each array.
Don't do *anything* else.
HTH,
Phil
^ permalink raw reply
* Resurrecting a Dirty RAID-5
From: jtroan @ 2015-07-08 12:14 UTC (permalink / raw)
To: linux-raid
I've accidentally toasted a software RAID-5 on one of my VM hosts, taking a
small number of VMs with it. (Fortunately, no unique data was lost as
every server has a "twinned" system or the data can be regenerated from
elsewhere via script.)
Short background -- the system is RHEL 6.5 and has a three-drive RAID-5
with most of the space. ( /boot is a RAID-1 and it seems to be fine in a
degraded state.) The system locked up when (presumably) one of the drives
died. Upon reboot, I saw errors for "ata2", so I naturally forgot that the
drives started at ata0 and unplugged the drive at "SATA2" (DOH!). The
system errored and halted its boot when the root file system was no longer
available. I then switched around cables and brought up ata0 and ata1, but
the RAID won't start because of the dirty broken raid.
I booted from the RHEL 6.5 DVD and entered rescue mode. Using information
from the Linux RAID wiki [1], I was able to confirm that my sd0 and sd1
drives are alive and have all three partitions I had originally deployed to
them -- /boot, /, and swap space.
Following the advice on the wiki [1], I'm asking for some help in using
mdadm via rescue mode in trying to get this dirty RAID-5 back on its feet
with the surviving two drives. (I've got a third drive already on hand
that's I'll then use to bring the RAID back to full robustness.)
[1] https://raid.wiki.kernel.org/index.php/RAID_Recovery
Many thanks for any assistance that can be provided.....
=======================================================================
John M. Troan <jtroan@jt-sw.com>
Maintainer: Football Site @ JT-SW.com
http://www.jt-sw.com/football
Chief of Computer Operations
U.S.S. Kitty Hawk / NCC-1659
=======================================================================
^ permalink raw reply
* Re: [PATCH V4 00/13] MD: a caching layer for raid5/6
From: Shaohua Li @ 2015-07-08 5:44 UTC (permalink / raw)
To: NeilBrown; +Cc: linux-raid, songliubraving, hch, dan.j.williams, Kernel-team
In-Reply-To: <20150708115636.6c972269@noble>
On Wed, Jul 08, 2015 at 11:56:36AM +1000, NeilBrown wrote:
>
> Hi,
> I made some time to look at these this morning - sorry for the delay.
>
> Having it all broken down with more complete comments helps a lot -
> thanks.
>
> Unfortunately ... it helps confirm that I really don't like this. It
> seems much more complex that it should be. There certainly are a lot
> of details that need to be carefully considered, but the result needs
> to be simpler.
>
> A big part of my concern is that you choose to avoid making proper use
> of the current stripe cache. Your argument is that it is already
> too complex. That is probably true but I don't think you decrease
> total complexity by adding extra bits on the side that themselves are
> complex in completely different ways.
>
> I would like to start small and keep it as simple as possible. I
> think the minimum useful functionality is closing the write-hole.
> Anything else could be done outside raid5 (e.g. bcache), but the
> write-hole needs integration. Once that is closed, it may make sense
> to add more functionality.
>
> To do that we need a log, and it was quite sensible for you to put
> that first in the list of patches. So let's start with that.
>
> I would like to propose a single metadata block format. This block
> includes the general RAID5 parameters (both 'old' and 'new' for when a
> reshape is happening), it includes linkage to find nearby metadata.
> It includes a list of the data written after the metadata. And it
> includes a list of device-addresses of parity blocks which are now safe
> on the RAID and so any log content is now ignored.
>
> The general mechanism for writing to the log is:
> - collect a list of bios. As each arrives update the metadata
> block with index information.
> - when ready (there are various triggers) write the metadata block
> with FLUSH/FUA and write all the data blocks normally. This
> metadata block plus data/parity blocks is a 'transaction'.
>
> We never trigger a log transaction until all writes for the previous
> transaction have completed. This means that once the metadata block is
> safe, all previous data must also be safe. If this imposes too much
> waiting we could allow a "double-buffer" approach were each metadata
> block completes the previous-but-one transaction.
>
> When all of the previous writes complete we trigger a new log write
> if there is an outstanding SYNC write, or maybe an outstanding
> FLUSH/FUA, but otherwise we wait until the metadata block is full, or
> some amount of time has passed.
>
> This one metadata block serves all of the purposes that you
> identified. It records where data is, it commits previous writes, and
> records which stripes are being written to RAID and which have been
> fully committed.
>
> With that log in place, we add the code to "hijack ops_run_io" to
> write all dirty blocks from the stripe_head to the cache. This
> includes the parity of course. Immediately that data is safe the
> write requests can be returned and the data is written the the RAID
> devices.
>
> "Recovery" at boot time simply involves:
> 1. find and load first metadata block
> 2. load the "next" metadata block. If it doesn't exist, stop.
> 3. For each parity/data block listed in current block, get the
> stripe_head and read the block in.
> 4. For each completed stripe listed, find the stripe_head and
> invalidate it.
> 5. Make the "next" metadata block the current block and goto 2
> 6. Flush out all dirty data in stripe cache.
>
>
> I think that all of this *needs* to use the stripe_cache.
> Intercepting normal processing between party computation and writing
> to the RAID devices must involve the stripe_cache.
>
> So this is a "simple" start. Once this is written and agreed and
> tested and debugged, then we can look at the latency-hiding side of
> the cache. I think this should still use the stripe_cache - just
> schedule the writes to the log a bit earlier.
> If a case can be made for a separate cache then I'm not completely
> closed to the idea but I don't really want to think about it until the
> write-hole closing code is done.
>
> So:
> [PATCH V4 01/13] MD: add a new disk role to present cache device
> probably OK
> [PATCH V4 02/13] raid5: directly use mddev->queue
> OK
> [PATCH V4 03/13] raid5: cache log handling
> strip this down to a bare minimum. It needs:
> -on-disk metadata block format
> -two or three active transactions which identify a list of bios
> that are currently in the stripe cache. As the bio is added,
> its address info is added to the metadata block.
> -interface to "add a bio to a transaction" and "record a
> completed stripe".
> -timer flush the current transaction if previous one was not
> empty.
>
> I don't think there needs to be a separate superblock in the log.
> Each device in the array already has an 'md' superblock. Use e.g
> the 'recovery_offset' field in there (which is per-device and
> meaningless for a log) to store the address of the most recent
> metadata blocks at the time the superblock was written. Search
> forward and backward from there to find whole log. Each metadata
> block holds everything else that might be useful.
>
> [PATCH V4 05/13] raid5: cache reclaim support
> Just want the "hijack ops_run_io" part of this so that when normal
> RAID5 processing wants to write a stripe, blocks get diverted to
> the log first and further processing of the stripe is delayed until
> those writes complete and the transaction is safe.
>
> [PATCH V4 08/13] raid5: cache recovery support
> Just load all the log into the stripe cache.
>
>
> That should be much more manageable and would be a good start towards
> getting all the functionality that you want.
>
> Feel free to ask questions if I haven't explained things, or if you
> want me to look over the metadata format or whatever. I tend to answer
> direct questions more quickly than I answer 100kB patchsets :-)
Hi,
So we are back to the original point (eg, only fix the write hole issue) after
3 months development, which is really disappointed. I don't object just fixing
the write hole issue as a start, it's simple which is my arguement of first
post too, and probably a slight change of patch 3 meets the goal. The cache
does significantly increase complexity, which is well known when we move from
'just fix write hole' to 'fix write hole and do cache'. But the code is right
there, I'm wondering how bad it is. Just because of stripe_cache? I don't see
any significant technical merit stripe_cache is a must-have here. You said the
code is complex. Yes, it is, but it's not because of the new data cache. It's a
simple radix tree and takes less than 20% of the total code. The remaining code
does what any log device should do (write to log, reclaim when log is full,
recovery when log crash). Similar things must be done even stripe_cache is
used. And if we don't use stripe_cache, we don't deeply couple with current
state machine, the new data cache actually brings flexibility (for example, we
can choose not using cache, or we can easily put the cache to NVRAM).
Thanks,
Shaohua
^ permalink raw reply
* Re: [PATCH V4 00/13] MD: a caching layer for raid5/6
From: NeilBrown @ 2015-07-08 1:56 UTC (permalink / raw)
To: Shaohua Li; +Cc: linux-raid, songliubraving, hch, dan.j.williams
In-Reply-To: <cover.1435094582.git.shli@fb.com>
Hi,
I made some time to look at these this morning - sorry for the delay.
Having it all broken down with more complete comments helps a lot -
thanks.
Unfortunately ... it helps confirm that I really don't like this. It
seems much more complex that it should be. There certainly are a lot
of details that need to be carefully considered, but the result needs
to be simpler.
A big part of my concern is that you choose to avoid making proper use
of the current stripe cache. Your argument is that it is already
too complex. That is probably true but I don't think you decrease
total complexity by adding extra bits on the side that themselves are
complex in completely different ways.
I would like to start small and keep it as simple as possible. I
think the minimum useful functionality is closing the write-hole.
Anything else could be done outside raid5 (e.g. bcache), but the
write-hole needs integration. Once that is closed, it may make sense
to add more functionality.
To do that we need a log, and it was quite sensible for you to put
that first in the list of patches. So let's start with that.
I would like to propose a single metadata block format. This block
includes the general RAID5 parameters (both 'old' and 'new' for when a
reshape is happening), it includes linkage to find nearby metadata.
It includes a list of the data written after the metadata. And it
includes a list of device-addresses of parity blocks which are now safe
on the RAID and so any log content is now ignored.
The general mechanism for writing to the log is:
- collect a list of bios. As each arrives update the metadata
block with index information.
- when ready (there are various triggers) write the metadata block
with FLUSH/FUA and write all the data blocks normally. This
metadata block plus data/parity blocks is a 'transaction'.
We never trigger a log transaction until all writes for the previous
transaction have completed. This means that once the metadata block is
safe, all previous data must also be safe. If this imposes too much
waiting we could allow a "double-buffer" approach were each metadata
block completes the previous-but-one transaction.
When all of the previous writes complete we trigger a new log write
if there is an outstanding SYNC write, or maybe an outstanding
FLUSH/FUA, but otherwise we wait until the metadata block is full, or
some amount of time has passed.
This one metadata block serves all of the purposes that you
identified. It records where data is, it commits previous writes, and
records which stripes are being written to RAID and which have been
fully committed.
With that log in place, we add the code to "hijack ops_run_io" to
write all dirty blocks from the stripe_head to the cache. This
includes the parity of course. Immediately that data is safe the
write requests can be returned and the data is written the the RAID
devices.
"Recovery" at boot time simply involves:
1. find and load first metadata block
2. load the "next" metadata block. If it doesn't exist, stop.
3. For each parity/data block listed in current block, get the
stripe_head and read the block in.
4. For each completed stripe listed, find the stripe_head and
invalidate it.
5. Make the "next" metadata block the current block and goto 2
6. Flush out all dirty data in stripe cache.
I think that all of this *needs* to use the stripe_cache.
Intercepting normal processing between party computation and writing
to the RAID devices must involve the stripe_cache.
So this is a "simple" start. Once this is written and agreed and
tested and debugged, then we can look at the latency-hiding side of
the cache. I think this should still use the stripe_cache - just
schedule the writes to the log a bit earlier.
If a case can be made for a separate cache then I'm not completely
closed to the idea but I don't really want to think about it until the
write-hole closing code is done.
So:
[PATCH V4 01/13] MD: add a new disk role to present cache device
probably OK
[PATCH V4 02/13] raid5: directly use mddev->queue
OK
[PATCH V4 03/13] raid5: cache log handling
strip this down to a bare minimum. It needs:
-on-disk metadata block format
-two or three active transactions which identify a list of bios
that are currently in the stripe cache. As the bio is added,
its address info is added to the metadata block.
-interface to "add a bio to a transaction" and "record a
completed stripe".
-timer flush the current transaction if previous one was not
empty.
I don't think there needs to be a separate superblock in the log.
Each device in the array already has an 'md' superblock. Use e.g
the 'recovery_offset' field in there (which is per-device and
meaningless for a log) to store the address of the most recent
metadata blocks at the time the superblock was written. Search
forward and backward from there to find whole log. Each metadata
block holds everything else that might be useful.
[PATCH V4 05/13] raid5: cache reclaim support
Just want the "hijack ops_run_io" part of this so that when normal
RAID5 processing wants to write a stripe, blocks get diverted to
the log first and further processing of the stripe is delayed until
those writes complete and the transaction is safe.
[PATCH V4 08/13] raid5: cache recovery support
Just load all the log into the stripe cache.
That should be much more manageable and would be a good start towards
getting all the functionality that you want.
Feel free to ask questions if I haven't explained things, or if you
want me to look over the metadata format or whatever. I tend to answer
direct questions more quickly than I answer 100kB patchsets :-)
thanks,
NeilBrown
On Tue, 23 Jun 2015 14:37:50 -0700 Shaohua Li <shli@fb.com> wrote:
> Hi,
>
> This is the V4 version of the raid5/6 caching layer patches. The patches add
> a caching layer for raid5/6. The caching layer uses a SSD as a cache for a raid
> 5/6. It works like the similar way of a hardware raid controller. The purpose
> is to improve raid performance (reduce read-modify-write) and fix write hole
> issue.
>
> I split the patch to into smaller ones and hopefuly they are easier to
> understand. The splitted patches will not break bisect. Functions of main parts
> are divided well, though some data structures not.
>
> I detached the multiple reclaim thread patch, the patch set is aimed to make
> basic logic ready, and we can improve performance later (as long as we can make
> sure current logic is flexible to have improvement space for performance).
>
> Neil,
>
> For the issue if flush_start block is required, I double checked it. You are
> right we don't really need it, the data/parity checksum stored in cache disk
> will guarantee data integrity, but we can't use a simple disk cache flush as
> there is ordering issue. So the flush_start block does increase overhead, but
> might not too much. I didn't delete it yet, but I'm open to do it.
>
> Thanks,
> Shaohua
>
> V4:
> -split patche into smaller ones
> -add more comments into code and some code cleanup
> -bug fixes in recovery code
> -fix the feature bit
>
> V3:
> -make reclaim multi-thread
> -add statistics in sysfs
> -bug fixes
>
> V2:
> -metadata write doesn't use FUA
> -discard request is only issued when necessary
> -bug fixes and cleanup
>
> Shaohua Li (12):
> raid5: directly use mddev->queue
> raid5: cache log handling
> raid5: cache part of raid5 cache
> raid5: cache reclaim support
> raid5: cache IO error handling
> raid5: cache device quiesce support
> raid5: cache recovery support
> raid5: add some sysfs entries
> raid5: don't allow resize/reshape with cache support
> raid5: guarantee cache release stripes in correct way
> raid5: enable cache for raid array with cache disk
> raid5: skip resync if caching is enabled
>
> Song Liu (1):
> MD: add a new disk role to present cache device
>
> drivers/md/Makefile | 2 +-
> drivers/md/md.c | 24 +-
> drivers/md/md.h | 4 +
> drivers/md/raid5-cache.c | 3755 ++++++++++++++++++++++++++++++++++++++++
> drivers/md/raid5.c | 176 +-
> drivers/md/raid5.h | 24 +
> include/uapi/linux/raid/md_p.h | 79 +
> 7 files changed, 4016 insertions(+), 48 deletions(-)
> create mode 100644 drivers/md/raid5-cache.c
>
^ permalink raw reply
* My Dearest Friend
From: Rosemary Mudia @ 2015-07-07 23:29 UTC (permalink / raw)
Hello Dear,
I want to use this opportunity to introduce myself to you. My name is
Rosemary Mudia, a
house help of Late Rev Fr Joseph Poung of a catholic church here in Malaysia.
Late Rev Fr Joseph Poung left behind a huge amount of money for the
work of God. Some part of the money have been disbursed to faithful
children of God for the expansion of the kingdom. The sum of
$3,000,000.00 is left remaining and I'm contacting you with
faith,believing that you will be willing to use the money for the
purpose of the work of God.
Reply for more details: rosemudia@hotmail.com
Skype:rose.mudia1
Yours Sincerely,
Ms.Rosemary Mudia
^ permalink raw reply
* Re: [PATCH v2 0/3] dm-crypt: Adds support for wiping key when doing suspend/hibernation
From: Pali Rohár @ 2015-07-07 7:59 UTC (permalink / raw)
To: Alasdair Kergon, Mike Snitzer, Neil Brown, Rafael J. Wysocki,
Len Brown, Pavel Machek
Cc: dm-devel, linux-raid, linux-kernel, linux-pm
In-Reply-To: <1434885634-19895-1-git-send-email-pali.rohar@gmail.com>
On Sunday 21 June 2015 13:20:31 Pali Rohár wrote:
> This patch series increase security of suspend and hibernate actions. It allows
> user to safely wipe crypto keys before suspend and hibernate actions starts
> without race conditions on userspace process with heavy I/O.
>
> To automatically wipe cryto key for <device> before hibernate action call:
> $ dmsetup message <device> 0 key wipe_on_hibernation
>
> To automatically wipe cryto key for <device> before suspend action call:
> $ dmsetup message <device> 0 key wipe_on_suspend
>
> To disable automatic wipe call retain_on_suspend/retain_on_hibernation.
>
> Pali Rohár (3):
> PM suspend/hibernate: Call notifier after freezing processes
> dm: Export function dm_suspend_md()
> dm-crypt: Adds support for wiping key when doing suspend/hibernation
>
> drivers/md/dm-crypt.c | 126 +++++++++++++++++++++++++++++++++++++++++++---
> drivers/md/dm.c | 6 +++
> drivers/md/dm.h | 5 ++
> include/linux/suspend.h | 2 +
> kernel/power/hibernate.c | 2 +
> kernel/power/suspend.c | 4 +-
> 6 files changed, 136 insertions(+), 9 deletions(-)
>
Hello, can somebody look and review this (v2) patch series?
--
Pali Rohár
pali.rohar@gmail.com
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 0/1] RFC: Unreliable discard performance can cripple RAID1
From: Mike Snitzer @ 2015-07-07 4:42 UTC (permalink / raw)
To: Jes.Sorensen; +Cc: Neil Brown, linux-raid@vger.kernel.org, Mikulas Patocka
In-Reply-To: <1435105573-1373-1-git-send-email-Jes.Sorensen@redhat.com>
On Tue, Jun 23, 2015 at 8:26 PM, <Jes.Sorensen@redhat.com> wrote:
> From: Jes Sorensen <Jes.Sorensen@redhat.com>
>
> Neil,
>
> I have been hitting issues with discard being ridiculously slow on
> arrays with certain typs of SSDs that seem to serialize discard
> processing.
>
> This is particularly bad as I have seen systems where the IMSM BIOS
> defaults to 4KB chunk size, combined with these badly performing
> drives, it could bump the mkfs on an array from seconds to over 40
> minutes. Most users will stick to the defaults and then hit the
> problem during install without understanding why it goes wrong :(
>
> The problem is that there is no way to benchmark our way to this or
> somehow test if a drive performs discard at reasonable speed. I
> suggest we take an approach similar to that of RAID456 and default to
> disabling discard, except for the case where the user knows the drives
> are safe.
>
> Thoughts?
>
> Cheers,
> Jes
>
>
> Jes Sorensen (1):
> raid0: Disable discard per default due to performance uncertainty
>
> drivers/md/raid0.c | 20 ++++++++++++++++++++
> 1 file changed, 20 insertions(+)
As we've discussed in private (but for the benefit of others):
MD raid0 should do what dm-stripe does. Which is calculate the full
extent to discard for each member in the raid0.
This avoids issuing lots of small discards and hoping the block layer
merges them back up.
See this DM commit for reference:
http://git.kernel.org/linus/7b76ec11fec40203836b488496d2df082d5b2022
^ permalink raw reply
* raid1 - mismatches after resuming interrupted recovery
From: Nate Dailey @ 2015-07-06 16:19 UTC (permalink / raw)
To: linux-raid
I've found that if I interrupt a recovery by removing the target device, do IO
before the recovery checkpoint, then re-add the device and let the recovery
complete, the mismatch_cnt is non-zero after doing a check.
Here's exactly what I'm doing:
- create a 5 GB raid1 with internal bitmap
- do a check, verify zero mismatch_cnt
- remove one member device
- dd 256MB with 2GB seek
- lower sync_speed_min/max to 500
- re-add removed device
- wait 15 sec
- remove the same member device again
- dd 1MB with 1 GB seek
- restore sync_speed_min/max to system defaults
- re-add removed device
- when recovery competes, do another check
At this point the mismatch_cnt is non-zero.
I originally hit this on RHEL 7.1, but tested 4.1.1 from kernel.org and it
happens there too.
I'm out of my league in terms of trying to fix this, but would be happy to test
a fix. I wonder if it's really necessary to resume a bitmap recovery from the
checkpoint? Wouldn't the bitmap always reflect what needs to be copied?
Nate
^ permalink raw reply
* [PATCH trivial] dm: Spelling s/consitent/consistent/
From: Geert Uytterhoeven @ 2015-07-06 13:39 UTC (permalink / raw)
To: Neil Brown, Jiri Kosina; +Cc: linux-raid, linux-kernel, Geert Uytterhoeven
Signed-off-by: Geert Uytterhoeven <geert@linux-m68k.org>
---
drivers/md/Kconfig | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/md/Kconfig b/drivers/md/Kconfig
index b59727309072462f..87338a0aab32f4a4 100644
--- a/drivers/md/Kconfig
+++ b/drivers/md/Kconfig
@@ -478,7 +478,7 @@ config DM_LOG_WRITES
This device-mapper target takes two devices, one device to use
normally, one to log all write operations done to the first device.
This is for use by file system developers wishing to verify that
- their fs is writing a consitent file system at all times by allowing
+ their fs is writing a consistent file system at all times by allowing
them to replay the log in a variety of ways and to check the
contents.
--
1.9.1
^ permalink raw reply related
* [PATCH 3/3] Safeguard against writing to an active device of another node
From: Guoqing Jiang @ 2015-07-06 8:52 UTC (permalink / raw)
To: neilb; +Cc: rgoldwyn, linux-raid
In-Reply-To: <1436172732-21021-1-git-send-email-gqjiang@suse.com>
Modifying an exiting device's superblock or creating a new superblock
on an existing device needs to be checked because the device could be
in use by another node in another array. So, we check this by taking
all superblock locks in userspace so that we don't step onto an active
device used by another node and safeguard against accidental edits.
After the edit is complete, we release all locks and the lockspace so
that it can be used by the kernel space.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
---
Against cluster branch.
Makefile | 6 ++-
mdadm.h | 3 ++
super1.c | 62 +++++++++++++++++++++++++++++++
util.c | 127 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 197 insertions(+), 1 deletion(-)
diff --git a/Makefile b/Makefile
index c189279..a76ae13 100644
--- a/Makefile
+++ b/Makefile
@@ -81,11 +81,12 @@ FAILED_SLOTS_DIR = $(RUN_DIR)/failed-slots
SYSTEMD_DIR=/lib/systemd/system
COROSYNC:=$(shell [ -d /usr/include/corosync ] || echo -DNO_COROSYNC)
+DLM:=$(shell [ -f /usr/include/libdlm.h ] || echo -DNO_DLM)
DIRFLAGS = -DMAP_DIR=\"$(MAP_DIR)\" -DMAP_FILE=\"$(MAP_FILE)\"
DIRFLAGS += -DMDMON_DIR=\"$(MDMON_DIR)\"
DIRFLAGS += -DFAILED_SLOTS_DIR=\"$(FAILED_SLOTS_DIR)\"
-CFLAGS = $(CWFLAGS) $(CXFLAGS) -DSendmail=\""$(MAILCMD)"\" $(CONFFILEFLAGS) $(DIRFLAGS) $(COROSYNC)
+CFLAGS = $(CWFLAGS) $(CXFLAGS) -DSendmail=\""$(MAILCMD)"\" $(CONFFILEFLAGS) $(DIRFLAGS) $(COROSYNC) $(DLM)
VERSION = $(shell [ -d .git ] && git describe HEAD | sed 's/mdadm-//')
VERS_DATE = $(shell [ -d .git ] && date --date="`git log -n1 --format=format:%cd --date=short`" '+%0dth %B %Y' | sed -e 's/1th/1st/' -e 's/2th/2nd/' -e 's/11st/11th/' -e 's/12nd/12th/')
@@ -105,6 +106,9 @@ endif
# LDFLAGS = -static
# STRIP = -s
LDLIBS=-ldl
+ifneq ($(DLM), -DNO_DLM)
+LDLIBS += -ldlm_lt
+endif
INSTALL = /usr/bin/install
DESTDIR =
diff --git a/mdadm.h b/mdadm.h
index 97892e6..59f851e 100644
--- a/mdadm.h
+++ b/mdadm.h
@@ -1429,6 +1429,9 @@ extern char *fd2devnm(int fd);
extern int in_initrd(void);
extern int get_cluster_name(char **name);
+extern int is_clustered(struct supertype *st);
+extern int cluster_get_dlmlock(struct supertype *st, int *lockid);
+extern int cluster_release_dlmlock(struct supertype *st, int lockid);
#define _ROUND_UP(val, base) (((val) + (base) - 1) & ~(base - 1))
#define ROUND_UP(val, base) _ROUND_UP(val, (typeof(val))(base))
diff --git a/super1.c b/super1.c
index 9b991e6..f4d6345 100644
--- a/super1.c
+++ b/super1.c
@@ -1074,6 +1074,18 @@ static int update_super1(struct supertype *st, struct mdinfo *info,
int rv = 0;
struct mdp_superblock_1 *sb = st->sb;
+#ifndef NO_DLM
+ int lockid;
+ if (is_clustered(st)) {
+ rv = cluster_get_dlmlock(st, &lockid);
+ if (rv) {
+ pr_err("Cannot get dlmlock in %s return %d\n", __func__, rv);
+ cluster_release_dlmlock(st, lockid);
+ return rv;
+ }
+ }
+#endif
+
if (strcmp(update, "homehost") == 0 &&
homehost) {
/* Note that 'homehost' is special as it is really
@@ -1330,6 +1342,10 @@ static int update_super1(struct supertype *st, struct mdinfo *info,
rv = -1;
sb->sb_csum = calc_sb_1_csum(sb);
+#ifndef NO_DLM
+ if (is_clustered(st))
+ cluster_release_dlmlock(st, lockid);
+#endif
return rv;
}
@@ -1434,6 +1450,17 @@ static int add_to_super1(struct supertype *st, mdu_disk_info_t *dk,
__u16 *rp = sb->dev_roles + dk->number;
struct devinfo *di, **dip;
+#ifndef NO_DLM
+ int rv, lockid;
+ if (is_clustered(st)) {
+ rv = cluster_get_dlmlock(st, &lockid);
+ if (rv) {
+ pr_err("Cannot get dlmlock in %s return %d\n", __func__, rv);
+ cluster_release_dlmlock(st, lockid);
+ return rv;
+ }
+ }
+#endif
if ((dk->state & 6) == 6) /* active, sync */
*rp = __cpu_to_le16(dk->raid_disk);
else if ((dk->state & ~2) == 0) /* active or idle -> spare */
@@ -1460,6 +1487,10 @@ static int add_to_super1(struct supertype *st, mdu_disk_info_t *dk,
di->next = NULL;
*dip = di;
+#ifndef NO_DLM
+ if (is_clustered(st))
+ cluster_release_dlmlock(st, lockid);
+#endif
return 0;
}
#endif
@@ -1474,6 +1505,18 @@ static int store_super1(struct supertype *st, int fd)
int sbsize;
unsigned long long dsize;
+#ifndef NO_DLM
+ int rv, lockid;
+ if (is_clustered(st)) {
+ rv = cluster_get_dlmlock(st, &lockid);
+ if (rv) {
+ pr_err("Cannot get dlmlock in %s return %d\n", __func__, rv);
+ cluster_release_dlmlock(st, lockid);
+ return rv;
+ }
+ }
+#endif
+
if (!get_dev_size(fd, NULL, &dsize))
return 1;
@@ -1533,6 +1576,10 @@ static int store_super1(struct supertype *st, int fd)
}
}
fsync(fd);
+#ifndef NO_DLM
+ if (is_clustered(st))
+ cluster_release_dlmlock(st, lockid);
+#endif
return 0;
}
@@ -2282,6 +2329,17 @@ static int write_bitmap1(struct supertype *st, int fd, enum bitmap_update update
static void free_super1(struct supertype *st)
{
+#ifndef NO_DLM
+ int rv, lockid;
+ if (is_clustered(st)) {
+ rv = cluster_get_dlmlock(st, &lockid);
+ if (rv) {
+ pr_err("Cannot get dlmlock in %s return %d\n", __func__, rv);
+ cluster_release_dlmlock(st, lockid);
+ return;
+ }
+ }
+#endif
if (st->sb)
free(st->sb);
while (st->info) {
@@ -2292,6 +2350,10 @@ static void free_super1(struct supertype *st)
free(di);
}
st->sb = NULL;
+#ifndef NO_DLM
+ if (is_clustered(st))
+ cluster_release_dlmlock(st, lockid);
+#endif
}
#ifndef MDASSEMBLE
diff --git a/util.c b/util.c
index ea6e688..3357957 100644
--- a/util.c
+++ b/util.c
@@ -24,6 +24,7 @@
#include "mdadm.h"
#include "md_p.h"
+#include <sys/poll.h>
#include <sys/socket.h>
#include <sys/utsname.h>
#include <sys/wait.h>
@@ -42,6 +43,10 @@
#else
#include <corosync/cmap.h>
#endif
+#ifndef NO_DLM
+#include <libdlm.h>
+#include <errno.h>
+#endif
/*
@@ -88,6 +93,128 @@ struct blkpg_partition {
aren't permitted). */
#define BUILD_BUG_ON_ZERO(e) (sizeof(struct { int:-!!(e); }))
+#ifndef NO_DLM
+struct dlm_lock_resource {
+ dlm_lshandle_t *ls;
+ struct dlm_lksb lksb;
+};
+
+struct dlm_lock_resource *dlm_lock_res = NULL;
+static int ast_called = 0;
+
+int is_clustered(struct supertype *st)
+{
+ /* is it a cluster md or not */
+ return st->cluster_name ? 1 :0;
+}
+
+/* Using poll(2) to wait for and dispatch ASTs */
+static int poll_for_ast(dlm_lshandle_t ls)
+{
+ struct pollfd pfd;
+
+ pfd.fd = dlm_ls_get_fd(ls);
+ pfd.events = POLLIN;
+
+ while (!ast_called)
+ {
+ if (poll(&pfd, 1, 0) < 0)
+ {
+ perror("poll");
+ return -1;
+ }
+ dlm_dispatch(dlm_ls_get_fd(ls));
+ }
+ ast_called = 0;
+
+ return 0;
+}
+
+static void dlm_ast(void *arg)
+{
+ ast_called = 1;
+}
+
+/* Create the lockspace, take bitmapXXX locks on all the bitmaps. */
+int cluster_get_dlmlock(struct supertype *st, int *lockid)
+{
+ int ret;
+ char str[64];
+ int flags = LKF_NOQUEUE;
+
+ dlm_lock_res = xmalloc(sizeof(struct dlm_lock_resource));
+ if (!dlm_lock_res)
+ return -ENOMEM;
+
+ dlm_lock_res->ls = dlm_create_lockspace(st->cluster_name, O_RDWR);
+ if (!dlm_lock_res->ls) {
+ pr_err("%s failed to create lockspace\n", st->cluster_name);
+ return -1;
+ }
+
+ /* Conversions need the lockid in the LKSB */
+ if (flags & LKF_CONVERT)
+ dlm_lock_res->lksb.sb_lkid = *lockid;
+
+ snprintf(str, 64, "bitmap%04d", st->nodes);
+ /* if flags with LKF_CONVERT causes below return ENOENT which means
+ * "No such file or directory" */
+ ret = dlm_ls_lock(dlm_lock_res->ls, LKM_PWMODE, &dlm_lock_res->lksb,
+ flags, str, strlen(str), 0, dlm_ast,
+ dlm_lock_res, NULL, NULL);
+ if (ret) {
+ pr_err("error %d when get PW mode on lock %s\n", errno, str);
+ return ret;
+ }
+
+ /* Wait for it to complete */
+ poll_for_ast(dlm_lock_res->ls);
+ *lockid = dlm_lock_res->lksb.sb_lkid;
+
+ errno = dlm_lock_res->lksb.sb_status;
+ if (errno) {
+ pr_err("error %d happened in ast with lock %s\n", errno, str);
+ return -1;
+ }
+
+ return 0;
+}
+
+int cluster_release_dlmlock(struct supertype *st, int lockid)
+{
+ int ret;
+
+ /* if flags with LKF_CONVERT causes below return EINVAL which means
+ * "Invalid argument" */
+ ret = dlm_ls_unlock(dlm_lock_res->ls, lockid, 0, &dlm_lock_res->lksb, dlm_lock_res);
+ if (ret) {
+ pr_err("error %d happened when unlock\n", errno);
+ /* XXX make sure the lock is unlocked eventually */
+ return ret;
+ }
+
+ /* Wait for it to complete */
+ poll_for_ast(dlm_lock_res->ls);
+
+ errno = dlm_lock_res->lksb.sb_status;
+ if (errno != EUNLOCK) {
+ pr_err("error %d happened in ast when unlock lockspace\n", errno);
+ /* XXX make sure the lockspace is unlocked eventually */
+ return -1;
+ }
+
+ ret = dlm_release_lockspace(st->cluster_name, dlm_lock_res->ls, 1);
+ if (ret) {
+ pr_err("error %d happened when release lockspace\n", errno);
+ /* XXX make sure the lockspace is released eventually */
+ return ret;
+ }
+ free(dlm_lock_res);
+
+ return 0;
+}
+#endif
+
/*
* Parse a 128 bit uuid in 4 integers
* format is 32 hexx nibbles with options :.<space> separator
--
1.7.12.4
^ permalink raw reply related
* [PATCH 2/3] md-cluster: use %-64s to print cluster_name
From: Guoqing Jiang @ 2015-07-06 8:52 UTC (permalink / raw)
To: neilb; +Cc: rgoldwyn, linux-raid
In-Reply-To: <1436172732-21021-1-git-send-email-gqjiang@suse.com>
Left align is better for cluster with name less than 64. Also
make the output of cluster name is aligned with others.
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
---
Against cluster branch, this could be squashed with commit 0aa2f15
"mdadm: add the ability to change cluster name", sorry for inconvenience.
bitmap.c | 2 +-
super1.c | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/bitmap.c b/bitmap.c
index 0c3f6de..9847001 100644
--- a/bitmap.c
+++ b/bitmap.c
@@ -342,7 +342,7 @@ int ExamineBitmap(char *filename, int brief, struct supertype *st)
100.0 * info->dirty_bits / (info->total_bits?:1));
} else {
printf(" Cluster nodes : %d\n", sb->nodes);
- printf(" Cluster name : %64s\n", sb->cluster_name);
+ printf(" Cluster name : %-64s\n", sb->cluster_name);
for (i = 0; i < (int)sb->nodes; i++) {
if (i) {
free(info);
diff --git a/super1.c b/super1.c
index fda71e3..9b991e6 100644
--- a/super1.c
+++ b/super1.c
@@ -305,7 +305,7 @@ static void examine_super1(struct supertype *st, char *homehost)
printf(" (local to host %s)", homehost);
printf("\n");
if (bms->nodes > 0)
- printf("Cluster Name : %s", bms->cluster_name);
+ printf(" Cluster Name : %-64s\n", bms->cluster_name);
atime = __le64_to_cpu(sb->ctime) & 0xFFFFFFFFFFULL;
printf(" Creation Time : %.24s\n", ctime(&atime));
c=map_num(pers, __le32_to_cpu(sb->level));
@@ -763,7 +763,7 @@ static void detail_super1(struct supertype *st, char *homehost)
strncmp(sb->set_name, homehost, l) == 0)
printf(" (local to host %s)", homehost);
if (bms->nodes > 0)
- printf("Cluster Name : %64s", bms->cluster_name);
+ printf("\n Cluster Name : %-64s", bms->cluster_name);
printf("\n UUID : ");
for (i=0; i<16; i++) {
if ((i&3)==0 && i != 0) printf(":");
--
1.7.12.4
^ permalink raw reply related
* [PATCH 1/3] mdadm: fix wrong condition for go to abort
From: Guoqing Jiang @ 2015-07-06 8:52 UTC (permalink / raw)
To: neilb; +Cc: rgoldwyn, linux-raid
When parse_cluster_confirm_arg return 0, it means the
arg are parsed successfully, so change !rv to rv.
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
---
Against cluster branch, it could be squashed with commit 4de9091
"Add a new clustered disk", sorry for inconvenience.
Manage.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Manage.c b/Manage.c
index e3bdfb3..377df32 100644
--- a/Manage.c
+++ b/Manage.c
@@ -1302,7 +1302,7 @@ int Manage_subdevs(char *devname, int fd,
rv = parse_cluster_confirm_arg(dv->devname,
&dv->devname,
&raid_slot);
- if (!rv) {
+ if (rv) {
pr_err("Could not get the devname of cluster\n");
goto abort;
}
--
1.7.12.4
^ permalink raw reply related
* [PATCH v5 08/11] block: kill merge_bvec_fn() completely
From: Ming Lin @ 2015-07-06 7:44 UTC (permalink / raw)
To: linux-kernel
Cc: Christoph Hellwig, Jens Axboe, Kent Overstreet, Dongsu Park,
Mike Snitzer, Ming Lin, Lars Ellenberg, drbd-user, Jiri Kosina,
Yehuda Sadeh, Sage Weil, Alex Elder, ceph-devel, Alasdair Kergon,
dm-devel, Neil Brown, linux-raid, Christoph Hellwig,
Martin K. Petersen, Ming Lin
In-Reply-To: <1436168690-32102-1-git-send-email-mlin@kernel.org>
From: Kent Overstreet <kent.overstreet@gmail.com>
As generic_make_request() is now able to handle arbitrarily sized bios,
it's no longer necessary for each individual block driver to define its
own ->merge_bvec_fn() callback. Remove every invocation completely.
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Lars Ellenberg <drbd-dev@lists.linbit.com>
Cc: drbd-user@lists.linbit.com
Cc: Jiri Kosina <jkosina@suse.cz>
Cc: Yehuda Sadeh <yehuda@inktank.com>
Cc: Sage Weil <sage@inktank.com>
Cc: Alex Elder <elder@kernel.org>
Cc: ceph-devel@vger.kernel.org
Cc: Alasdair Kergon <agk@redhat.com>
Cc: Mike Snitzer <snitzer@redhat.com>
Cc: dm-devel@redhat.com
Cc: Neil Brown <neilb@suse.de>
Cc: linux-raid@vger.kernel.org
Cc: Christoph Hellwig <hch@infradead.org>
Cc: "Martin K. Petersen" <martin.petersen@oracle.com>
Acked-by: NeilBrown <neilb@suse.de> (for the 'md' bits)
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
[dpark: also remove ->merge_bvec_fn() in dm-thin as well as
dm-era-target, and resolve merge conflicts]
Signed-off-by: Dongsu Park <dpark@posteo.net>
Signed-off-by: Ming Lin <ming.l@ssi.samsung.com>
---
block/blk-merge.c | 17 +-----
block/blk-settings.c | 22 -------
drivers/block/drbd/drbd_int.h | 1 -
drivers/block/drbd/drbd_main.c | 1 -
drivers/block/drbd/drbd_req.c | 35 ------------
drivers/block/pktcdvd.c | 21 -------
drivers/block/rbd.c | 47 ---------------
drivers/md/dm-cache-target.c | 21 -------
drivers/md/dm-crypt.c | 16 ------
drivers/md/dm-era-target.c | 15 -----
drivers/md/dm-flakey.c | 16 ------
drivers/md/dm-linear.c | 16 ------
drivers/md/dm-log-writes.c | 16 ------
drivers/md/dm-raid.c | 19 ------
drivers/md/dm-snap.c | 15 -----
drivers/md/dm-stripe.c | 21 -------
drivers/md/dm-table.c | 8 ---
drivers/md/dm-thin.c | 31 ----------
drivers/md/dm-verity.c | 16 ------
drivers/md/dm.c | 127 +----------------------------------------
drivers/md/dm.h | 2 -
drivers/md/linear.c | 43 --------------
drivers/md/md.c | 26 ---------
drivers/md/md.h | 12 ----
drivers/md/multipath.c | 21 -------
drivers/md/raid0.c | 56 ------------------
drivers/md/raid0.h | 2 -
drivers/md/raid1.c | 58 +------------------
drivers/md/raid10.c | 121 +--------------------------------------
drivers/md/raid5.c | 32 -----------
include/linux/blkdev.h | 10 ----
include/linux/device-mapper.h | 4 --
32 files changed, 9 insertions(+), 859 deletions(-)
diff --git a/block/blk-merge.c b/block/blk-merge.c
index 3707f30..1f5dfa0 100644
--- a/block/blk-merge.c
+++ b/block/blk-merge.c
@@ -69,24 +69,13 @@ static struct bio *blk_bio_segment_split(struct request_queue *q,
struct bio *split;
struct bio_vec bv, bvprv;
struct bvec_iter iter;
- unsigned seg_size = 0, nsegs = 0;
+ unsigned seg_size = 0, nsegs = 0, sectors = 0;
int prev = 0;
- struct bvec_merge_data bvm = {
- .bi_bdev = bio->bi_bdev,
- .bi_sector = bio->bi_iter.bi_sector,
- .bi_size = 0,
- .bi_rw = bio->bi_rw,
- };
-
bio_for_each_segment(bv, bio, iter) {
- if (q->merge_bvec_fn &&
- q->merge_bvec_fn(q, &bvm, &bv) < (int) bv.bv_len)
- goto split;
-
- bvm.bi_size += bv.bv_len;
+ sectors += bv.bv_len >> 9;
- if (bvm.bi_size >> 9 > queue_max_sectors(q))
+ if (sectors > queue_max_sectors(q))
goto split;
/*
diff --git a/block/blk-settings.c b/block/blk-settings.c
index 12600bf..e90d477 100644
--- a/block/blk-settings.c
+++ b/block/blk-settings.c
@@ -53,28 +53,6 @@ void blk_queue_unprep_rq(struct request_queue *q, unprep_rq_fn *ufn)
}
EXPORT_SYMBOL(blk_queue_unprep_rq);
-/**
- * blk_queue_merge_bvec - set a merge_bvec function for queue
- * @q: queue
- * @mbfn: merge_bvec_fn
- *
- * Usually queues have static limitations on the max sectors or segments that
- * we can put in a request. Stacking drivers may have some settings that
- * are dynamic, and thus we have to query the queue whether it is ok to
- * add a new bio_vec to a bio at a given offset or not. If the block device
- * has such limitations, it needs to register a merge_bvec_fn to control
- * the size of bio's sent to it. Note that a block device *must* allow a
- * single page to be added to an empty bio. The block device driver may want
- * to use the bio_split() function to deal with these bio's. By default
- * no merge_bvec_fn is defined for a queue, and only the fixed limits are
- * honored.
- */
-void blk_queue_merge_bvec(struct request_queue *q, merge_bvec_fn *mbfn)
-{
- q->merge_bvec_fn = mbfn;
-}
-EXPORT_SYMBOL(blk_queue_merge_bvec);
-
void blk_queue_softirq_done(struct request_queue *q, softirq_done_fn *fn)
{
q->softirq_done_fn = fn;
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index efd19c2..7ac66f3 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -1450,7 +1450,6 @@ extern void do_submit(struct work_struct *ws);
extern void __drbd_make_request(struct drbd_device *, struct bio *, unsigned long);
extern void drbd_make_request(struct request_queue *q, struct bio *bio);
extern int drbd_read_remote(struct drbd_device *device, struct drbd_request *req);
-extern int drbd_merge_bvec(struct request_queue *q, struct bvec_merge_data *bvm, struct bio_vec *bvec);
extern int is_valid_ar_handle(struct drbd_request *, sector_t);
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index a151853..74d97f4 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -2774,7 +2774,6 @@ enum drbd_ret_code drbd_create_device(struct drbd_config_context *adm_ctx, unsig
This triggers a max_bio_size message upon first attach or connect */
blk_queue_max_hw_sectors(q, DRBD_MAX_BIO_SIZE_SAFE >> 8);
blk_queue_bounce_limit(q, BLK_BOUNCE_ANY);
- blk_queue_merge_bvec(q, drbd_merge_bvec);
q->queue_lock = &resource->req_lock;
device->md_io.page = alloc_page(GFP_KERNEL);
diff --git a/drivers/block/drbd/drbd_req.c b/drivers/block/drbd/drbd_req.c
index a6265bc..7523f00 100644
--- a/drivers/block/drbd/drbd_req.c
+++ b/drivers/block/drbd/drbd_req.c
@@ -1510,41 +1510,6 @@ void drbd_make_request(struct request_queue *q, struct bio *bio)
__drbd_make_request(device, bio, start_jif);
}
-/* This is called by bio_add_page().
- *
- * q->max_hw_sectors and other global limits are already enforced there.
- *
- * We need to call down to our lower level device,
- * in case it has special restrictions.
- *
- * We also may need to enforce configured max-bio-bvecs limits.
- *
- * As long as the BIO is empty we have to allow at least one bvec,
- * regardless of size and offset, so no need to ask lower levels.
- */
-int drbd_merge_bvec(struct request_queue *q, struct bvec_merge_data *bvm, struct bio_vec *bvec)
-{
- struct drbd_device *device = (struct drbd_device *) q->queuedata;
- unsigned int bio_size = bvm->bi_size;
- int limit = DRBD_MAX_BIO_SIZE;
- int backing_limit;
-
- if (bio_size && get_ldev(device)) {
- unsigned int max_hw_sectors = queue_max_hw_sectors(q);
- struct request_queue * const b =
- device->ldev->backing_bdev->bd_disk->queue;
- if (b->merge_bvec_fn) {
- bvm->bi_bdev = device->ldev->backing_bdev;
- backing_limit = b->merge_bvec_fn(b, bvm, bvec);
- limit = min(limit, backing_limit);
- }
- put_ldev(device);
- if ((limit >> 9) > max_hw_sectors)
- limit = max_hw_sectors << 9;
- }
- return limit;
-}
-
void request_timer_fn(unsigned long data)
{
struct drbd_device *device = (struct drbd_device *) data;
diff --git a/drivers/block/pktcdvd.c b/drivers/block/pktcdvd.c
index 05a81ae..190d7d7 100644
--- a/drivers/block/pktcdvd.c
+++ b/drivers/block/pktcdvd.c
@@ -2506,26 +2506,6 @@ end_io:
-static int pkt_merge_bvec(struct request_queue *q, struct bvec_merge_data *bmd,
- struct bio_vec *bvec)
-{
- struct pktcdvd_device *pd = q->queuedata;
- sector_t zone = get_zone(bmd->bi_sector, pd);
- int used = ((bmd->bi_sector - zone) << 9) + bmd->bi_size;
- int remaining = (pd->settings.size << 9) - used;
- int remaining2;
-
- /*
- * A bio <= PAGE_SIZE must be allowed. If it crosses a packet
- * boundary, pkt_make_request() will split the bio.
- */
- remaining2 = PAGE_SIZE - bmd->bi_size;
- remaining = max(remaining, remaining2);
-
- BUG_ON(remaining < 0);
- return remaining;
-}
-
static void pkt_init_queue(struct pktcdvd_device *pd)
{
struct request_queue *q = pd->disk->queue;
@@ -2533,7 +2513,6 @@ static void pkt_init_queue(struct pktcdvd_device *pd)
blk_queue_make_request(q, pkt_make_request);
blk_queue_logical_block_size(q, CD_FRAMESIZE);
blk_queue_max_hw_sectors(q, PACKET_MAX_SECTORS);
- blk_queue_merge_bvec(q, pkt_merge_bvec);
q->queuedata = pd;
}
diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c
index d94529d..055f9fa 100644
--- a/drivers/block/rbd.c
+++ b/drivers/block/rbd.c
@@ -3462,52 +3462,6 @@ static int rbd_queue_rq(struct blk_mq_hw_ctx *hctx,
return BLK_MQ_RQ_QUEUE_OK;
}
-/*
- * a queue callback. Makes sure that we don't create a bio that spans across
- * multiple osd objects. One exception would be with a single page bios,
- * which we handle later at bio_chain_clone_range()
- */
-static int rbd_merge_bvec(struct request_queue *q, struct bvec_merge_data *bmd,
- struct bio_vec *bvec)
-{
- struct rbd_device *rbd_dev = q->queuedata;
- sector_t sector_offset;
- sector_t sectors_per_obj;
- sector_t obj_sector_offset;
- int ret;
-
- /*
- * Find how far into its rbd object the partition-relative
- * bio start sector is to offset relative to the enclosing
- * device.
- */
- sector_offset = get_start_sect(bmd->bi_bdev) + bmd->bi_sector;
- sectors_per_obj = 1 << (rbd_dev->header.obj_order - SECTOR_SHIFT);
- obj_sector_offset = sector_offset & (sectors_per_obj - 1);
-
- /*
- * Compute the number of bytes from that offset to the end
- * of the object. Account for what's already used by the bio.
- */
- ret = (int) (sectors_per_obj - obj_sector_offset) << SECTOR_SHIFT;
- if (ret > bmd->bi_size)
- ret -= bmd->bi_size;
- else
- ret = 0;
-
- /*
- * Don't send back more than was asked for. And if the bio
- * was empty, let the whole thing through because: "Note
- * that a block device *must* allow a single page to be
- * added to an empty bio."
- */
- rbd_assert(bvec->bv_len <= PAGE_SIZE);
- if (ret > (int) bvec->bv_len || !bmd->bi_size)
- ret = (int) bvec->bv_len;
-
- return ret;
-}
-
static void rbd_free_disk(struct rbd_device *rbd_dev)
{
struct gendisk *disk = rbd_dev->disk;
@@ -3806,7 +3760,6 @@ static int rbd_init_disk(struct rbd_device *rbd_dev)
q->limits.max_discard_sectors = segment_size / SECTOR_SIZE;
q->limits.discard_zeroes_data = 1;
- blk_queue_merge_bvec(q, rbd_merge_bvec);
disk->queue = q;
q->queuedata = rbd_dev;
diff --git a/drivers/md/dm-cache-target.c b/drivers/md/dm-cache-target.c
index 1b4e175..256e54c 100644
--- a/drivers/md/dm-cache-target.c
+++ b/drivers/md/dm-cache-target.c
@@ -3769,26 +3769,6 @@ static int cache_iterate_devices(struct dm_target *ti,
return r;
}
-/*
- * We assume I/O is going to the origin (which is the volume
- * more likely to have restrictions e.g. by being striped).
- * (Looking up the exact location of the data would be expensive
- * and could always be out of date by the time the bio is submitted.)
- */
-static int cache_bvec_merge(struct dm_target *ti,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct cache *cache = ti->private;
- struct request_queue *q = bdev_get_queue(cache->origin_dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = cache->origin_dev->bdev;
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static void set_discard_limits(struct cache *cache, struct queue_limits *limits)
{
/*
@@ -3832,7 +3812,6 @@ static struct target_type cache_target = {
.status = cache_status,
.message = cache_message,
.iterate_devices = cache_iterate_devices,
- .merge = cache_bvec_merge,
.io_hints = cache_io_hints,
};
diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c
index 0f48fed..a1f1d09 100644
--- a/drivers/md/dm-crypt.c
+++ b/drivers/md/dm-crypt.c
@@ -2035,21 +2035,6 @@ error:
return -EINVAL;
}
-static int crypt_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct crypt_config *cc = ti->private;
- struct request_queue *q = bdev_get_queue(cc->dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = cc->dev->bdev;
- bvm->bi_sector = cc->start + dm_target_offset(ti, bvm->bi_sector);
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static int crypt_iterate_devices(struct dm_target *ti,
iterate_devices_callout_fn fn, void *data)
{
@@ -2070,7 +2055,6 @@ static struct target_type crypt_target = {
.preresume = crypt_preresume,
.resume = crypt_resume,
.message = crypt_message,
- .merge = crypt_merge,
.iterate_devices = crypt_iterate_devices,
};
diff --git a/drivers/md/dm-era-target.c b/drivers/md/dm-era-target.c
index ad913cd..0119ebf 100644
--- a/drivers/md/dm-era-target.c
+++ b/drivers/md/dm-era-target.c
@@ -1673,20 +1673,6 @@ static int era_iterate_devices(struct dm_target *ti,
return fn(ti, era->origin_dev, 0, get_dev_size(era->origin_dev), data);
}
-static int era_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct era *era = ti->private;
- struct request_queue *q = bdev_get_queue(era->origin_dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = era->origin_dev->bdev;
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static void era_io_hints(struct dm_target *ti, struct queue_limits *limits)
{
struct era *era = ti->private;
@@ -1717,7 +1703,6 @@ static struct target_type era_target = {
.status = era_status,
.message = era_message,
.iterate_devices = era_iterate_devices,
- .merge = era_merge,
.io_hints = era_io_hints
};
diff --git a/drivers/md/dm-flakey.c b/drivers/md/dm-flakey.c
index b257e46..d955b3e 100644
--- a/drivers/md/dm-flakey.c
+++ b/drivers/md/dm-flakey.c
@@ -387,21 +387,6 @@ static int flakey_ioctl(struct dm_target *ti, unsigned int cmd, unsigned long ar
return r ? : __blkdev_driver_ioctl(dev->bdev, dev->mode, cmd, arg);
}
-static int flakey_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct flakey_c *fc = ti->private;
- struct request_queue *q = bdev_get_queue(fc->dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = fc->dev->bdev;
- bvm->bi_sector = flakey_map_sector(ti, bvm->bi_sector);
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static int flakey_iterate_devices(struct dm_target *ti, iterate_devices_callout_fn fn, void *data)
{
struct flakey_c *fc = ti->private;
@@ -419,7 +404,6 @@ static struct target_type flakey_target = {
.end_io = flakey_end_io,
.status = flakey_status,
.ioctl = flakey_ioctl,
- .merge = flakey_merge,
.iterate_devices = flakey_iterate_devices,
};
diff --git a/drivers/md/dm-linear.c b/drivers/md/dm-linear.c
index 53e848c..7dd5fc8 100644
--- a/drivers/md/dm-linear.c
+++ b/drivers/md/dm-linear.c
@@ -130,21 +130,6 @@ static int linear_ioctl(struct dm_target *ti, unsigned int cmd,
return r ? : __blkdev_driver_ioctl(dev->bdev, dev->mode, cmd, arg);
}
-static int linear_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct linear_c *lc = ti->private;
- struct request_queue *q = bdev_get_queue(lc->dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = lc->dev->bdev;
- bvm->bi_sector = linear_map_sector(ti, bvm->bi_sector);
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static int linear_iterate_devices(struct dm_target *ti,
iterate_devices_callout_fn fn, void *data)
{
@@ -162,7 +147,6 @@ static struct target_type linear_target = {
.map = linear_map,
.status = linear_status,
.ioctl = linear_ioctl,
- .merge = linear_merge,
.iterate_devices = linear_iterate_devices,
};
diff --git a/drivers/md/dm-log-writes.c b/drivers/md/dm-log-writes.c
index ad1b049..883595c 100644
--- a/drivers/md/dm-log-writes.c
+++ b/drivers/md/dm-log-writes.c
@@ -728,21 +728,6 @@ static int log_writes_ioctl(struct dm_target *ti, unsigned int cmd,
return r ? : __blkdev_driver_ioctl(dev->bdev, dev->mode, cmd, arg);
}
-static int log_writes_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct log_writes_c *lc = ti->private;
- struct request_queue *q = bdev_get_queue(lc->dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = lc->dev->bdev;
- bvm->bi_sector = dm_target_offset(ti, bvm->bi_sector);
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static int log_writes_iterate_devices(struct dm_target *ti,
iterate_devices_callout_fn fn,
void *data)
@@ -796,7 +781,6 @@ static struct target_type log_writes_target = {
.end_io = normal_end_io,
.status = log_writes_status,
.ioctl = log_writes_ioctl,
- .merge = log_writes_merge,
.message = log_writes_message,
.iterate_devices = log_writes_iterate_devices,
.io_hints = log_writes_io_hints,
diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c
index 2daa677..97e1651 100644
--- a/drivers/md/dm-raid.c
+++ b/drivers/md/dm-raid.c
@@ -1717,24 +1717,6 @@ static void raid_resume(struct dm_target *ti)
mddev_resume(&rs->md);
}
-static int raid_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct raid_set *rs = ti->private;
- struct md_personality *pers = rs->md.pers;
-
- if (pers && pers->mergeable_bvec)
- return min(max_size, pers->mergeable_bvec(&rs->md, bvm, biovec));
-
- /*
- * In case we can't request the personality because
- * the raid set is not running yet
- *
- * -> return safe minimum
- */
- return rs->md.chunk_sectors;
-}
-
static struct target_type raid_target = {
.name = "raid",
.version = {1, 7, 0},
@@ -1749,7 +1731,6 @@ static struct target_type raid_target = {
.presuspend = raid_presuspend,
.postsuspend = raid_postsuspend,
.resume = raid_resume,
- .merge = raid_merge,
};
static int __init dm_raid_init(void)
diff --git a/drivers/md/dm-snap.c b/drivers/md/dm-snap.c
index 7c82d3c..eabc805 100644
--- a/drivers/md/dm-snap.c
+++ b/drivers/md/dm-snap.c
@@ -2330,20 +2330,6 @@ static void origin_status(struct dm_target *ti, status_type_t type,
}
}
-static int origin_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct dm_origin *o = ti->private;
- struct request_queue *q = bdev_get_queue(o->dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = o->dev->bdev;
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static int origin_iterate_devices(struct dm_target *ti,
iterate_devices_callout_fn fn, void *data)
{
@@ -2362,7 +2348,6 @@ static struct target_type origin_target = {
.resume = origin_resume,
.postsuspend = origin_postsuspend,
.status = origin_status,
- .merge = origin_merge,
.iterate_devices = origin_iterate_devices,
};
diff --git a/drivers/md/dm-stripe.c b/drivers/md/dm-stripe.c
index a672a15..c7c8ced 100644
--- a/drivers/md/dm-stripe.c
+++ b/drivers/md/dm-stripe.c
@@ -412,26 +412,6 @@ static void stripe_io_hints(struct dm_target *ti,
blk_limits_io_opt(limits, chunk_size * sc->stripes);
}
-static int stripe_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct stripe_c *sc = ti->private;
- sector_t bvm_sector = bvm->bi_sector;
- uint32_t stripe;
- struct request_queue *q;
-
- stripe_map_sector(sc, bvm_sector, &stripe, &bvm_sector);
-
- q = bdev_get_queue(sc->stripe[stripe].dev->bdev);
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = sc->stripe[stripe].dev->bdev;
- bvm->bi_sector = sc->stripe[stripe].physical_start + bvm_sector;
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static struct target_type stripe_target = {
.name = "striped",
.version = {1, 5, 1},
@@ -443,7 +423,6 @@ static struct target_type stripe_target = {
.status = stripe_status,
.iterate_devices = stripe_iterate_devices,
.io_hints = stripe_io_hints,
- .merge = stripe_merge,
};
int __init dm_stripe_init(void)
diff --git a/drivers/md/dm-table.c b/drivers/md/dm-table.c
index 16ba55a..afb4ad3 100644
--- a/drivers/md/dm-table.c
+++ b/drivers/md/dm-table.c
@@ -440,14 +440,6 @@ static int dm_set_device_limits(struct dm_target *ti, struct dm_dev *dev,
q->limits.alignment_offset,
(unsigned long long) start << SECTOR_SHIFT);
- /*
- * Check if merge fn is supported.
- * If not we'll force DM to use PAGE_SIZE or
- * smaller I/O, just to be safe.
- */
- if (dm_queue_merge_is_compulsory(q) && !ti->type->merge)
- blk_limits_max_hw_sectors(limits,
- (unsigned int) (PAGE_SIZE >> 9));
return 0;
}
diff --git a/drivers/md/dm-thin.c b/drivers/md/dm-thin.c
index c33f61a..37ef8b8 100644
--- a/drivers/md/dm-thin.c
+++ b/drivers/md/dm-thin.c
@@ -3840,20 +3840,6 @@ static int pool_iterate_devices(struct dm_target *ti,
return fn(ti, pt->data_dev, 0, ti->len, data);
}
-static int pool_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct pool_c *pt = ti->private;
- struct request_queue *q = bdev_get_queue(pt->data_dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = pt->data_dev->bdev;
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static void pool_io_hints(struct dm_target *ti, struct queue_limits *limits)
{
struct pool_c *pt = ti->private;
@@ -3930,7 +3916,6 @@ static struct target_type pool_target = {
.resume = pool_resume,
.message = pool_message,
.status = pool_status,
- .merge = pool_merge,
.iterate_devices = pool_iterate_devices,
.io_hints = pool_io_hints,
};
@@ -4257,21 +4242,6 @@ err:
DMEMIT("Error");
}
-static int thin_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct thin_c *tc = ti->private;
- struct request_queue *q = bdev_get_queue(tc->pool_dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = tc->pool_dev->bdev;
- bvm->bi_sector = dm_target_offset(ti, bvm->bi_sector);
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static int thin_iterate_devices(struct dm_target *ti,
iterate_devices_callout_fn fn, void *data)
{
@@ -4315,7 +4285,6 @@ static struct target_type thin_target = {
.presuspend = thin_presuspend,
.postsuspend = thin_postsuspend,
.status = thin_status,
- .merge = thin_merge,
.iterate_devices = thin_iterate_devices,
.io_hints = thin_io_hints,
};
diff --git a/drivers/md/dm-verity.c b/drivers/md/dm-verity.c
index bb9c6a0..4f2cdd9 100644
--- a/drivers/md/dm-verity.c
+++ b/drivers/md/dm-verity.c
@@ -648,21 +648,6 @@ static int verity_ioctl(struct dm_target *ti, unsigned cmd,
cmd, arg);
}
-static int verity_merge(struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size)
-{
- struct dm_verity *v = ti->private;
- struct request_queue *q = bdev_get_queue(v->data_dev->bdev);
-
- if (!q->merge_bvec_fn)
- return max_size;
-
- bvm->bi_bdev = v->data_dev->bdev;
- bvm->bi_sector = verity_map_sector(v, bvm->bi_sector);
-
- return min(max_size, q->merge_bvec_fn(q, bvm, biovec));
-}
-
static int verity_iterate_devices(struct dm_target *ti,
iterate_devices_callout_fn fn, void *data)
{
@@ -995,7 +980,6 @@ static struct target_type verity_target = {
.map = verity_map,
.status = verity_status,
.ioctl = verity_ioctl,
- .merge = verity_merge,
.iterate_devices = verity_iterate_devices,
.io_hints = verity_io_hints,
};
diff --git a/drivers/md/dm.c b/drivers/md/dm.c
index 530fc9a..c5d0d19 100644
--- a/drivers/md/dm.c
+++ b/drivers/md/dm.c
@@ -124,9 +124,8 @@ EXPORT_SYMBOL_GPL(dm_get_rq_mapinfo);
#define DMF_FREEING 3
#define DMF_DELETING 4
#define DMF_NOFLUSH_SUSPENDING 5
-#define DMF_MERGE_IS_OPTIONAL 6
-#define DMF_DEFERRED_REMOVE 7
-#define DMF_SUSPENDED_INTERNALLY 8
+#define DMF_DEFERRED_REMOVE 6
+#define DMF_SUSPENDED_INTERNALLY 7
/*
* A dummy definition to make RCU happy.
@@ -1726,67 +1725,6 @@ static void __split_and_process_bio(struct mapped_device *md,
* CRUD END
*---------------------------------------------------------------*/
-static int dm_merge_bvec(struct request_queue *q,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec)
-{
- struct mapped_device *md = q->queuedata;
- struct dm_table *map = dm_get_live_table_fast(md);
- struct dm_target *ti;
- sector_t max_sectors, max_size = 0;
-
- if (unlikely(!map))
- goto out;
-
- ti = dm_table_find_target(map, bvm->bi_sector);
- if (!dm_target_is_valid(ti))
- goto out;
-
- /*
- * Find maximum amount of I/O that won't need splitting
- */
- max_sectors = min(max_io_len(bvm->bi_sector, ti),
- (sector_t) queue_max_sectors(q));
- max_size = (max_sectors << SECTOR_SHIFT) - bvm->bi_size;
-
- /*
- * FIXME: this stop-gap fix _must_ be cleaned up (by passing a sector_t
- * to the targets' merge function since it holds sectors not bytes).
- * Just doing this as an interim fix for stable@ because the more
- * comprehensive cleanup of switching to sector_t will impact every
- * DM target that implements a ->merge hook.
- */
- if (max_size > INT_MAX)
- max_size = INT_MAX;
-
- /*
- * merge_bvec_fn() returns number of bytes
- * it can accept at this offset
- * max is precomputed maximal io size
- */
- if (max_size && ti->type->merge)
- max_size = ti->type->merge(ti, bvm, biovec, (int) max_size);
- /*
- * If the target doesn't support merge method and some of the devices
- * provided their merge_bvec method (we know this by looking for the
- * max_hw_sectors that dm_set_device_limits may set), then we can't
- * allow bios with multiple vector entries. So always set max_size
- * to 0, and the code below allows just one page.
- */
- else if (queue_max_hw_sectors(q) <= PAGE_SIZE >> 9)
- max_size = 0;
-
-out:
- dm_put_live_table_fast(md);
- /*
- * Always allow an entire first page
- */
- if (max_size <= biovec->bv_len && !(bvm->bi_size >> SECTOR_SHIFT))
- max_size = biovec->bv_len;
-
- return max_size;
-}
-
/*
* The request function that just remaps the bio built up by
* dm_merge_bvec.
@@ -2509,59 +2447,6 @@ static void __set_size(struct mapped_device *md, sector_t size)
}
/*
- * Return 1 if the queue has a compulsory merge_bvec_fn function.
- *
- * If this function returns 0, then the device is either a non-dm
- * device without a merge_bvec_fn, or it is a dm device that is
- * able to split any bios it receives that are too big.
- */
-int dm_queue_merge_is_compulsory(struct request_queue *q)
-{
- struct mapped_device *dev_md;
-
- if (!q->merge_bvec_fn)
- return 0;
-
- if (q->make_request_fn == dm_make_request) {
- dev_md = q->queuedata;
- if (test_bit(DMF_MERGE_IS_OPTIONAL, &dev_md->flags))
- return 0;
- }
-
- return 1;
-}
-
-static int dm_device_merge_is_compulsory(struct dm_target *ti,
- struct dm_dev *dev, sector_t start,
- sector_t len, void *data)
-{
- struct block_device *bdev = dev->bdev;
- struct request_queue *q = bdev_get_queue(bdev);
-
- return dm_queue_merge_is_compulsory(q);
-}
-
-/*
- * Return 1 if it is acceptable to ignore merge_bvec_fn based
- * on the properties of the underlying devices.
- */
-static int dm_table_merge_is_optional(struct dm_table *table)
-{
- unsigned i = 0;
- struct dm_target *ti;
-
- while (i < dm_table_get_num_targets(table)) {
- ti = dm_table_get_target(table, i++);
-
- if (ti->type->iterate_devices &&
- ti->type->iterate_devices(ti, dm_device_merge_is_compulsory, NULL))
- return 0;
- }
-
- return 1;
-}
-
-/*
* Returns old map, which caller must destroy.
*/
static struct dm_table *__bind(struct mapped_device *md, struct dm_table *t,
@@ -2570,7 +2455,6 @@ static struct dm_table *__bind(struct mapped_device *md, struct dm_table *t,
struct dm_table *old_map;
struct request_queue *q = md->queue;
sector_t size;
- int merge_is_optional;
size = dm_table_get_size(t);
@@ -2596,17 +2480,11 @@ static struct dm_table *__bind(struct mapped_device *md, struct dm_table *t,
__bind_mempools(md, t);
- merge_is_optional = dm_table_merge_is_optional(t);
-
old_map = rcu_dereference_protected(md->map, lockdep_is_held(&md->suspend_lock));
rcu_assign_pointer(md->map, t);
md->immutable_target_type = dm_table_get_immutable_target_type(t);
dm_table_set_restrictions(t, q, limits);
- if (merge_is_optional)
- set_bit(DMF_MERGE_IS_OPTIONAL, &md->flags);
- else
- clear_bit(DMF_MERGE_IS_OPTIONAL, &md->flags);
if (old_map)
dm_sync_table(md);
@@ -2887,7 +2765,6 @@ int dm_setup_md_queue(struct mapped_device *md)
case DM_TYPE_BIO_BASED:
dm_init_old_md_queue(md);
blk_queue_make_request(md->queue, dm_make_request);
- blk_queue_merge_bvec(md->queue, dm_merge_bvec);
break;
}
diff --git a/drivers/md/dm.h b/drivers/md/dm.h
index 4e98499..7edcf97 100644
--- a/drivers/md/dm.h
+++ b/drivers/md/dm.h
@@ -78,8 +78,6 @@ bool dm_table_mq_request_based(struct dm_table *t);
void dm_table_free_md_mempools(struct dm_table *t);
struct dm_md_mempools *dm_table_get_md_mempools(struct dm_table *t);
-int dm_queue_merge_is_compulsory(struct request_queue *q);
-
void dm_lock_md_type(struct mapped_device *md);
void dm_unlock_md_type(struct mapped_device *md);
void dm_set_md_type(struct mapped_device *md, unsigned type);
diff --git a/drivers/md/linear.c b/drivers/md/linear.c
index fa7d577..8721ef9 100644
--- a/drivers/md/linear.c
+++ b/drivers/md/linear.c
@@ -52,48 +52,6 @@ static inline struct dev_info *which_dev(struct mddev *mddev, sector_t sector)
return conf->disks + lo;
}
-/**
- * linear_mergeable_bvec -- tell bio layer if two requests can be merged
- * @q: request queue
- * @bvm: properties of new bio
- * @biovec: the request that could be merged to it.
- *
- * Return amount of bytes we can take at this offset
- */
-static int linear_mergeable_bvec(struct mddev *mddev,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec)
-{
- struct dev_info *dev0;
- unsigned long maxsectors, bio_sectors = bvm->bi_size >> 9;
- sector_t sector = bvm->bi_sector + get_start_sect(bvm->bi_bdev);
- int maxbytes = biovec->bv_len;
- struct request_queue *subq;
-
- dev0 = which_dev(mddev, sector);
- maxsectors = dev0->end_sector - sector;
- subq = bdev_get_queue(dev0->rdev->bdev);
- if (subq->merge_bvec_fn) {
- bvm->bi_bdev = dev0->rdev->bdev;
- bvm->bi_sector -= dev0->end_sector - dev0->rdev->sectors;
- maxbytes = min(maxbytes, subq->merge_bvec_fn(subq, bvm,
- biovec));
- }
-
- if (maxsectors < bio_sectors)
- maxsectors = 0;
- else
- maxsectors -= bio_sectors;
-
- if (maxsectors <= (PAGE_SIZE >> 9 ) && bio_sectors == 0)
- return maxbytes;
-
- if (maxsectors > (maxbytes >> 9))
- return maxbytes;
- else
- return maxsectors << 9;
-}
-
static int linear_congested(struct mddev *mddev, int bits)
{
struct linear_conf *conf;
@@ -338,7 +296,6 @@ static struct md_personality linear_personality =
.size = linear_size,
.quiesce = linear_quiesce,
.congested = linear_congested,
- .mergeable_bvec = linear_mergeable_bvec,
};
static int __init linear_init (void)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index 10bf6d4..a4aa6e5 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -352,29 +352,6 @@ static int md_congested(void *data, int bits)
return mddev_congested(mddev, bits);
}
-static int md_mergeable_bvec(struct request_queue *q,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec)
-{
- struct mddev *mddev = q->queuedata;
- int ret;
- rcu_read_lock();
- if (mddev->suspended) {
- /* Must always allow one vec */
- if (bvm->bi_size == 0)
- ret = biovec->bv_len;
- else
- ret = 0;
- } else {
- struct md_personality *pers = mddev->pers;
- if (pers && pers->mergeable_bvec)
- ret = pers->mergeable_bvec(mddev, bvm, biovec);
- else
- ret = biovec->bv_len;
- }
- rcu_read_unlock();
- return ret;
-}
/*
* Generic flush handling for md
*/
@@ -5188,7 +5165,6 @@ int md_run(struct mddev *mddev)
if (mddev->queue) {
mddev->queue->backing_dev_info.congested_data = mddev;
mddev->queue->backing_dev_info.congested_fn = md_congested;
- blk_queue_merge_bvec(mddev->queue, md_mergeable_bvec);
}
if (pers->sync_request) {
if (mddev->kobj.sd &&
@@ -5317,7 +5293,6 @@ static void md_clean(struct mddev *mddev)
mddev->degraded = 0;
mddev->safemode = 0;
mddev->private = NULL;
- mddev->merge_check_needed = 0;
mddev->bitmap_info.offset = 0;
mddev->bitmap_info.default_offset = 0;
mddev->bitmap_info.default_space = 0;
@@ -5514,7 +5489,6 @@ static int do_md_stop(struct mddev *mddev, int mode,
__md_stop_writes(mddev);
__md_stop(mddev);
- mddev->queue->merge_bvec_fn = NULL;
mddev->queue->backing_dev_info.congested_fn = NULL;
/* tell userspace to handle 'inactive' */
diff --git a/drivers/md/md.h b/drivers/md/md.h
index 7da6e9c..ab33957 100644
--- a/drivers/md/md.h
+++ b/drivers/md/md.h
@@ -134,10 +134,6 @@ enum flag_bits {
Bitmap_sync, /* ..actually, not quite In_sync. Need a
* bitmap-based recovery to get fully in sync
*/
- Unmerged, /* device is being added to array and should
- * be considerred for bvec_merge_fn but not
- * yet for actual IO
- */
WriteMostly, /* Avoid reading if at all possible */
AutoDetected, /* added by auto-detect */
Blocked, /* An error occurred but has not yet
@@ -374,10 +370,6 @@ struct mddev {
int degraded; /* whether md should consider
* adding a spare
*/
- int merge_check_needed; /* at least one
- * member device
- * has a
- * merge_bvec_fn */
atomic_t recovery_active; /* blocks scheduled, but not written */
wait_queue_head_t recovery_wait;
@@ -532,10 +524,6 @@ struct md_personality
/* congested implements bdi.congested_fn().
* Will not be called while array is 'suspended' */
int (*congested)(struct mddev *mddev, int bits);
- /* mergeable_bvec is use to implement ->merge_bvec_fn */
- int (*mergeable_bvec)(struct mddev *mddev,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec);
};
struct md_sysfs_entry {
diff --git a/drivers/md/multipath.c b/drivers/md/multipath.c
index ac3ede2..7ee27fb 100644
--- a/drivers/md/multipath.c
+++ b/drivers/md/multipath.c
@@ -257,18 +257,6 @@ static int multipath_add_disk(struct mddev *mddev, struct md_rdev *rdev)
disk_stack_limits(mddev->gendisk, rdev->bdev,
rdev->data_offset << 9);
- /* as we don't honour merge_bvec_fn, we must never risk
- * violating it, so limit ->max_segments to one, lying
- * within a single page.
- * (Note: it is very unlikely that a device with
- * merge_bvec_fn will be involved in multipath.)
- */
- if (q->merge_bvec_fn) {
- blk_queue_max_segments(mddev->queue, 1);
- blk_queue_segment_boundary(mddev->queue,
- PAGE_CACHE_SIZE - 1);
- }
-
spin_lock_irq(&conf->device_lock);
mddev->degraded--;
rdev->raid_disk = path;
@@ -432,15 +420,6 @@ static int multipath_run (struct mddev *mddev)
disk_stack_limits(mddev->gendisk, rdev->bdev,
rdev->data_offset << 9);
- /* as we don't honour merge_bvec_fn, we must never risk
- * violating it, not that we ever expect a device with
- * a merge_bvec_fn to be involved in multipath */
- if (rdev->bdev->bd_disk->queue->merge_bvec_fn) {
- blk_queue_max_segments(mddev->queue, 1);
- blk_queue_segment_boundary(mddev->queue,
- PAGE_CACHE_SIZE - 1);
- }
-
if (!test_bit(Faulty, &rdev->flags))
working_disks++;
}
diff --git a/drivers/md/raid0.c b/drivers/md/raid0.c
index efb654e..c853331 100644
--- a/drivers/md/raid0.c
+++ b/drivers/md/raid0.c
@@ -192,9 +192,6 @@ static int create_strip_zones(struct mddev *mddev, struct r0conf **private_conf)
disk_stack_limits(mddev->gendisk, rdev1->bdev,
rdev1->data_offset << 9);
- if (rdev1->bdev->bd_disk->queue->merge_bvec_fn)
- conf->has_merge_bvec = 1;
-
if (!smallest || (rdev1->sectors < smallest->sectors))
smallest = rdev1;
cnt++;
@@ -351,58 +348,6 @@ static struct md_rdev *map_sector(struct mddev *mddev, struct strip_zone *zone,
+ sector_div(sector, zone->nb_dev)];
}
-/**
- * raid0_mergeable_bvec -- tell bio layer if two requests can be merged
- * @mddev: the md device
- * @bvm: properties of new bio
- * @biovec: the request that could be merged to it.
- *
- * Return amount of bytes we can accept at this offset
- */
-static int raid0_mergeable_bvec(struct mddev *mddev,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec)
-{
- struct r0conf *conf = mddev->private;
- sector_t sector = bvm->bi_sector + get_start_sect(bvm->bi_bdev);
- sector_t sector_offset = sector;
- int max;
- unsigned int chunk_sectors = mddev->chunk_sectors;
- unsigned int bio_sectors = bvm->bi_size >> 9;
- struct strip_zone *zone;
- struct md_rdev *rdev;
- struct request_queue *subq;
-
- if (is_power_of_2(chunk_sectors))
- max = (chunk_sectors - ((sector & (chunk_sectors-1))
- + bio_sectors)) << 9;
- else
- max = (chunk_sectors - (sector_div(sector, chunk_sectors)
- + bio_sectors)) << 9;
- if (max < 0)
- max = 0; /* bio_add cannot handle a negative return */
- if (max <= biovec->bv_len && bio_sectors == 0)
- return biovec->bv_len;
- if (max < biovec->bv_len)
- /* too small already, no need to check further */
- return max;
- if (!conf->has_merge_bvec)
- return max;
-
- /* May need to check subordinate device */
- sector = sector_offset;
- zone = find_zone(mddev->private, §or_offset);
- rdev = map_sector(mddev, zone, sector, §or_offset);
- subq = bdev_get_queue(rdev->bdev);
- if (subq->merge_bvec_fn) {
- bvm->bi_bdev = rdev->bdev;
- bvm->bi_sector = sector_offset + zone->dev_start +
- rdev->data_offset;
- return min(max, subq->merge_bvec_fn(subq, bvm, biovec));
- } else
- return max;
-}
-
static sector_t raid0_size(struct mddev *mddev, sector_t sectors, int raid_disks)
{
sector_t array_sectors = 0;
@@ -727,7 +672,6 @@ static struct md_personality raid0_personality=
.takeover = raid0_takeover,
.quiesce = raid0_quiesce,
.congested = raid0_congested,
- .mergeable_bvec = raid0_mergeable_bvec,
};
static int __init raid0_init (void)
diff --git a/drivers/md/raid0.h b/drivers/md/raid0.h
index 05539d9..7127a62 100644
--- a/drivers/md/raid0.h
+++ b/drivers/md/raid0.h
@@ -12,8 +12,6 @@ struct r0conf {
struct md_rdev **devlist; /* lists of rdevs, pointed to
* by strip_zone->dev */
int nr_strip_zones;
- int has_merge_bvec; /* at least one member has
- * a merge_bvec_fn */
};
#endif
diff --git a/drivers/md/raid1.c b/drivers/md/raid1.c
index f80f1af..bb90edb 100644
--- a/drivers/md/raid1.c
+++ b/drivers/md/raid1.c
@@ -557,7 +557,6 @@ static int read_balance(struct r1conf *conf, struct r1bio *r1_bio, int *max_sect
rdev = rcu_dereference(conf->mirrors[disk].rdev);
if (r1_bio->bios[disk] == IO_BLOCKED
|| rdev == NULL
- || test_bit(Unmerged, &rdev->flags)
|| test_bit(Faulty, &rdev->flags))
continue;
if (!test_bit(In_sync, &rdev->flags) &&
@@ -708,38 +707,6 @@ static int read_balance(struct r1conf *conf, struct r1bio *r1_bio, int *max_sect
return best_disk;
}
-static int raid1_mergeable_bvec(struct mddev *mddev,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec)
-{
- struct r1conf *conf = mddev->private;
- sector_t sector = bvm->bi_sector + get_start_sect(bvm->bi_bdev);
- int max = biovec->bv_len;
-
- if (mddev->merge_check_needed) {
- int disk;
- rcu_read_lock();
- for (disk = 0; disk < conf->raid_disks * 2; disk++) {
- struct md_rdev *rdev = rcu_dereference(
- conf->mirrors[disk].rdev);
- if (rdev && !test_bit(Faulty, &rdev->flags)) {
- struct request_queue *q =
- bdev_get_queue(rdev->bdev);
- if (q->merge_bvec_fn) {
- bvm->bi_sector = sector +
- rdev->data_offset;
- bvm->bi_bdev = rdev->bdev;
- max = min(max, q->merge_bvec_fn(
- q, bvm, biovec));
- }
- }
- }
- rcu_read_unlock();
- }
- return max;
-
-}
-
static int raid1_congested(struct mddev *mddev, int bits)
{
struct r1conf *conf = mddev->private;
@@ -1268,8 +1235,7 @@ read_again:
break;
}
r1_bio->bios[i] = NULL;
- if (!rdev || test_bit(Faulty, &rdev->flags)
- || test_bit(Unmerged, &rdev->flags)) {
+ if (!rdev || test_bit(Faulty, &rdev->flags)) {
if (i < conf->raid_disks)
set_bit(R1BIO_Degraded, &r1_bio->state);
continue;
@@ -1614,7 +1580,6 @@ static int raid1_add_disk(struct mddev *mddev, struct md_rdev *rdev)
struct raid1_info *p;
int first = 0;
int last = conf->raid_disks - 1;
- struct request_queue *q = bdev_get_queue(rdev->bdev);
if (mddev->recovery_disabled == conf->recovery_disabled)
return -EBUSY;
@@ -1622,11 +1587,6 @@ static int raid1_add_disk(struct mddev *mddev, struct md_rdev *rdev)
if (rdev->raid_disk >= 0)
first = last = rdev->raid_disk;
- if (q->merge_bvec_fn) {
- set_bit(Unmerged, &rdev->flags);
- mddev->merge_check_needed = 1;
- }
-
for (mirror = first; mirror <= last; mirror++) {
p = conf->mirrors+mirror;
if (!p->rdev) {
@@ -1658,19 +1618,6 @@ static int raid1_add_disk(struct mddev *mddev, struct md_rdev *rdev)
break;
}
}
- if (err == 0 && test_bit(Unmerged, &rdev->flags)) {
- /* Some requests might not have seen this new
- * merge_bvec_fn. We must wait for them to complete
- * before merging the device fully.
- * First we make sure any code which has tested
- * our function has submitted the request, then
- * we wait for all outstanding requests to complete.
- */
- synchronize_sched();
- freeze_array(conf, 0);
- unfreeze_array(conf);
- clear_bit(Unmerged, &rdev->flags);
- }
md_integrity_add_rdev(rdev, mddev);
if (mddev->queue && blk_queue_discard(bdev_get_queue(rdev->bdev)))
queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, mddev->queue);
@@ -2807,8 +2754,6 @@ static struct r1conf *setup_conf(struct mddev *mddev)
goto abort;
disk->rdev = rdev;
q = bdev_get_queue(rdev->bdev);
- if (q->merge_bvec_fn)
- mddev->merge_check_needed = 1;
disk->head_position = 0;
disk->seq_start = MaxSector;
@@ -3173,7 +3118,6 @@ static struct md_personality raid1_personality =
.quiesce = raid1_quiesce,
.takeover = raid1_takeover,
.congested = raid1_congested,
- .mergeable_bvec = raid1_mergeable_bvec,
};
static int __init raid_init(void)
diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c
index 940f2f3..84cb6bc 100644
--- a/drivers/md/raid10.c
+++ b/drivers/md/raid10.c
@@ -672,93 +672,6 @@ static sector_t raid10_find_virt(struct r10conf *conf, sector_t sector, int dev)
return (vchunk << geo->chunk_shift) + offset;
}
-/**
- * raid10_mergeable_bvec -- tell bio layer if a two requests can be merged
- * @mddev: the md device
- * @bvm: properties of new bio
- * @biovec: the request that could be merged to it.
- *
- * Return amount of bytes we can accept at this offset
- * This requires checking for end-of-chunk if near_copies != raid_disks,
- * and for subordinate merge_bvec_fns if merge_check_needed.
- */
-static int raid10_mergeable_bvec(struct mddev *mddev,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec)
-{
- struct r10conf *conf = mddev->private;
- sector_t sector = bvm->bi_sector + get_start_sect(bvm->bi_bdev);
- int max;
- unsigned int chunk_sectors;
- unsigned int bio_sectors = bvm->bi_size >> 9;
- struct geom *geo = &conf->geo;
-
- chunk_sectors = (conf->geo.chunk_mask & conf->prev.chunk_mask) + 1;
- if (conf->reshape_progress != MaxSector &&
- ((sector >= conf->reshape_progress) !=
- conf->mddev->reshape_backwards))
- geo = &conf->prev;
-
- if (geo->near_copies < geo->raid_disks) {
- max = (chunk_sectors - ((sector & (chunk_sectors - 1))
- + bio_sectors)) << 9;
- if (max < 0)
- /* bio_add cannot handle a negative return */
- max = 0;
- if (max <= biovec->bv_len && bio_sectors == 0)
- return biovec->bv_len;
- } else
- max = biovec->bv_len;
-
- if (mddev->merge_check_needed) {
- struct {
- struct r10bio r10_bio;
- struct r10dev devs[conf->copies];
- } on_stack;
- struct r10bio *r10_bio = &on_stack.r10_bio;
- int s;
- if (conf->reshape_progress != MaxSector) {
- /* Cannot give any guidance during reshape */
- if (max <= biovec->bv_len && bio_sectors == 0)
- return biovec->bv_len;
- return 0;
- }
- r10_bio->sector = sector;
- raid10_find_phys(conf, r10_bio);
- rcu_read_lock();
- for (s = 0; s < conf->copies; s++) {
- int disk = r10_bio->devs[s].devnum;
- struct md_rdev *rdev = rcu_dereference(
- conf->mirrors[disk].rdev);
- if (rdev && !test_bit(Faulty, &rdev->flags)) {
- struct request_queue *q =
- bdev_get_queue(rdev->bdev);
- if (q->merge_bvec_fn) {
- bvm->bi_sector = r10_bio->devs[s].addr
- + rdev->data_offset;
- bvm->bi_bdev = rdev->bdev;
- max = min(max, q->merge_bvec_fn(
- q, bvm, biovec));
- }
- }
- rdev = rcu_dereference(conf->mirrors[disk].replacement);
- if (rdev && !test_bit(Faulty, &rdev->flags)) {
- struct request_queue *q =
- bdev_get_queue(rdev->bdev);
- if (q->merge_bvec_fn) {
- bvm->bi_sector = r10_bio->devs[s].addr
- + rdev->data_offset;
- bvm->bi_bdev = rdev->bdev;
- max = min(max, q->merge_bvec_fn(
- q, bvm, biovec));
- }
- }
- }
- rcu_read_unlock();
- }
- return max;
-}
-
/*
* This routine returns the disk from which the requested read should
* be done. There is a per-array 'next expected sequential IO' sector
@@ -821,12 +734,10 @@ retry:
disk = r10_bio->devs[slot].devnum;
rdev = rcu_dereference(conf->mirrors[disk].replacement);
if (rdev == NULL || test_bit(Faulty, &rdev->flags) ||
- test_bit(Unmerged, &rdev->flags) ||
r10_bio->devs[slot].addr + sectors > rdev->recovery_offset)
rdev = rcu_dereference(conf->mirrors[disk].rdev);
if (rdev == NULL ||
- test_bit(Faulty, &rdev->flags) ||
- test_bit(Unmerged, &rdev->flags))
+ test_bit(Faulty, &rdev->flags))
continue;
if (!test_bit(In_sync, &rdev->flags) &&
r10_bio->devs[slot].addr + sectors > rdev->recovery_offset)
@@ -1326,11 +1237,9 @@ retry_write:
blocked_rdev = rrdev;
break;
}
- if (rdev && (test_bit(Faulty, &rdev->flags)
- || test_bit(Unmerged, &rdev->flags)))
+ if (rdev && (test_bit(Faulty, &rdev->flags)))
rdev = NULL;
- if (rrdev && (test_bit(Faulty, &rrdev->flags)
- || test_bit(Unmerged, &rrdev->flags)))
+ if (rrdev && (test_bit(Faulty, &rrdev->flags)))
rrdev = NULL;
r10_bio->devs[i].bio = NULL;
@@ -1777,7 +1686,6 @@ static int raid10_add_disk(struct mddev *mddev, struct md_rdev *rdev)
int mirror;
int first = 0;
int last = conf->geo.raid_disks - 1;
- struct request_queue *q = bdev_get_queue(rdev->bdev);
if (mddev->recovery_cp < MaxSector)
/* only hot-add to in-sync arrays, as recovery is
@@ -1790,11 +1698,6 @@ static int raid10_add_disk(struct mddev *mddev, struct md_rdev *rdev)
if (rdev->raid_disk >= 0)
first = last = rdev->raid_disk;
- if (q->merge_bvec_fn) {
- set_bit(Unmerged, &rdev->flags);
- mddev->merge_check_needed = 1;
- }
-
if (rdev->saved_raid_disk >= first &&
conf->mirrors[rdev->saved_raid_disk].rdev == NULL)
mirror = rdev->saved_raid_disk;
@@ -1833,19 +1736,6 @@ static int raid10_add_disk(struct mddev *mddev, struct md_rdev *rdev)
rcu_assign_pointer(p->rdev, rdev);
break;
}
- if (err == 0 && test_bit(Unmerged, &rdev->flags)) {
- /* Some requests might not have seen this new
- * merge_bvec_fn. We must wait for them to complete
- * before merging the device fully.
- * First we make sure any code which has tested
- * our function has submitted the request, then
- * we wait for all outstanding requests to complete.
- */
- synchronize_sched();
- freeze_array(conf, 0);
- unfreeze_array(conf);
- clear_bit(Unmerged, &rdev->flags);
- }
md_integrity_add_rdev(rdev, mddev);
if (mddev->queue && blk_queue_discard(bdev_get_queue(rdev->bdev)))
queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, mddev->queue);
@@ -2394,7 +2284,6 @@ static void fix_read_error(struct r10conf *conf, struct mddev *mddev, struct r10
d = r10_bio->devs[sl].devnum;
rdev = rcu_dereference(conf->mirrors[d].rdev);
if (rdev &&
- !test_bit(Unmerged, &rdev->flags) &&
test_bit(In_sync, &rdev->flags) &&
is_badblock(rdev, r10_bio->devs[sl].addr + sect, s,
&first_bad, &bad_sectors) == 0) {
@@ -2448,7 +2337,6 @@ static void fix_read_error(struct r10conf *conf, struct mddev *mddev, struct r10
d = r10_bio->devs[sl].devnum;
rdev = rcu_dereference(conf->mirrors[d].rdev);
if (!rdev ||
- test_bit(Unmerged, &rdev->flags) ||
!test_bit(In_sync, &rdev->flags))
continue;
@@ -3642,8 +3530,6 @@ static int run(struct mddev *mddev)
disk->rdev = rdev;
}
q = bdev_get_queue(rdev->bdev);
- if (q->merge_bvec_fn)
- mddev->merge_check_needed = 1;
diff = (rdev->new_data_offset - rdev->data_offset);
if (!mddev->reshape_backwards)
diff = -diff;
@@ -4697,7 +4583,6 @@ static struct md_personality raid10_personality =
.start_reshape = raid10_start_reshape,
.finish_reshape = raid10_finish_reshape,
.congested = raid10_congested,
- .mergeable_bvec = raid10_mergeable_bvec,
};
static int __init raid_init(void)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 8bdf81a..5f938c0 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -4662,35 +4662,6 @@ static int raid5_congested(struct mddev *mddev, int bits)
return 0;
}
-/* We want read requests to align with chunks where possible,
- * but write requests don't need to.
- */
-static int raid5_mergeable_bvec(struct mddev *mddev,
- struct bvec_merge_data *bvm,
- struct bio_vec *biovec)
-{
- sector_t sector = bvm->bi_sector + get_start_sect(bvm->bi_bdev);
- int max;
- unsigned int chunk_sectors = mddev->chunk_sectors;
- unsigned int bio_sectors = bvm->bi_size >> 9;
-
- /*
- * always allow writes to be mergeable, read as well if array
- * is degraded as we'll go through stripe cache anyway.
- */
- if ((bvm->bi_rw & 1) == WRITE || mddev->degraded)
- return biovec->bv_len;
-
- if (mddev->new_chunk_sectors < mddev->chunk_sectors)
- chunk_sectors = mddev->new_chunk_sectors;
- max = (chunk_sectors - ((sector & (chunk_sectors - 1)) + bio_sectors)) << 9;
- if (max < 0) max = 0;
- if (max <= biovec->bv_len && bio_sectors == 0)
- return biovec->bv_len;
- else
- return max;
-}
-
static int in_chunk_boundary(struct mddev *mddev, struct bio *bio)
{
sector_t sector = bio->bi_iter.bi_sector + get_start_sect(bio->bi_bdev);
@@ -7763,7 +7734,6 @@ static struct md_personality raid6_personality =
.quiesce = raid5_quiesce,
.takeover = raid6_takeover,
.congested = raid5_congested,
- .mergeable_bvec = raid5_mergeable_bvec,
};
static struct md_personality raid5_personality =
{
@@ -7787,7 +7757,6 @@ static struct md_personality raid5_personality =
.quiesce = raid5_quiesce,
.takeover = raid5_takeover,
.congested = raid5_congested,
- .mergeable_bvec = raid5_mergeable_bvec,
};
static struct md_personality raid4_personality =
@@ -7812,7 +7781,6 @@ static struct md_personality raid4_personality =
.quiesce = raid5_quiesce,
.takeover = raid4_takeover,
.congested = raid5_congested,
- .mergeable_bvec = raid5_mergeable_bvec,
};
static int __init raid5_init(void)
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index dc89cc8..56cf082 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -213,14 +213,6 @@ typedef int (prep_rq_fn) (struct request_queue *, struct request *);
typedef void (unprep_rq_fn) (struct request_queue *, struct request *);
struct bio_vec;
-struct bvec_merge_data {
- struct block_device *bi_bdev;
- sector_t bi_sector;
- unsigned bi_size;
- unsigned long bi_rw;
-};
-typedef int (merge_bvec_fn) (struct request_queue *, struct bvec_merge_data *,
- struct bio_vec *);
typedef void (softirq_done_fn)(struct request *);
typedef int (dma_drain_needed_fn)(struct request *);
typedef int (lld_busy_fn) (struct request_queue *q);
@@ -305,7 +297,6 @@ struct request_queue {
make_request_fn *make_request_fn;
prep_rq_fn *prep_rq_fn;
unprep_rq_fn *unprep_rq_fn;
- merge_bvec_fn *merge_bvec_fn;
softirq_done_fn *softirq_done_fn;
rq_timed_out_fn *rq_timed_out_fn;
dma_drain_needed_fn *dma_drain_needed;
@@ -991,7 +982,6 @@ extern void blk_queue_lld_busy(struct request_queue *q, lld_busy_fn *fn);
extern void blk_queue_segment_boundary(struct request_queue *, unsigned long);
extern void blk_queue_prep_rq(struct request_queue *, prep_rq_fn *pfn);
extern void blk_queue_unprep_rq(struct request_queue *, unprep_rq_fn *ufn);
-extern void blk_queue_merge_bvec(struct request_queue *, merge_bvec_fn *);
extern void blk_queue_dma_alignment(struct request_queue *, int);
extern void blk_queue_update_dma_alignment(struct request_queue *, int);
extern void blk_queue_softirq_done(struct request_queue *, softirq_done_fn *);
diff --git a/include/linux/device-mapper.h b/include/linux/device-mapper.h
index 51cc1de..76d23fa 100644
--- a/include/linux/device-mapper.h
+++ b/include/linux/device-mapper.h
@@ -82,9 +82,6 @@ typedef int (*dm_message_fn) (struct dm_target *ti, unsigned argc, char **argv);
typedef int (*dm_ioctl_fn) (struct dm_target *ti, unsigned int cmd,
unsigned long arg);
-typedef int (*dm_merge_fn) (struct dm_target *ti, struct bvec_merge_data *bvm,
- struct bio_vec *biovec, int max_size);
-
/*
* These iteration functions are typically used to check (and combine)
* properties of underlying devices.
@@ -160,7 +157,6 @@ struct target_type {
dm_status_fn status;
dm_message_fn message;
dm_ioctl_fn ioctl;
- dm_merge_fn merge;
dm_busy_fn busy;
dm_iterate_devices_fn iterate_devices;
dm_io_hints_fn io_hints;
--
1.9.1
^ permalink raw reply related
* [PATCH v5 07/11] md/raid5: get rid of bio_fits_rdev()
From: Ming Lin @ 2015-07-06 7:44 UTC (permalink / raw)
To: linux-kernel
Cc: Christoph Hellwig, Jens Axboe, Kent Overstreet, Dongsu Park,
Mike Snitzer, Ming Lin, Neil Brown, linux-raid, Ming Lin
In-Reply-To: <1436168690-32102-1-git-send-email-mlin@kernel.org>
From: Kent Overstreet <kent.overstreet@gmail.com>
Remove bio_fits_rdev() as sufficient merge_bvec_fn() handling is now
performed by blk_queue_split() in md_make_request().
Cc: Neil Brown <neilb@suse.de>
Cc: linux-raid@vger.kernel.org
Acked-by: NeilBrown <neilb@suse.de>
Signed-off-by: Kent Overstreet <kent.overstreet@gmail.com>
[dpark: add more description in commit message]
Signed-off-by: Dongsu Park <dpark@posteo.net>
Signed-off-by: Ming Lin <ming.l@ssi.samsung.com>
---
drivers/md/raid5.c | 23 +----------------------
1 file changed, 1 insertion(+), 22 deletions(-)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 8377e72..8bdf81a 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -4780,25 +4780,6 @@ static void raid5_align_endio(struct bio *bi, int error)
add_bio_to_retry(raid_bi, conf);
}
-static int bio_fits_rdev(struct bio *bi)
-{
- struct request_queue *q = bdev_get_queue(bi->bi_bdev);
-
- if (bio_sectors(bi) > queue_max_sectors(q))
- return 0;
- blk_recount_segments(q, bi);
- if (bi->bi_phys_segments > queue_max_segments(q))
- return 0;
-
- if (q->merge_bvec_fn)
- /* it's too hard to apply the merge_bvec_fn at this stage,
- * just just give up
- */
- return 0;
-
- return 1;
-}
-
static int raid5_read_one_chunk(struct mddev *mddev, struct bio *raid_bio)
{
struct r5conf *conf = mddev->private;
@@ -4852,11 +4833,9 @@ static int raid5_read_one_chunk(struct mddev *mddev, struct bio *raid_bio)
align_bi->bi_bdev = rdev->bdev;
__clear_bit(BIO_SEG_VALID, &align_bi->bi_flags);
- if (!bio_fits_rdev(align_bi) ||
- is_badblock(rdev, align_bi->bi_iter.bi_sector,
+ if (is_badblock(rdev, align_bi->bi_iter.bi_sector,
bio_sectors(align_bi),
&first_bad, &bad_sectors)) {
- /* too big in some way, or has a known bad block */
bio_put(align_bi);
rdev_dec_pending(rdev, mddev);
return 0;
--
1.9.1
^ permalink raw reply related
* [PATCH v5 06/11] md/raid5: split bio for chunk_aligned_read
From: Ming Lin @ 2015-07-06 7:44 UTC (permalink / raw)
To: linux-kernel
Cc: Christoph Hellwig, Jens Axboe, Kent Overstreet, Dongsu Park,
Mike Snitzer, Ming Lin, Ming Lin, Neil Brown, linux-raid
In-Reply-To: <1436168690-32102-1-git-send-email-mlin@kernel.org>
From: Ming Lin <ming.l@ssi.samsung.com>
If a read request fits entirely in a chunk, it will be passed directly to the
underlying device (providing it hasn't failed of course). If it doesn't fit,
the slightly less efficient path that uses the stripe_cache is used.
Requests that get to the stripe cache are always completely split up as
necessary.
So with RAID5, ripping out the merge_bvec_fn doesn't cause it to stop work,
but could cause it to take the less efficient path more often.
All that is needed to manage this is for 'chunk_aligned_read' do some bio
splitting, much like the RAID0 code does.
Cc: Neil Brown <neilb@suse.de>
Cc: linux-raid@vger.kernel.org
Acked-by: NeilBrown <neilb@suse.de>
Signed-off-by: Ming Lin <ming.l@ssi.samsung.com>
---
drivers/md/raid5.c | 37 ++++++++++++++++++++++++++++++++-----
1 file changed, 32 insertions(+), 5 deletions(-)
diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c
index 59e44e9..8377e72 100644
--- a/drivers/md/raid5.c
+++ b/drivers/md/raid5.c
@@ -4799,7 +4799,7 @@ static int bio_fits_rdev(struct bio *bi)
return 1;
}
-static int chunk_aligned_read(struct mddev *mddev, struct bio * raid_bio)
+static int raid5_read_one_chunk(struct mddev *mddev, struct bio *raid_bio)
{
struct r5conf *conf = mddev->private;
int dd_idx;
@@ -4808,7 +4808,7 @@ static int chunk_aligned_read(struct mddev *mddev, struct bio * raid_bio)
sector_t end_sector;
if (!in_chunk_boundary(mddev, raid_bio)) {
- pr_debug("chunk_aligned_read : non aligned\n");
+ pr_debug("%s: non aligned\n", __func__);
return 0;
}
/*
@@ -4885,6 +4885,31 @@ static int chunk_aligned_read(struct mddev *mddev, struct bio * raid_bio)
}
}
+static struct bio *chunk_aligned_read(struct mddev *mddev, struct bio *raid_bio)
+{
+ struct bio *split;
+
+ do {
+ sector_t sector = raid_bio->bi_iter.bi_sector;
+ unsigned chunk_sects = mddev->chunk_sectors;
+ unsigned sectors = chunk_sects - (sector & (chunk_sects-1));
+
+ if (sectors < bio_sectors(raid_bio)) {
+ split = bio_split(raid_bio, sectors, GFP_NOIO, fs_bio_set);
+ bio_chain(split, raid_bio);
+ } else
+ split = raid_bio;
+
+ if (!raid5_read_one_chunk(mddev, split)) {
+ if (split != raid_bio)
+ generic_make_request(raid_bio);
+ return split;
+ }
+ } while (split != raid_bio);
+
+ return NULL;
+}
+
/* __get_priority_stripe - get the next stripe to process
*
* Full stripe writes are allowed to pass preread active stripes up until
@@ -5162,9 +5187,11 @@ static void make_request(struct mddev *mddev, struct bio * bi)
* data on failed drives.
*/
if (rw == READ && mddev->degraded == 0 &&
- mddev->reshape_position == MaxSector &&
- chunk_aligned_read(mddev,bi))
- return;
+ mddev->reshape_position == MaxSector) {
+ bi = chunk_aligned_read(mddev, bi);
+ if (!bi)
+ return;
+ }
if (unlikely(bi->bi_rw & REQ_DISCARD)) {
make_discard_request(mddev, bi);
--
1.9.1
^ permalink raw reply related
* Re[2]: Issue removing failed drive and re adding on raid 6
From: Justin Stephenson @ 2015-07-06 0:15 UTC (permalink / raw)
To: Mikael Abrahamsson, Wols Lists; +Cc: linux-raid
In-Reply-To: <alpine.DEB.2.02.1507041009040.11810@uplift.swm.pp.se>
>I recommend this as minimum (in rc.local for instance):
>
>for x in /sys/block/sd[a-z] ; do
> echo 180 > $x/device/timeout
>done
>
>echo 4096 > /sys/block/md0/md/stripe_cache_size
>
>
Done. I will see how this nets out for me here. I will work on switching
over to enterprise or nas rated drives with proper error timing. In the
meantime, I will keep my LTO backup up to date.
Thanks for your help.
- Justin
>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox