[GIT PULL] core block IO bits for 3.8

[GIT PULL] core block IO bits for 3.8

[Jens Axboe]

Hi Linus,

Here are the core block IO bits for 3.8. The branch contains:

- The final version of the surprise device removal fixups from Bart.

- Don't hide EFI partitions under advanced partition types. It's fairly
  wide spread these days. This is especially dangerous for systems that
  have both msdos and efi partition tables, where you want to keep them
  in sync.

- Cleanup of using -1 instead of the proper NUMA_NO_NODE

- Export control of bdi flusher thread CPU mask and default to using the
  home node (if known) from Jeff.

- Export unplug tracepoint for MD.

- Core improvements from Shaohua. Reinstate the recursive merge, as the
  original bug has been fixed. Add plugging for discard and also fix a
  problem handling non pow-of-2 discard limits.

Please pull! There's a trivial merge in block/blk-exec.c due to a fix
that went into 3.7-rc at a later point than -rc4 where this is based.


  git://git.kernel.dk/linux-block.git for-3.8/core


----------------------------------------------------------------
Bart Van Assche (6):
      block: Rename queue dead flag
      block: Let blk_drain_queue() caller obtain the queue lock
      block: Avoid that request_fn is invoked on a dead queue
      block: Avoid scheduling delayed work on a dead queue
      block: Make blk_cleanup_queue() wait until request_fn finished
      bsg: Remove unused function bsg_goose_queue()

Diego Calleja (1):
      partitions: enable EFI/GPT support by default

Ezequiel Garcia (1):
      block: use NUMA_NO_NODE instead of -1

Jeff Moyer (1):
      bdi: add a user-tunable cpu_list for the bdi flusher threads

NeilBrown (1):
      block: export block_unplug tracepoint

Shaohua Li (4):
      block CFQ: avoid moving request to different queue
      block: recursive merge requests
      block: discard granularity might not be power of 2
      block: add plug for blkdev_issue_discard

xiaobing tu (1):
      deadline: Allow 0ms deadline latency, increase the read speed

 block/blk-cgroup.c          |    2 +-
 block/blk-core.c            |  127 ++++++++++++++++++++++++++-----------------
 block/blk-exec.c            |    4 +-
 block/blk-lib.c             |   26 +++++----
 block/blk-settings.c        |    6 +-
 block/blk-sysfs.c           |    4 +-
 block/blk-throttle.c        |    2 +-
 block/blk.h                 |    4 +-
 block/bsg-lib.c             |   13 -----
 block/cfq-iosched.c         |    3 +-
 block/deadline-iosched.c    |    2 +-
 block/elevator.c            |   16 ++++--
 block/genhd.c               |    2 +-
 block/partitions/Kconfig    |    4 +-
 drivers/scsi/scsi_lib.c     |    2 +-
 include/linux/backing-dev.h |    4 ++
 include/linux/blkdev.h      |   17 ++++--
 include/linux/bsg-lib.h     |    1 -
 mm/backing-dev.c            |   84 ++++++++++++++++++++++++++++
 19 files changed, 224 insertions(+), 99 deletions(-)

-- 
Jens Axboe

Re: [GIT PULL] core block IO bits for 3.8

[Jeff Moyer]

Jens Axboe <axboe@kernel.dk> writes:

> - Export control of bdi flusher thread CPU mask and default to using the
>   home node (if known) from Jeff.

I certainly wouldn't mind this going in, but I think Tejun NACKed this
patch.  Tejun?

Cheers,
Jeff

Re: [GIT PULL] core block IO bits for 3.8

[Tejun Heo]

On Mon, Dec 17, 2012 at 11:06:39AM -0500, Jeff Moyer wrote:
> Jens Axboe <axboe@kernel.dk> writes:
> 
> > - Export control of bdi flusher thread CPU mask and default to using the
> >   home node (if known) from Jeff.
> 
> I certainly wouldn't mind this going in, but I think Tejun NACKed this
> patch.  Tejun?

Hmm... yeah, I would much prefer if this doesn't go in this time.  I
think we need something more generic for threadpools w/ scheduling
restrictions.  It would be possible to retrofit so that the interface
eventually matches but given that this feature isn't an immediate
must-have thing, I think it would be better to not do this right now.

Thanks.

-- 
tejun

Re: [GIT PULL] core block IO bits for 3.8

[Jens Axboe]

On 2012-12-17 18:00, Tejun Heo wrote:
> On Mon, Dec 17, 2012 at 11:06:39AM -0500, Jeff Moyer wrote:
>> Jens Axboe <axboe@kernel.dk> writes:
>>
>>> - Export control of bdi flusher thread CPU mask and default to using the
>>>   home node (if known) from Jeff.
>>
>> I certainly wouldn't mind this going in, but I think Tejun NACKed this
>> patch.  Tejun?
> 
> Hmm... yeah, I would much prefer if this doesn't go in this time.  I
> think we need something more generic for threadpools w/ scheduling
> restrictions.  It would be possible to retrofit so that the interface
> eventually matches but given that this feature isn't an immediate
> must-have thing, I think it would be better to not do this right now.

OK, if you have something generic for threadpools in mind, we can wait
with this one. I just thought that a "real" interface was nicer than
manual setting, especially since the flusher threads come and go. Then
you'd need some udev hook to get it set, not even sure how easy that
would be with the weird linkage.

Jeff, lets default to the node mask at least. Can you resend a patch
that doesn't add the API, but just defaults to setting the affinity to
the cpu mask of the home node?

Linus, how do you want to handle this? I can do a revert and re-send a
pull.

-- 
Jens Axboe

Re: [GIT PULL] core block IO bits for 3.8

[Linus Torvalds]

On Mon, Dec 17, 2012 at 9:04 AM, Jens Axboe <axboe@kernel.dk> wrote:
>
> Linus, how do you want to handle this? I can do a revert and re-send a
> pull.

Well, I already did the pull. Can we just stick in a commit that hides
the API while this gets discussed/fixes?

             Linus

Re: [GIT PULL] core block IO bits for 3.8

[Jens Axboe]

On 2012-12-17 18:08, Linus Torvalds wrote:
> On Mon, Dec 17, 2012 at 9:04 AM, Jens Axboe <axboe@kernel.dk> wrote:
>>
>> Linus, how do you want to handle this? I can do a revert and re-send a
>> pull.
> 
> Well, I already did the pull. Can we just stick in a commit that hides
> the API while this gets discussed/fixes?

Sure, works for me. Then we'll either yank it for 3.8 final, or adopt.
Whatever fits the timeframe.

I'll send you a pull later.

-- 
Jens Axboe

Re: [GIT PULL] core block IO bits for 3.8

[Jeff Moyer]

Jens Axboe <axboe@kernel.dk> writes:

> On 2012-12-17 18:00, Tejun Heo wrote:
>> On Mon, Dec 17, 2012 at 11:06:39AM -0500, Jeff Moyer wrote:
>>> Jens Axboe <axboe@kernel.dk> writes:
>>>
>>>> - Export control of bdi flusher thread CPU mask and default to using the
>>>>   home node (if known) from Jeff.
>>>
>>> I certainly wouldn't mind this going in, but I think Tejun NACKed this
>>> patch.  Tejun?
>> 
>> Hmm... yeah, I would much prefer if this doesn't go in this time.  I
>> think we need something more generic for threadpools w/ scheduling
>> restrictions.  It would be possible to retrofit so that the interface
>> eventually matches but given that this feature isn't an immediate
>> must-have thing, I think it would be better to not do this right now.
>
> OK, if you have something generic for threadpools in mind, we can wait
> with this one. I just thought that a "real" interface was nicer than
> manual setting, especially since the flusher threads come and go. Then
> you'd need some udev hook to get it set, not even sure how easy that
> would be with the weird linkage.
>
> Jeff, lets default to the node mask at least. Can you resend a patch
> that doesn't add the API, but just defaults to setting the affinity to
> the cpu mask of the home node?

Well, it turns out that the numa node is always -1, since bdi devices
always have a null parent.  I have a patch set to clean that up, but it
depends on my other numa patches in the scsi area that will hopefully
go in 3.9.

> Linus, how do you want to handle this? I can do a revert and re-send a
> pull.

For now, I think it would be cleanest if we just reverted the change.

Cheers,
Jeff

Re: [GIT PULL] core block IO bits for 3.8

[Jens Axboe]

On 2012-12-17 18:15, Jeff Moyer wrote:
> Jens Axboe <axboe@kernel.dk> writes:
> 
>> On 2012-12-17 18:00, Tejun Heo wrote:
>>> On Mon, Dec 17, 2012 at 11:06:39AM -0500, Jeff Moyer wrote:
>>>> Jens Axboe <axboe@kernel.dk> writes:
>>>>
>>>>> - Export control of bdi flusher thread CPU mask and default to using the
>>>>>   home node (if known) from Jeff.
>>>>
>>>> I certainly wouldn't mind this going in, but I think Tejun NACKed this
>>>> patch.  Tejun?
>>>
>>> Hmm... yeah, I would much prefer if this doesn't go in this time.  I
>>> think we need something more generic for threadpools w/ scheduling
>>> restrictions.  It would be possible to retrofit so that the interface
>>> eventually matches but given that this feature isn't an immediate
>>> must-have thing, I think it would be better to not do this right now.
>>
>> OK, if you have something generic for threadpools in mind, we can wait
>> with this one. I just thought that a "real" interface was nicer than
>> manual setting, especially since the flusher threads come and go. Then
>> you'd need some udev hook to get it set, not even sure how easy that
>> would be with the weird linkage.
>>
>> Jeff, lets default to the node mask at least. Can you resend a patch
>> that doesn't add the API, but just defaults to setting the affinity to
>> the cpu mask of the home node?
> 
> Well, it turns out that the numa node is always -1, since bdi devices
> always have a null parent.  I have a patch set to clean that up, but it
> depends on my other numa patches in the scsi area that will hopefully
> go in 3.9.

Sigh, in that case, yes lets just revert it.

-- 
Jens Axboe

Re: [GIT PULL] core block IO bits for 3.8

[Linus Torvalds]

On Mon, Dec 17, 2012 at 11:06 AM, Jens Axboe <axboe@kernel.dk> wrote:
>
> Sigh, in that case, yes lets just revert it.

So just to check - we're talking about commit 8fa72d234da9 ("bdi: add
a user-tunable cpu_list for the bdi flusher threads"). It reverts
cleanly, but I wanted to make sure we're all on the same page..

            Linus

Re: [GIT PULL] core block IO bits for 3.8

[Jens Axboe]

On 2012-12-17 20:27, Linus Torvalds wrote:
> On Mon, Dec 17, 2012 at 11:06 AM, Jens Axboe <axboe@kernel.dk> wrote:
>>
>> Sigh, in that case, yes lets just revert it.
> 
> So just to check - we're talking about commit 8fa72d234da9 ("bdi: add
> a user-tunable cpu_list for the bdi flusher threads"). It reverts
> cleanly, but I wanted to make sure we're all on the same page..

That is the one, revert at will :-)

-- 
Jens Axboe

Re: [GIT PULL] core block IO bits for 3.8

[Shentino]

On Mon, Dec 17, 2012 at 11:28 AM, Jens Axboe <axboe@kernel.dk> wrote:
> On 2012-12-17 20:27, Linus Torvalds wrote:
>> On Mon, Dec 17, 2012 at 11:06 AM, Jens Axboe <axboe@kernel.dk> wrote:
>>>
>>> Sigh, in that case, yes lets just revert it.
>>
>> So just to check - we're talking about commit 8fa72d234da9 ("bdi: add
>> a user-tunable cpu_list for the bdi flusher threads"). It reverts
>> cleanly, but I wanted to make sure we're all on the same page..

Page, heh.

> That is the one, revert at will :-)
>
> --
> Jens Axboe
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [GIT PULL] core block IO bits for 3.8

[Tejun Heo]

Hello, Jens.

On Mon, Dec 17, 2012 at 06:04:42PM +0100, Jens Axboe wrote:
> OK, if you have something generic for threadpools in mind, we can wait
> with this one. I just thought that a "real" interface was nicer than
> manual setting, especially since the flusher threads come and go. Then
> you'd need some udev hook to get it set, not even sure how easy that
> would be with the weird linkage.

I've been thinking about this and I think what we really need is a
generic interface for thread pools where the user can specify various
attributes - priority, affinity and probably cgroup association, so
that those attributes are exposed in generic manner while backend
creates and manages worker pools identified by actual attriutes so
that we don't end up with gazillion pools.  crypto and writeback would
be the obvious users and probably the virtio too.  I'll write more
when things get more concrete.

Thanks.

-- 
tejun

Re: [GIT PULL] core block IO bits for 3.8

[Jens Axboe]

On 2012-12-17 18:47, Tejun Heo wrote:
> Hello, Jens.
> 
> On Mon, Dec 17, 2012 at 06:04:42PM +0100, Jens Axboe wrote:
>> OK, if you have something generic for threadpools in mind, we can wait
>> with this one. I just thought that a "real" interface was nicer than
>> manual setting, especially since the flusher threads come and go. Then
>> you'd need some udev hook to get it set, not even sure how easy that
>> would be with the weird linkage.
> 
> I've been thinking about this and I think what we really need is a
> generic interface for thread pools where the user can specify various
> attributes - priority, affinity and probably cgroup association, so
> that those attributes are exposed in generic manner while backend
> creates and manages worker pools identified by actual attriutes so
> that we don't end up with gazillion pools.  crypto and writeback would
> be the obvious users and probably the virtio too.  I'll write more
> when things get more concrete.

I agree. That part of the writeback code is pretty generic
functionality.

So new course of action is to revert the commit, and then I'm sure Jeff
will re-send the code to just set the affinity to something sane by
default when that is easily doable.

-- 
Jens Axboe