[GIT PULL] Core block IO bits for 3.15-rc

[GIT PULL] Core block IO bits for 3.15-rc

[Jens Axboe]

Hi Linus,

This is the pull request for the core block IO bits for the 3.15 kernel.
It's a smaller round this time, it contains:

- Various little blk-mq fixes and additions from Christoph and myself.

- Cleanup of the IPI usage from the block layer, and associated helper
  code. From Frederic Weisbecker and Jan Kara.

- Duplicate code cleanup in bio-integrity from Gu Zheng. This will give
  you a merge conflict, but that should be easy to resolve.

- blk-mq notify spinlock fix for RT from Mike Galbraith.

- A blktrace partial accounting bug fix from Roman Pen.

- Missing REQ_SYNC detection fix for blk-mq from Shaohua Li.


Please pull!


  git://git.kernel.dk/linux-block.git for-3.15/core

for you to fetch changes up to 27fbf4e87c16bb3e40730890169a643a494b7c64:

  blk-mq: add REQ_SYNC early (2014-03-21 08:57:58 -0600)

----------------------------------------------------------------
Christoph Hellwig (3):
      blk-mq: remove blk_mq_alloc_rq
      blk-mq: merge blk_mq_insert_request and blk_mq_run_request
      blk-mq: support partial I/O completions

Frederic Weisbecker (6):
      block: Remove useless IPI struct initialization
      smp: Consolidate the various smp_call_function_single() declensions
      smp: Move __smp_call_function_single() below its safe version
      watchdog: Simplify a little the IPI call
      smp: Remove wait argument from __smp_call_function_single()
      smp: Rename __smp_call_function_single() to smp_call_function_single_async()

Gideon Israel Dsouza (1):
      block: Use macros from compiler.h instead of __attribute__((...))

Gu Zheng (1):
      fs/bio-integrity: remove duplicate code

Jan Kara (5):
      block: Stop abusing csd.list for fifo_time
      block: Stop abusing rq->csd.list in blk-softirq
      smp: Iterate functions through llist_for_each_entry_safe()
      smp: Remove unused list_head from csd
      smp: Teach __smp_call_function_single() to check for offline cpus

Jens Axboe (4):
      block: remove old blk_iopoll_enabled variable
      blk-mq: allow blk_mq_init_commands() to return failure
      blk-mq: fix wrong usage of hctx->state vs hctx->flags
      blk-mq: don't dump CPU -> hw queue map on driver load

Mike Galbraith (1):
      rt,blk,mq: Make blk_mq_cpu_notify_lock a raw spinlock

Mikulas Patocka (1):
      bio: don't write "bio: create slab" messages to syslog

Paul E. McKenney (1):
      block: Substitute rcu_access_pointer() for rcu_dereference_raw()

Roman Pen (1):
      blktrace: fix accounting of partially completed requests

Shaohua Li (1):
      blk-mq: add REQ_SYNC early

 block/blk-cgroup.c              |   2 +-
 block/blk-core.c                |   2 +-
 block/blk-exec.c                |   2 +-
 block/blk-flush.c               |   4 +-
 block/blk-ioc.c                 |   2 +-
 block/blk-iopoll.c              |   3 -
 block/blk-mq-cpu.c              |  14 +--
 block/blk-mq-cpumap.c           |  10 --
 block/blk-mq-sysfs.c            |  31 ++++++
 block/blk-mq.c                  | 172 +++++++++++++++------------------
 block/blk-mq.h                  |   3 +-
 block/blk-softirq.c             |  19 ++--
 block/cfq-iosched.c             |   8 +-
 block/deadline-iosched.c        |   8 +-
 block/partitions/atari.h        |   4 +-
 block/partitions/efi.h          |   9 +-
 block/partitions/karma.c        |   3 +-
 drivers/block/virtio_blk.c      |   3 +-
 drivers/cpuidle/coupled.c       |   2 +-
 drivers/scsi/be2iscsi/be_main.c | 206 ++++++++++++----------------------------
 drivers/scsi/ipr.c              |  15 +--
 fs/bio-integrity.c              |  86 ++++++++---------
 fs/bio.c                        |   1 -
 include/linux/blk-iopoll.h      |   2 -
 include/linux/blk-mq.h          |   8 +-
 include/linux/blkdev.h          |   1 +
 include/linux/elevator.h        |  11 +--
 include/linux/smp.h             |   8 +-
 include/trace/events/block.h    |  33 ++++++-
 kernel/sched/core.c             |   2 +-
 kernel/smp.c                    | 139 +++++++++++++--------------
 kernel/sysctl.c                 |  12 ---
 kernel/trace/blktrace.c         |  20 ++--
 kernel/up.c                     |   6 +-
 kernel/watchdog.c               |   3 +-
 net/core/dev.c                  |   4 +-
 36 files changed, 383 insertions(+), 475 deletions(-)

-- 
Jens Axboe

Re: [GIT PULL] Core block IO bits for 3.15-rc

[Linus Torvalds]

On Tue, Apr 1, 2014 at 12:05 PM, Jens Axboe <axboe@fb.com> wrote:
>
> - Cleanup of the IPI usage from the block layer, and associated helper
>   code. From Frederic Weisbecker and Jan Kara.

So I absolutely *hate* how this was done.

Why the hell was it mixed in with the block layer code? It's not even
in some clean separate branch, it's just randomly in the middle of the
block code, for no obvious reason.

I'm pulling it this time, but quite frankly, next time I see this kind
of ugly AND TOTALLY POINTLESS layering violation, I will just drop the
stupid pull request.

If you want to push me cleanups that are to generic code and are in no
way specific to the block layer, fine. But I want a separate pull
request that is not in any way mixed up with block code.

In other words, this was NOT OK. This was stupid and wrong, and
violated all sanity. I can see absolutely no reason why that
smp_call_function_single_async() renaming and the other cleanups are
in the block branch. They are totally separate in every single way.

What the hell was going on here?

                  Linus

Re: [GIT PULL] Core block IO bits for 3.15-rc

[Jens Axboe]

On 2014-04-01 20:43, Linus Torvalds wrote:
> On Tue, Apr 1, 2014 at 12:05 PM, Jens Axboe <axboe@fb.com> wrote:
>>
>> - Cleanup of the IPI usage from the block layer, and associated helper
>>    code. From Frederic Weisbecker and Jan Kara.
>
> So I absolutely *hate* how this was done.
>
> Why the hell was it mixed in with the block layer code? It's not even
> in some clean separate branch, it's just randomly in the middle of the
> block code, for no obvious reason.
>
> I'm pulling it this time, but quite frankly, next time I see this kind
> of ugly AND TOTALLY POINTLESS layering violation, I will just drop the
> stupid pull request.
>
> If you want to push me cleanups that are to generic code and are in no
> way specific to the block layer, fine. But I want a separate pull
> request that is not in any way mixed up with block code.
>
> In other words, this was NOT OK. This was stupid and wrong, and
> violated all sanity. I can see absolutely no reason why that
> smp_call_function_single_async() renaming and the other cleanups are
> in the block branch. They are totally separate in every single way.
>
> What the hell was going on here?

I agree, it should have been in its own topic branch. No real arguments 
there. And probably should have been split further, so only the block 
required parts ended up in there, and not eg the watchdog parts.

-- 
Jens Axboe

Re: [GIT PULL] Core block IO bits for 3.15-rc

[Frederic Weisbecker]

On Tue, Apr 01, 2014 at 08:48:48PM -0600, Jens Axboe wrote:
> On 2014-04-01 20:43, Linus Torvalds wrote:
> >On Tue, Apr 1, 2014 at 12:05 PM, Jens Axboe <axboe@fb.com> wrote:
> >>
> >>- Cleanup of the IPI usage from the block layer, and associated helper
> >>   code. From Frederic Weisbecker and Jan Kara.
> >
> >So I absolutely *hate* how this was done.
> >
> >Why the hell was it mixed in with the block layer code? It's not even
> >in some clean separate branch, it's just randomly in the middle of the
> >block code, for no obvious reason.
> >
> >I'm pulling it this time, but quite frankly, next time I see this kind
> >of ugly AND TOTALLY POINTLESS layering violation, I will just drop the
> >stupid pull request.
> >
> >If you want to push me cleanups that are to generic code and are in no
> >way specific to the block layer, fine. But I want a separate pull
> >request that is not in any way mixed up with block code.
> >
> >In other words, this was NOT OK. This was stupid and wrong, and
> >violated all sanity. I can see absolutely no reason why that
> >smp_call_function_single_async() renaming and the other cleanups are
> >in the block branch. They are totally separate in every single way.
> >
> >What the hell was going on here?
> 
> I agree, it should have been in its own topic branch. No real
> arguments there. And probably should have been split further, so
> only the block required parts ended up in there, and not eg the
> watchdog parts.

I think that having block and watchdog changes, among many other non-core SMP
changes, in the SMP branch is still unavoidable. If we change a core API,
we must convert the users all along. Of course such peripheral changes should
be strictly limited to these API conversion.

But then the right approach would have been to keep this branch standalone and
deal with conflictings changes like block bits once they are pulled in Linus tree.

Anyway sorry again, I initiated the mistake by making these smp changes on top
of the block bits.

Re: [GIT PULL] Core block IO bits for 3.15-rc

[Frederic Weisbecker]

On Tue, Apr 01, 2014 at 07:43:08PM -0700, Linus Torvalds wrote:
> On Tue, Apr 1, 2014 at 12:05 PM, Jens Axboe <axboe@fb.com> wrote:
> >
> > - Cleanup of the IPI usage from the block layer, and associated helper
> >   code. From Frederic Weisbecker and Jan Kara.
> 
> So I absolutely *hate* how this was done.
> 
> Why the hell was it mixed in with the block layer code? It's not even
> in some clean separate branch, it's just randomly in the middle of the
> block code, for no obvious reason.
> 
> I'm pulling it this time, but quite frankly, next time I see this kind
> of ugly AND TOTALLY POINTLESS layering violation, I will just drop the
> stupid pull request.
> 
> If you want to push me cleanups that are to generic code and are in no
> way specific to the block layer, fine. But I want a separate pull
> request that is not in any way mixed up with block code.
> 
> In other words, this was NOT OK. This was stupid and wrong, and
> violated all sanity. I can see absolutely no reason why that
> smp_call_function_single_async() renaming and the other cleanups are
> in the block branch. They are totally separate in every single way.
> 
> What the hell was going on here?

I think it's because we were worried that these smp cleanups had strong
conflicts against block changes.

Changes on core SMP APIs resulted in changes on these APIs users (ie: block
for the most part here). But then these changes in API users had to face
quite some conflicting changes in the block code.

So the mistake started with me: I did a branch with the smp changes on top of the
block core in order to resolve the conflict early.

Then Jens pulled it in the block tree. Then again it looked right to me as further
block changes would then have no risk of conflict, and Jens is the maintainer of
both parts.

So yeah that's because I was worried about strong conflicts. What kind of approach
do you prefer then to solve that kind of issue? Do you prefer that we create a seperate
branch and deal with non trivial nor small conflicts on merge window time?

Yeah now that I think about it, maybe when the conflicts are significant we should
have a seperate SMP branch which has the conflicts resolved once the block bits are pulled in.

Re: [GIT PULL] Core block IO bits for 3.15-rc

[Linus Torvalds]

On Wed, Apr 2, 2014 at 7:00 AM, Frederic Weisbecker <fweisbec@gmail.com> wrote:
>
> So yeah that's because I was worried about strong conflicts. What kind of approach
> do you prefer then to solve that kind of issue? Do you prefer that we create a seperate
> branch and deal with non trivial nor small conflicts on merge window time?

I'd indeed rather see a separate branch, and deal with the conflicts.

And in fact I think you over-estimate the conflicts. The smp function
naming changes were trivial as far as outside users were concerned,
and while the "stop abusing fileds in csd" might have clashed more
with the rest of the block changes (because they were actually to the
block functions), I doubt it would have been painful. In fact, looking
at "fifo_time" there should be no conflicts at all, and the queuelist
changes look like they would have had a *trivial* conflict with
"blk-mq: merge blk_mq_insert_request and blk_mq_run_request" just
because there were changes nearby. Even that is debatable - it's
possible git would just have resolved that one automatically too.

So I think that the patches from you and Honza could easily have been
in another branch, and had trivial or no conflicts with the other
block changes.

                Linus

Re: [GIT PULL] Core block IO bits for 3.15-rc

[Frederic Weisbecker]

On Wed, Apr 02, 2014 at 08:02:13AM -0700, Linus Torvalds wrote:
> On Wed, Apr 2, 2014 at 7:00 AM, Frederic Weisbecker <fweisbec@gmail.com> wrote:
> >
> > So yeah that's because I was worried about strong conflicts. What kind of approach
> > do you prefer then to solve that kind of issue? Do you prefer that we create a seperate
> > branch and deal with non trivial nor small conflicts on merge window time?
> 
> I'd indeed rather see a separate branch, and deal with the conflicts.
> 
> And in fact I think you over-estimate the conflicts. The smp function
> naming changes were trivial as far as outside users were concerned,
> and while the "stop abusing fileds in csd" might have clashed more
> with the rest of the block changes (because they were actually to the
> block functions), I doubt it would have been painful. In fact, looking
> at "fifo_time" there should be no conflicts at all, and the queuelist
> changes look like they would have had a *trivial* conflict with
> "blk-mq: merge blk_mq_insert_request and blk_mq_run_request" just
> because there were changes nearby. Even that is debatable - it's
> possible git would just have resolved that one automatically too.
> 
> So I think that the patches from you and Honza could easily have been
> in another branch, and had trivial or no conflicts with the other
> block changes.
> 
>                 Linus

Yeah indeed. I think maybe I started to work on top of a stale tree and got
confused with conflicts against pre v3.13 commits that were actually merged
upstream for a while already.

But you're right, looking at it closer, the real conflicts against pending -block
patches weren't that bad actually

Anyway, thanks for pulling it in the end, I'll be more careful!