[GIT PULL] block bits

[GIT PULL] block bits

[Jens Axboe]

Hi Linus,

A small collection of fixes for 2.6.27, please apply.

  git://git.kernel.dk/linux-2.6-block.git for-linus

Adel Gadllah (1):
      block: clean up cmdfilter sysfs interface

FUJITA Tomonori (5):
      block: move cmdfilter from gendisk to request_queue
      sg: restore command permission for TYPE_SCANNER
      block: rename blk_scsi_cmd_filter to blk_cmd_filter
      bio: fix bio_copy_kern() handling of bio->bv_len
      bio: fix __bio_copy_iov() handling of bio->bv_len

Jens Axboe (2):
      block: submit_bh() inadvertently discards barrier flag on a sync write
      block: remove blk_queue_tag_depth() and blk_queue_tag_queue()

Matthew Wilcox (1):
      block: remove unused ->busy part of the block queue tag map

xiphmont@xiph.org (1):
      SG_IO block filter whitelist missing MMC SET READ AHEAD command

 block/blk-core.c       |    2 +
 block/blk-tag.c        |    6 +-
 block/bsg.c            |   44 +++--------
 block/cmd-filter.c     |  196 ++++++++++++------------------------------------
 block/scsi_ioctl.c     |   95 ++++++++++++++++++++++-
 drivers/scsi/sg.c      |   17 ++++-
 fs/bio.c               |   48 ++++++++----
 fs/buffer.c            |   13 ++-
 include/linux/blkdev.h |   18 +++-
 include/linux/genhd.h  |   10 ---
 10 files changed, 224 insertions(+), 225 deletions(-)

-- 
Jens Axboe

Re: [GIT PULL] block bits

[Linus Torvalds]

On Tue, 26 Aug 2008, Jens Axboe wrote:
> 
> I'll move those 1-2 patches to the 2.6.28 branch and resubmit the rest
> for you.

You will *delete* the absolute crap locking patch, that has no way of 
being even 2.6.28 material.

And you need to be a whole lot more careful.

			Linus

Re: [GIT PULL] block bits

[Jens Axboe]

On Tue, Aug 26 2008, Linus Torvalds wrote:
> 
> 
> On Tue, 26 Aug 2008, Jens Axboe wrote:
> > 
> > I'll move those 1-2 patches to the 2.6.28 branch and resubmit the rest
> > for you.
> 
> You will *delete* the absolute crap locking patch, that has no way of 
> being even 2.6.28 material.

Yeah, I thought that was obvious, I'll bounce it back to the Xen folks.

> And you need to be a whole lot more careful.

Noted, I'll stick to the straight and narrow.

-- 
Jens Axboe

Re: [GIT PULL] block bits

[Christoph Hellwig]

On Tue, Aug 26, 2008 at 12:02:18PM -0700, Linus Torvalds wrote:
> 
> 
> On Tue, 26 Aug 2008, Jens Axboe wrote:
> > 
> > I'll move those 1-2 patches to the 2.6.28 branch and resubmit the rest
> > for you.
> 
> You will *delete* the absolute crap locking patch, that has no way of 
> being even 2.6.28 material.
> 
> And you need to be a whole lot more careful.

I can't find the mail you're replying to either in my inbox or on
marc.info.  I also can't really see any xen patch in the rev listing in
Jens' initial mail.  Something got lost her..

Re: [GIT PULL] block bits

[Jeremy Fitzhardinge]

Jens Axboe wrote:
> On Tue, Aug 26 2008, Linus Torvalds wrote:
>   
>> On Tue, 26 Aug 2008, Jens Axboe wrote:
>>     
>>> I'll move those 1-2 patches to the 2.6.28 branch and resubmit the rest
>>> for you.
>>>       
>> You will *delete* the absolute crap locking patch, that has no way of 
>> being even 2.6.28 material.
>>     
>
> Yeah, I thought that was obvious, I'll bounce it back to the Xen folks.
>   

Which, what?  Was there an objection to one of the blkfront patches?  (I
don't remember anything locking related in there anyway.)

    J

Re: [GIT PULL] block bits

[Jens Axboe]

On Tue, Aug 26 2008, Jeremy Fitzhardinge wrote:
> Jens Axboe wrote:
> > On Tue, Aug 26 2008, Linus Torvalds wrote:
> >   
> >> On Tue, 26 Aug 2008, Jens Axboe wrote:
> >>     
> >>> I'll move those 1-2 patches to the 2.6.28 branch and resubmit the rest
> >>> for you.
> >>>       
> >> You will *delete* the absolute crap locking patch, that has no way of 
> >> being even 2.6.28 material.
> >>     
> >
> > Yeah, I thought that was obvious, I'll bounce it back to the Xen folks.
> >   
> 
> Which, what?  Was there an objection to one of the blkfront patches?  (I
> don't remember anything locking related in there anyway.)

I see Linus' initial reply didn't make it to the list, not sure why.
This is what he said:

"it's so _incredibly_ broken that I refuse to pull any of the rest
either, because the queue is obviously utter crap.

In fact, even the "explanation" for that one is shit:

   "It shouldn't matter if an interrupt comes in whilst blkif_io_lock is
    held, as it will block on the lock [...]"

where "as it will block" is apparently shorthand for "as it will
deadlock and kill the machine"."

This is the patch in question:

http://git.kernel.dk/?p=linux-2.6-block.git;a=commitdiff;h=b58e9034c5e2424723948a63712eaf1332a76ab8;hp=5814f8d589a1daef3b1a6f3731fd65a858dc8466

and it doesn indeed look like crap. It DOES matter if an interrupt comes
in on the local CPU while you are holding the blkif_io_lock, you'd be
dead on the spot spinning forever in the irq handler.

-- 
Jens Axboe

[GIT PULL] block bits

[Jens Axboe]

Hi Linus,

One critical bit in here, which fixes a reported oops from the iostat
patch. The others are either trivial/minor or documentation. Please
pull.

  git://git.kernel.dk/linux-2.6-block.git for-linus

Alberto Bertogli (2):
      Fix misleading comment in bio.h
      bio.h: If they MUST be inlined, then use __always_inline

Jens Axboe (3):
      block: fix oops in blk_queue_io_stat()
      block: fix inconsistent parenthesisation of QUEUE_FLAG_DEFAULT
      block: add text file detailing queue/ sysfs files

 Documentation/block/queue-sysfs.txt |   63 +++++++++++++++++++++++++++++++++++
 block/blk-core.c                    |    6 ++--
 block/blk.h                         |    8 ++++
 include/linux/bio.h                 |   10 +++--
 include/linux/blkdev.h              |    2 +-
 5 files changed, 81 insertions(+), 8 deletions(-)
 create mode 100644 Documentation/block/queue-sysfs.txt

diff --git a/Documentation/block/queue-sysfs.txt b/Documentation/block/queue-sysfs.txt
new file mode 100644
index 0000000..e164403
--- /dev/null
+++ b/Documentation/block/queue-sysfs.txt
@@ -0,0 +1,63 @@
+Queue sysfs files
+=================
+
+This text file will detail the queue files that are located in the sysfs tree
+for each block device. Note that stacked devices typically do not export
+any settings, since their queue merely functions are a remapping target.
+These files are the ones found in the /sys/block/xxx/queue/ directory.
+
+Files denoted with a RO postfix are readonly and the RW postfix means
+read-write.
+
+hw_sector_size (RO)
+-------------------
+This is the hardware sector size of the device, in bytes.
+
+max_hw_sectors_kb (RO)
+----------------------
+This is the maximum number of kilobytes supported in a single data transfer.
+
+max_sectors_kb (RW)
+-------------------
+This is the maximum number of kilobytes that the block layer will allow
+for a filesystem request. Must be smaller than or equal to the maximum
+size allowed by the hardware.
+
+nomerges (RW)
+-------------
+This enables the user to disable the lookup logic involved with IO merging
+requests in the block layer. Merging may still occur through a direct
+1-hit cache, since that comes for (almost) free. The IO scheduler will not
+waste cycles doing tree/hash lookups for merges if nomerges is 1. Defaults
+to 0, enabling all merges.
+
+nr_requests (RW)
+----------------
+This controls how many requests may be allocated in the block layer for
+read or write requests. Note that the total allocated number may be twice
+this amount, since it applies only to reads or writes (not the accumulated
+sum).
+
+read_ahead_kb (RW)
+------------------
+Maximum number of kilobytes to read-ahead for filesystems on this block
+device.
+
+rq_affinity (RW)
+----------------
+If this option is enabled, the block layer will migrate request completions
+to the CPU that originally submitted the request. For some workloads
+this provides a significant reduction in CPU cycles due to caching effects.
+
+scheduler (RW)
+--------------
+When read, this file will display the current and available IO schedulers
+for this block device. The currently active IO scheduler will be enclosed
+in [] brackets. Writing an IO scheduler name to this file will switch
+control of this block device to that new IO scheduler. Note that writing
+an IO scheduler name to this file will attempt to load that IO scheduler
+module, if it isn't already present in the system.
+
+
+
+Jens Axboe <jens.axboe@oracle.com>, February 2009
diff --git a/block/blk-core.c b/block/blk-core.c
index ca69f3d..29bcfac 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -69,7 +69,7 @@ static void drive_stat_acct(struct request *rq, int new_io)
 	int rw = rq_data_dir(rq);
 	int cpu;
 
-	if (!blk_fs_request(rq) || !disk || !blk_queue_io_stat(disk->queue))
+	if (!blk_fs_request(rq) || !disk || !blk_do_io_stat(disk->queue))
 		return;
 
 	cpu = part_stat_lock();
@@ -1667,7 +1667,7 @@ static void blk_account_io_completion(struct request *req, unsigned int bytes)
 {
 	struct gendisk *disk = req->rq_disk;
 
-	if (!disk || !blk_queue_io_stat(disk->queue))
+	if (!disk || !blk_do_io_stat(disk->queue))
 		return;
 
 	if (blk_fs_request(req)) {
@@ -1686,7 +1686,7 @@ static void blk_account_io_done(struct request *req)
 {
 	struct gendisk *disk = req->rq_disk;
 
-	if (!disk || !blk_queue_io_stat(disk->queue))
+	if (!disk || !blk_do_io_stat(disk->queue))
 		return;
 
 	/*
diff --git a/block/blk.h b/block/blk.h
index 6e1ed40..0dce92c 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -108,4 +108,12 @@ static inline int blk_cpu_to_group(int cpu)
 #endif
 }
 
+static inline int blk_do_io_stat(struct request_queue *q)
+{
+	if (q)
+		return blk_queue_io_stat(q);
+
+	return 0;
+}
+
 #endif
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 0942765..2aa283a 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -451,12 +451,13 @@ extern struct biovec_slab bvec_slabs[BIOVEC_NR_POOLS] __read_mostly;
 
 #ifdef CONFIG_HIGHMEM
 /*
- * remember to add offset! and never ever reenable interrupts between a
- * bvec_kmap_irq and bvec_kunmap_irq!!
+ * remember never ever reenable interrupts between a bvec_kmap_irq and
+ * bvec_kunmap_irq!
  *
  * This function MUST be inlined - it plays with the CPU interrupt flags.
  */
-static inline char *bvec_kmap_irq(struct bio_vec *bvec, unsigned long *flags)
+static __always_inline char *bvec_kmap_irq(struct bio_vec *bvec,
+		unsigned long *flags)
 {
 	unsigned long addr;
 
@@ -472,7 +473,8 @@ static inline char *bvec_kmap_irq(struct bio_vec *bvec, unsigned long *flags)
 	return (char *) addr + bvec->bv_offset;
 }
 
-static inline void bvec_kunmap_irq(char *buffer, unsigned long *flags)
+static __always_inline void bvec_kunmap_irq(char *buffer,
+		unsigned long *flags)
 {
 	unsigned long ptr = (unsigned long) buffer & PAGE_MASK;
 
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index d08c4b8..dcaa0fd 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -455,7 +455,7 @@ struct request_queue
 
 #define QUEUE_FLAG_DEFAULT	((1 << QUEUE_FLAG_IO_STAT) |		\
 				 (1 << QUEUE_FLAG_CLUSTER) |		\
-				  1 << QUEUE_FLAG_STACKABLE)
+				 (1 << QUEUE_FLAG_STACKABLE))
 
 static inline int queue_is_locked(struct request_queue *q)
 {

-- 
Jens Axboe

Re: [GIT PULL] block bits

[Boaz Harrosh]

Jens Axboe wrote:
> Hi Linus,
> 
> One critical bit in here, which fixes a reported oops from the iostat
> patch. The others are either trivial/minor or documentation. Please
> pull.
> 
>   git://git.kernel.dk/linux-2.6-block.git for-linus
> 
> Alberto Bertogli (2):
>       Fix misleading comment in bio.h
>       bio.h: If they MUST be inlined, then use __always_inline
> 
> Jens Axboe (3):
>       block: fix oops in blk_queue_io_stat()
>       block: fix inconsistent parenthesisation of QUEUE_FLAG_DEFAULT
>       block: add text file detailing queue/ sysfs files
> 

Hi Jens.

The bsg sense handling bug fix, is I think, a security breach in that
Kernel copies dangling Kernel stack back to user-mode. It might even
need a stable release.

At the time I was thinking that since bsg is marked EXPERIMENTAL in
Kconfig then it must not be included in distro's Kernels by default,
but Benny noted that it is included in Fedora since FC8 or FC9.

The patch I sent still applies I think?

Thanks
Boaz

Re: [GIT PULL] block bits

[Jens Axboe]

On Mon, Feb 02 2009, Boaz Harrosh wrote:
> Jens Axboe wrote:
> > Hi Linus,
> > 
> > One critical bit in here, which fixes a reported oops from the iostat
> > patch. The others are either trivial/minor or documentation. Please
> > pull.
> > 
> >   git://git.kernel.dk/linux-2.6-block.git for-linus
> > 
> > Alberto Bertogli (2):
> >       Fix misleading comment in bio.h
> >       bio.h: If they MUST be inlined, then use __always_inline
> > 
> > Jens Axboe (3):
> >       block: fix oops in blk_queue_io_stat()
> >       block: fix inconsistent parenthesisation of QUEUE_FLAG_DEFAULT
> >       block: add text file detailing queue/ sysfs files
> > 
> 
> Hi Jens.
> 
> The bsg sense handling bug fix, is I think, a security breach in that
> Kernel copies dangling Kernel stack back to user-mode. It might even
> need a stable release.
> 
> At the time I was thinking that since bsg is marked EXPERIMENTAL in
> Kconfig then it must not be included in distro's Kernels by default,
> but Benny noted that it is included in Fedora since FC8 or FC9.
> 
> The patch I sent still applies I think?

I'm sure it does, I didn't know if Tomo had swoopt it up himself. I'll
add it to this branch.

-- 
Jens Axboe

[PATCH resend] bsg: Fix sense buffer bug in SG_IO

[Boaz Harrosh]

When submitting requests via SG_IO, which does a sync io, a
bsg_command is not allocated. So an in-Kernel sense_buffer was not
set. However when calling blk_execute_rq() with no sense buffer
one is provided from the stack. Now bsg at blk_complete_sgv4_hdr_rq()
would check if rq->sense_len and a sense was requested by sg_io_v4
the rq->sense was copy_user() back, but by now it is already mangled
stack memory.

I have fixed that by forcing a sense_buffer when calling bsg_map_hdr().
The bsg_command->sense is provided in the write/read path like before,
and on-the-stack buffer is provided when doing SG_IO.

I have also fixed a dprintk message to print rq->errors in hex because
of the scsi bit-field use of this member. For other block devices it
does not matter anyway.

Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
---
 block/bsg.c |   17 ++++++++++-------
 1 files changed, 10 insertions(+), 7 deletions(-)

diff --git a/block/bsg.c b/block/bsg.c
index d414bb5..0ce8806 100644
--- a/block/bsg.c
+++ b/block/bsg.c
@@ -244,7 +244,8 @@ bsg_validate_sgv4_hdr(struct request_queue *q, struct sg_io_v4 *hdr, int *rw)
  * map sg_io_v4 to a request.
  */
 static struct request *
-bsg_map_hdr(struct bsg_device *bd, struct sg_io_v4 *hdr, fmode_t has_write_perm)
+bsg_map_hdr(struct bsg_device *bd, struct sg_io_v4 *hdr, fmode_t has_write_perm,
+	    u8 *sense)
 {
 	struct request_queue *q = bd->queue;
 	struct request *rq, *next_rq = NULL;
@@ -306,6 +307,10 @@ bsg_map_hdr(struct bsg_device *bd, struct sg_io_v4 *hdr, fmode_t has_write_perm)
 		if (ret)
 			goto out;
 	}
+
+	rq->sense = sense;
+	rq->sense_len = 0;
+
 	return rq;
 out:
 	if (rq->cmd != rq->__cmd)
@@ -348,9 +353,6 @@ static void bsg_rq_end_io(struct request *rq, int uptodate)
 static void bsg_add_command(struct bsg_device *bd, struct request_queue *q,
 			    struct bsg_command *bc, struct request *rq)
 {
-	rq->sense = bc->sense;
-	rq->sense_len = 0;
-
 	/*
 	 * add bc command to busy queue and submit rq for io
 	 */
@@ -419,7 +421,7 @@ static int blk_complete_sgv4_hdr_rq(struct request *rq, struct sg_io_v4 *hdr,
 {
 	int ret = 0;
 
-	dprintk("rq %p bio %p %u\n", rq, bio, rq->errors);
+	dprintk("rq %p bio %p 0x%x\n", rq, bio, rq->errors);
 	/*
 	 * fill in all the output members
 	 */
@@ -635,7 +637,7 @@ static int __bsg_write(struct bsg_device *bd, const char __user *buf,
 		/*
 		 * get a request, fill in the blanks, and add to request queue
 		 */
-		rq = bsg_map_hdr(bd, &bc->hdr, has_write_perm);
+		rq = bsg_map_hdr(bd, &bc->hdr, has_write_perm, bc->sense);
 		if (IS_ERR(rq)) {
 			ret = PTR_ERR(rq);
 			rq = NULL;
@@ -922,11 +924,12 @@ static long bsg_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
 		struct request *rq;
 		struct bio *bio, *bidi_bio = NULL;
 		struct sg_io_v4 hdr;
+		u8 sense[SCSI_SENSE_BUFFERSIZE];
 
 		if (copy_from_user(&hdr, uarg, sizeof(hdr)))
 			return -EFAULT;
 
-		rq = bsg_map_hdr(bd, &hdr, file->f_mode & FMODE_WRITE);
+		rq = bsg_map_hdr(bd, &hdr, file->f_mode & FMODE_WRITE, sense);
 		if (IS_ERR(rq))
 			return PTR_ERR(rq);
 
-- 
1.6.0.1

Re: [PATCH resend] bsg: Fix sense buffer bug in SG_IO

[Jens Axboe]

On Tue, Feb 03 2009, Boaz Harrosh wrote:
> 
> When submitting requests via SG_IO, which does a sync io, a
> bsg_command is not allocated. So an in-Kernel sense_buffer was not
> set. However when calling blk_execute_rq() with no sense buffer
> one is provided from the stack. Now bsg at blk_complete_sgv4_hdr_rq()
> would check if rq->sense_len and a sense was requested by sg_io_v4
> the rq->sense was copy_user() back, but by now it is already mangled
> stack memory.
> 
> I have fixed that by forcing a sense_buffer when calling bsg_map_hdr().
> The bsg_command->sense is provided in the write/read path like before,
> and on-the-stack buffer is provided when doing SG_IO.
> 
> I have also fixed a dprintk message to print rq->errors in hex because
> of the scsi bit-field use of this member. For other block devices it
> does not matter anyway.

Thanks for the resend Boaz, I'll make sure it gets upstream for 2.6.29.

-- 
Jens Axboe

Re: [PATCH resend] bsg: Fix sense buffer bug in SG_IO

[FUJITA Tomonori]

On Tue, 03 Feb 2009 08:34:07 +0200
Boaz Harrosh <bharrosh@panasas.com> wrote:

> 
> When submitting requests via SG_IO, which does a sync io, a
> bsg_command is not allocated. So an in-Kernel sense_buffer was not
> set. However when calling blk_execute_rq() with no sense buffer
> one is provided from the stack. Now bsg at blk_complete_sgv4_hdr_rq()
> would check if rq->sense_len and a sense was requested by sg_io_v4
> the rq->sense was copy_user() back, but by now it is already mangled
> stack memory.
> 
> I have fixed that by forcing a sense_buffer when calling bsg_map_hdr().
> The bsg_command->sense is provided in the write/read path like before,
> and on-the-stack buffer is provided when doing SG_IO.
> 
> I have also fixed a dprintk message to print rq->errors in hex because
> of the scsi bit-field use of this member. For other block devices it
> does not matter anyway.
> 
> Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
> ---
>  block/bsg.c |   17 ++++++++++-------
>  1 files changed, 10 insertions(+), 7 deletions(-)

Acked-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>


I think I did this in the previous submission though.

Re: [PATCH resend] bsg: Fix sense buffer bug in SG_IO

[Jens Axboe]

On Tue, Feb 03 2009, FUJITA Tomonori wrote:
> On Tue, 03 Feb 2009 08:34:07 +0200
> Boaz Harrosh <bharrosh@panasas.com> wrote:
> 
> > 
> > When submitting requests via SG_IO, which does a sync io, a
> > bsg_command is not allocated. So an in-Kernel sense_buffer was not
> > set. However when calling blk_execute_rq() with no sense buffer
> > one is provided from the stack. Now bsg at blk_complete_sgv4_hdr_rq()
> > would check if rq->sense_len and a sense was requested by sg_io_v4
> > the rq->sense was copy_user() back, but by now it is already mangled
> > stack memory.
> > 
> > I have fixed that by forcing a sense_buffer when calling bsg_map_hdr().
> > The bsg_command->sense is provided in the write/read path like before,
> > and on-the-stack buffer is provided when doing SG_IO.
> > 
> > I have also fixed a dprintk message to print rq->errors in hex because
> > of the scsi bit-field use of this member. For other block devices it
> > does not matter anyway.
> > 
> > Signed-off-by: Boaz Harrosh <bharrosh@panasas.com>
> > ---
> >  block/bsg.c |   17 ++++++++++-------
> >  1 files changed, 10 insertions(+), 7 deletions(-)
> 
> Acked-by: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
> 
> 
> I think I did this in the previous submission though.

Yeah, I also added it.

-- 
Jens Axboe