From: Kent Overstreet <koverstreet@google.com>
To: axboe@kernel.dk, tytso@mit.edu, linux-kernel@vger.kernel.org,
linux-fsdevel@vger.kernel.org
Subject: Immutable biovecs, dio rewrite
Date: Sat, 8 Jun 2013 19:18:42 -0700 [thread overview]
Message-ID: <1370744348-15407-1-git-send-email-koverstreet@google.com> (raw)
Immutable biovecs: Drivers no longer modify the biovec array directly
(bv_len/bv_offset in particular) - we add a real iterator to struct bio
that lets drivers partially complete a bio while only modifying the
iterator. The iterator has the existing bi_sector, bi_size, bi_idx
memembers, and also bi_bvec_done.
This gets us a couple things:
* Changing all the drivers to go through the iterator means that we can
submit a partially completed bio to generic_make_request() - this
previously worked on some drivers, but worked on others.
This makes it much easier for upper layers to process bios
incrementally - not just stacking drivers, my dio rewrite relies
heavily on this strategy.
* Previously, any code that might need to retry a bio somehow if it
errored (mainly stacking drivers) had to clone not just the bio, but
the entire biovec. The biovec can be up to BIO_MAX_PAGES, which works
out to 4k...
* When cloning a bio, now we don't have to clone the biovec unless we
want to modify it. Bio splitting also becomes just a special case of
cloning a bio.
We also get to delete a lot of code. And this patch series barely
scratches the surface - I've got more patches that delete another 1.5k
lines of code, without trying all that hard.
I'd like to get as much of this into 3.11 as possible - I don't know if
the dio rewrite is a realistic possibility (it currently breaks btrfs -
we need to add a different hook for them) and it does need a lot of
review and testing from the various driver maintainers. The dio rewrite
does pass xfstests for me, though.
Patch series is on top of v3.10-rc4, and it's available in my git tree:
git://evilpiepirate.org/~kent/linux-bcache.git block
http://evilpiepirate.org/git/linux-bcache.git block
Documentation/block/biodoc.txt | 7 +-
arch/m68k/emu/nfblock.c | 13 +-
arch/powerpc/sysdev/axonram.c | 21 +-
block/blk-core.c | 74 +-
block/blk-flush.c | 2 +-
block/blk-integrity.c | 40 +-
block/blk-lib.c | 179 +---
block/blk-map.c | 6 +-
block/blk-merge.c | 193 ++++-
block/blk-throttle.c | 13 +-
block/blk.h | 3 +
block/elevator.c | 2 +-
drivers/block/aoe/aoe.h | 10 +-
drivers/block/aoe/aoecmd.c | 145 ++--
drivers/block/brd.c | 16 +-
drivers/block/drbd/drbd_actlog.c | 2 +-
drivers/block/drbd/drbd_bitmap.c | 2 +-
drivers/block/drbd/drbd_main.c | 27 +-
drivers/block/drbd/drbd_receiver.c | 19 +-
drivers/block/drbd/drbd_req.c | 6 +-
drivers/block/drbd/drbd_req.h | 2 +-
drivers/block/drbd/drbd_worker.c | 8 +-
drivers/block/floppy.c | 16 +-
drivers/block/loop.c | 27 +-
drivers/block/mtip32xx/mtip32xx.c | 22 +-
drivers/block/nbd.c | 14 +-
drivers/block/nvme-core.c | 144 +---
drivers/block/pktcdvd.c | 178 ++--
drivers/block/ps3disk.c | 7 +-
drivers/block/ps3vram.c | 10 +-
drivers/block/rbd.c | 89 +-
drivers/block/rsxx/dev.c | 4 +-
drivers/block/rsxx/dma.c | 15 +-
drivers/block/umem.c | 53 +-
drivers/block/virtio_blk.c | 4 +-
drivers/block/xen-blkback/blkback.c | 2 +-
drivers/block/xen-blkfront.c | 14 +-
drivers/md/bcache/alloc.c | 4 +-
drivers/md/bcache/bcache.h | 20 -
drivers/md/bcache/btree.c | 32 +-
drivers/md/bcache/debug.c | 33 +-
drivers/md/bcache/io.c | 260 +-----
drivers/md/bcache/journal.c | 16 +-
drivers/md/bcache/movinggc.c | 11 +-
drivers/md/bcache/request.c | 203 ++---
drivers/md/bcache/request.h | 1 -
drivers/md/bcache/super.c | 60 +-
drivers/md/bcache/util.c | 21 +-
drivers/md/bcache/util.h | 8 +-
drivers/md/bcache/writeback.c | 17 +-
drivers/md/dm-bio-record.h | 37 +-
drivers/md/dm-bufio.c | 2 +-
drivers/md/dm-cache-policy-mq.c | 4 +-
drivers/md/dm-cache-target.c | 16 +-
drivers/md/dm-crypt.c | 68 +-
drivers/md/dm-delay.c | 7 +-
drivers/md/dm-flakey.c | 7 +-
drivers/md/dm-io.c | 38 +-
drivers/md/dm-linear.c | 3 +-
drivers/md/dm-raid1.c | 20 +-
drivers/md/dm-region-hash.c | 3 +-
drivers/md/dm-snap.c | 13 +-
drivers/md/dm-stripe.c | 13 +-
drivers/md/dm-thin.c | 23 +-
drivers/md/dm-verity.c | 61 +-
drivers/md/dm.c | 181 +---
drivers/md/faulty.c | 19 +-
drivers/md/linear.c | 96 +--
drivers/md/md.c | 35 +-
drivers/md/multipath.c | 13 +-
drivers/md/raid0.c | 79 +-
drivers/md/raid1.c | 63 +-
drivers/md/raid10.c | 198 +++--
drivers/md/raid5.c | 84 +-
drivers/message/fusion/mptsas.c | 8 +-
drivers/s390/block/dcssblk.c | 19 +-
drivers/s390/block/xpram.c | 19 +-
drivers/scsi/libsas/sas_expander.c | 8 +-
drivers/scsi/mpt2sas/mpt2sas_transport.c | 41 +-
drivers/scsi/mpt3sas/mpt3sas_transport.c | 39 +-
drivers/scsi/osd/osd_initiator.c | 2 +-
drivers/scsi/sd.c | 2 +-
drivers/scsi/sd_dif.c | 30 +-
drivers/staging/zram/zram_drv.c | 31 +-
drivers/target/target_core_iblock.c | 2 +-
fs/bio-integrity.c | 189 +----
fs/bio.c | 527 ++++++------
fs/btrfs/check-integrity.c | 10 +-
fs/btrfs/compression.c | 17 +-
fs/btrfs/extent_io.c | 16 +-
fs/btrfs/file-item.c | 13 +-
fs/btrfs/inode.c | 17 +-
fs/btrfs/raid56.c | 22 +-
fs/btrfs/scrub.c | 12 +-
fs/btrfs/volumes.c | 12 +-
fs/buffer.c | 12 +-
fs/direct-io.c | 1318 ++++++++----------------------
fs/ext4/page-io.c | 4 +-
fs/f2fs/data.c | 2 +-
fs/f2fs/segment.c | 3 +-
fs/gfs2/lops.c | 2 +-
fs/gfs2/ops_fstype.c | 2 +-
fs/hfsplus/wrapper.c | 2 +-
fs/jfs/jfs_logmgr.c | 10 +-
fs/jfs/jfs_metapage.c | 9 +-
fs/logfs/dev_bdev.c | 20 +-
fs/mpage.c | 2 +-
fs/nfs/blocklayout/blocklayout.c | 9 +-
fs/nilfs2/segbuf.c | 3 +-
fs/ocfs2/cluster/heartbeat.c | 2 +-
fs/xfs/xfs_aops.c | 2 +-
fs/xfs/xfs_buf.c | 4 +-
include/linux/bio.h | 226 +++--
include/linux/blk_types.h | 28 +-
include/linux/blkdev.h | 13 +-
include/linux/dm-io.h | 4 +-
include/trace/events/bcache.h | 20 +-
include/trace/events/block.h | 26 +-
include/trace/events/f2fs.h | 4 +-
kernel/power/block_io.c | 2 +-
kernel/trace/blktrace.c | 15 +-
mm/bounce.c | 45 +-
mm/page_io.c | 10 +-
123 files changed, 2236 insertions(+), 3757 deletions(-)
next reply other threads:[~2013-06-09 2:19 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-06-09 2:18 Kent Overstreet [this message]
2013-06-09 2:18 ` [PATCH 01/26] bcache: Use standard utility code Kent Overstreet
2013-06-09 2:18 ` [PATCH 02/26] bcache: Kill unaligned bvec hack Kent Overstreet
2013-06-09 2:18 ` [PATCH 03/26] block: Abstract out bvec iterator Kent Overstreet
2013-06-09 2:18 ` [PATCH 04/26] dm: Use bvec_iter for dm_bio_record() Kent Overstreet
2013-06-09 2:18 ` [PATCH 05/26] block: Convert bio_iovec() to bvec_iter Kent Overstreet
2013-06-09 2:18 ` [PATCH 07/26] block: Immutable bio vecs Kent Overstreet
2013-06-09 2:18 ` [PATCH 08/26] block: Convert bio_copy_data() to bvec_iter Kent Overstreet
2013-06-09 2:18 ` [PATCH 09/26] bio-integrity: Convert " Kent Overstreet
2013-06-09 2:18 ` [PATCH 10/26] block: Convert drivers to immutable biovecs Kent Overstreet
2013-06-28 19:39 ` Ed Cashin
2013-06-09 2:18 ` [PATCH 11/26] block: Kill bio_iovec_idx(), __bio_iovec() Kent Overstreet
2013-06-09 2:18 ` [PATCH 12/26] rbd: Refactor bio cloning, don't clone biovecs Kent Overstreet
2013-06-09 2:18 ` [PATCH 13/26] dm: Refactor for new bio cloning/splitting Kent Overstreet
2013-06-09 2:18 ` [PATCH 14/26] md, bcache: Remove bi_idx hacks Kent Overstreet
2013-06-09 2:18 ` [PATCH 15/26] block: Generic bio chaining Kent Overstreet
2013-06-09 2:18 ` [PATCH 16/26] block: Rename bio_split() -> bio_pair_split() Kent Overstreet
2013-06-09 2:18 ` [PATCH 17/26] block: Introduce new bio_split() Kent Overstreet
2013-06-09 2:19 ` [PATCH 18/26] block: Kill bio_pair_split() Kent Overstreet
2013-06-09 2:19 ` [PATCH 19/26] block: Kill bio_segments() Kent Overstreet
2013-06-09 2:19 ` [PATCH 20/26] block: Don't save/copy bvec array anymore, share when cloning Kent Overstreet
2013-06-09 2:19 ` [PATCH 21/26] block: Move bouncing to generic_make_request() Kent Overstreet
2013-06-09 2:19 ` [PATCH 22/26] block: Make generic_make_request handle arbitrary sized bios Kent Overstreet
2013-06-11 17:12 ` David Sterba
2013-06-12 4:26 ` Kent Overstreet
2013-06-09 2:19 ` [PATCH 23/26] blk-lib.c: generic_make_request() handles large bios now Kent Overstreet
2013-06-09 2:19 ` [PATCH 24/26] bcache: " Kent Overstreet
2013-06-09 2:19 ` [PATCH 25/26] block: Add bio_get_user_pages() Kent Overstreet
2013-06-09 2:19 ` [PATCH 26/26] Apply fire to dio code Kent Overstreet
2013-06-09 8:34 ` Immutable biovecs, dio rewrite Geert Uytterhoeven
2013-06-09 8:55 ` Kent Overstreet
[not found] ` <1370744348-15407-7-git-send-email-koverstreet@google.com>
2013-06-09 14:21 ` [PATCH 06/26] block: Convert bio_for_each_segment() to bvec_iter Geoff Levand
2013-06-11 5:20 ` Immutable biovecs, dio rewrite Dave Chinner
2013-06-12 20:30 ` Kent Overstreet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1370744348-15407-1-git-send-email-koverstreet@google.com \
--to=koverstreet@google.com \
--cc=axboe@kernel.dk \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox