From: Douglas Gilbert <dgilbert@interlog.com>
To: linux-scsi@vger.kernel.org
Cc: martin.petersen@oracle.com, jejb@linux.vnet.ibm.com, hare@suse.de
Subject: [PATCH v18 00/45] sg: add v4 interface
Date: Tue, 27 Apr 2021 17:56:10 -0400 [thread overview]
Message-ID: <20210427215733.417746-2-dgilbert@interlog.com> (raw)
In-Reply-To: <20210427215733.417746-1-dgilbert@interlog.com>
This patchset is the first stage of a two stage rewrite of the scsi
generic (sg) driver. The main goal of the first stage is to introduce
the sg v4 interface that uses 'struct sg_io_v4' as well as keeping and
modernizing the sg v3 interface (based on 'struct sg_io_hdr'). The
async interface formerly requiring the use of write() and read()
system calls now have ioctl(SG_IOSUBMIT) and ioctl(SG_IORECEIVE)
replacements.
A recent patch to fio allows its sg engine to issue iopoll requests
by using hipri=1 in the fio script. When testing sg device nodes
with fio this option must be set: direct=0. The sync option may
be set to 0 (use sg's write/read async interface) or 1 (uses
ioctl(SG_IO) interface).
For documentation see:
https://sg.danny.cz/sg/sg_v40.html which is mirrored at:
https://doug-gilbert.github.io/sg_v40.html
This patchset is against Martin Petersen's 5.13/scsi-queue branch.
Changes since v17 (sent to linux-scsi list on 20210407)
- make clearer distinction between user pollable (i.e. async)
requests and (user) non-pollable requests (e.g. those injected
with ioctl(SG_IO), IOWs sync requests)
- fix crash in sg_start_req() when blk_get_request() yields an
error (e.g. -EAGAIN when low on resources)
- sg_finish_scsi_blk_rq(): remove now_zero variable as suggested
by Hannes R.
- change deprecation warning url reference from http to https
Changes since v16 (sent to linux-scsi list on 20210208)
- sg_start_req() fix double free on error path [KASAN]
- sg_rq_map_kern() fix uninitialized variable [coverity]
- sg_add_sfp() fix use after free [coverity]
- sg_remove_sfp_usercontext(): remove pointless NULL check [coverity]
- fix misuse of WARN_ONCE in sg_rq_end_io_usercontext() [D. Carpenter]
- remove unused error checks: tracking blk_put_request() calls and
multiple SG_XA_RQ_FREE calls
- hipri: as blk_poll() can return > 0 for requests other than the one
that is being checked for, need to re-check that request is ready
- rebased on MKP's 5.13/scsi-queue
Changes since v15 (sent to linux-scsi list on 20210125)
- tweak state machine which sets INFLIGHT state _before_
blk_execute_rq_nowait() is called. Add a bit flag that indicates
the logic flow has returned from that call. This guards against
blk_poll() being called before the block layer has really
launched the request.
- fix bug clearing SG_FFD_HIPRI_SEEN bit as
atomic_dec_and_test() returns true when the post-decrement value
is zero, the opposite of what a C conditional does.
Changes since v14 (sent to linux-scsi list on 20210124)
- two fixes based on report from Dan Carpenter and kernel test
robot
- fix Johannes Thumshirn's email address
- separate patch issued on fio to add 'hipri' option to its sg
engine. Enables fio to test new sg driver blk_poll() support
Changes since v13 (sent to linux-scsi list on 20210113)
- fix obscure compile error reported by "kernel test robot
<lkp@intel.com>"
- harden code around blk_poll() invocation; needed based on
fio testing
- remove SG_FFD_MMAP_CALLED bit code after Hannes Reinecke pointed
out it was redundant
Changes since v12 (sent to linux-scsi list on 20201115)
- add blk_poll() support, prefix that patch's subject with 'RFC'
Changes since v11 (sent to linux-scsi list on 20201014)
- no author originated changes since v11
- port from lk 5.9.0-rc1 to lk 5.10.0-rc1 picks up a change to
the import_iovec() which requires a change to patch 25/44
- only publish this cover letter and v12 of patch 25/44 to
the linux-scsi list. The other 43 patches remain as published
on 20201014.
Changes since v10 (sent to linux-scsi list on 20200823)
- unchanged: 0001 to 0009, 0010 to 0017
- rename sg_add_req() to sg_setup_req() [0010]
- patches 40,41,42 and 43 are new, see their commit messages
- remove SG_RS_RCV_DONE request state leaving 3.5 states
[the 0.5 state is SG_RS_BUSY]
- rework sg_rq_chg_state() code that enforces request
state changes and associated xarray marks
- track lowest used and unused indexes in the request arrays so
iterations over the request xarray are efficient. This is a
significant saving when the iodepth queue length is large
Changes since v9 (sent to linux-scsi list on 20200421)
- rebase on MKP's 5.10/scsi-queue branch
- remove some master/slave terminology that had bled in from
the part 2 patchset
- change sg_request::start_ns type from ktime_t to u64
- pick up several error path correction fixes applied to the
sg driver by other authors
Changes since v8 to v1 in earlier patchsets
- see: the v10 patchset sent to linux-scsi on 20200823
Douglas Gilbert (45):
sg: move functions around
sg: remove typedefs, type+formatting cleanup
sg: sg_log and is_enabled
sg: rework sg_poll(), minor changes
sg: bitops in sg_device
sg: make open count an atomic
sg: move header to uapi section
sg: speed sg_poll and sg_get_num_waiting
sg: sg_allow_if_err_recovery and renames
sg: improve naming
sg: change rwlock to spinlock
sg: ioctl handling
sg: split sg_read
sg: sg_common_write add structure for arguments
sg: rework sg_vma_fault
sg: rework sg_mmap
sg: replace sg_allow_access
sg: rework scatter gather handling
sg: introduce request state machine
sg: sg_find_srp_by_id
sg: sg_fill_request_element
sg: printk change %p to %pK
sg: xarray for fds in device
sg: xarray for reqs in fd
sg: replace rq array with xarray
sg: sense buffer rework
sg: add sg v4 interface support
sg: rework debug info
sg: add 8 byte SCSI LUN to sg_scsi_id
sg: expand sg_comm_wr_t
sg: add sg_iosubmit_v3 and sg_ioreceive_v3 ioctls
sg: add some __must_hold macros
sg: move procfs objects to avoid forward decls
sg: protect multiple receivers
sg: first debugfs support
sg: rework mmap support
sg: defang allow_dio
sg: warn v3 write system call users
sg: add mmap_sz tracking
sg: remove rcv_done request state
sg: track lowest inactive and await indexes
sg: remove unit attention check for device changed
sg: no_dxfer: move to/from kernel buffers
sg: add blk_poll support
sg: bump version to 4.0.12
drivers/scsi/sg.c | 5409 +++++++++++++++++++++++++++-------------
include/scsi/sg.h | 273 +-
include/uapi/scsi/sg.h | 375 +++
3 files changed, 4116 insertions(+), 1941 deletions(-)
create mode 100644 include/uapi/scsi/sg.h
--
2.25.1
next prev parent reply other threads:[~2021-04-27 21:57 UTC|newest]
Thread overview: 86+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-27 21:56 [PATCH v18 00/83] sg: add v4 interface, request sharing Douglas Gilbert
2021-04-27 21:56 ` Douglas Gilbert [this message]
2021-04-27 21:56 ` [PATCH v18 01/83] sg: move functions around Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 02/83] sg: remove typedefs, type+formatting cleanup Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 03/83] sg: sg_log and is_enabled Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 04/83] sg: rework sg_poll(), minor changes Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 05/83] sg: bitops in sg_device Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 06/83] sg: make open count an atomic Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 07/83] sg: move header to uapi section Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 08/83] sg: speed sg_poll and sg_get_num_waiting Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 09/83] sg: sg_allow_if_err_recovery and renames Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 10/83] sg: improve naming Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 11/83] sg: change rwlock to spinlock Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 12/83] sg: ioctl handling Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 13/83] sg: split sg_read Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 14/83] sg: sg_common_write add structure for arguments Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 15/83] sg: rework sg_vma_fault Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 16/83] sg: rework sg_mmap Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 17/83] sg: replace sg_allow_access Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 18/83] sg: rework scatter gather handling Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 19/83] sg: introduce request state machine Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 20/83] sg: sg_find_srp_by_id Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 21/83] sg: sg_fill_request_element Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 22/83] sg: printk change %p to %pK Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 23/83] sg: xarray for fds in device Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 24/83] sg: xarray for reqs in fd Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 25/83] sg: replace rq array with xarray Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 26/83] sg: sense buffer rework Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 27/83] sg: add sg v4 interface support Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 28/83] sg: rework debug info Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 29/83] sg: add 8 byte SCSI LUN to sg_scsi_id Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 30/83] sg: expand sg_comm_wr_t Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 31/83] sg: add sg_iosubmit_v3 and sg_ioreceive_v3 ioctls Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 32/83] sg: add some __must_hold macros Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 33/83] sg: move procfs objects to avoid forward decls Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 34/83] sg: protect multiple receivers Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 35/83] sg: first debugfs support Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 36/83] sg: rework mmap support Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 37/83] sg: defang allow_dio Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 38/83] sg: warn v3 write system call users Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 39/83] sg: add mmap_sz tracking Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 40/83] sg: remove rcv_done request state Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 41/83] sg: track lowest inactive and await indexes Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 42/83] sg: remove unit attention check for device changed Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 43/83] sg: no_dxfer: move to/from kernel buffers Douglas Gilbert
2021-04-28 7:07 ` Hannes Reinecke
2021-04-27 21:56 ` [PATCH v18 44/83] sg: add blk_poll support Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 45/83] sg: bump version to 4.0.12 Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 46/83] sg: add sg_ioabort ioctl Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 47/83] sg: add sg_set_get_extended ioctl Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 48/83] sg: sgat_elem_sz and sum_fd_dlens Douglas Gilbert
2021-04-27 21:56 ` [PATCH v18 49/83] sg: tag and more_async Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 50/83] sg: add fd sharing , change, unshare Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 51/83] sg: add shared requests Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 52/83] sg: add multiple request support Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 53/83] sg: rename some mrq variables Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 54/83] sg: unlikely likely Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 55/83] sg: mrq abort Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 56/83] sg: reduce atomic operations Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 57/83] sg: add excl_wait flag Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 58/83] sg: tweak sg_find_sfp_by_fd() Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 59/83] sg: add snap_dev flag and snapped in debugfs Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 60/83] sg: compress usercontext to uc Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 61/83] sg: optionally output sg_request.frq_bm flags Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 62/83] sg: work on sg_mrq_sanity() Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 63/83] sg: shared variable blocking Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 64/83] sg: device timestamp Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 65/83] sg: condition met is not an error Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 66/83] sg: split sg_setup_req Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 67/83] sg: finish after read-side request Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 68/83] sg: keep share and dout offset flags Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 69/83] sg: add dlen to sg_comm_wr_t Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 70/83] sg: make use of struct sg_mrq_hold Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 71/83] sg: add mmap IO option for mrq metadata Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 72/83] sg: add eventfd support Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 73/83] sg: table of error number explanations Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 74/83] sg: add ordered write flag Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 75/83] sg: expand source line length to 100 characters Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 76/83] sg: add no_attach_msg parameter Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 77/83] sg: add SGV4_FLAG_REC_ORDER Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 78/83] sg: max to read for mrq sg_ioreceive Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 79/83] sg: mrq: if uniform svb then re-use bio_s Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 80/83] sg: expand bvec usage; " Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 81/83] sg: blk_poll/hipri work for mrq Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 82/83] sg: pollable and non-pollable requests Douglas Gilbert
2021-04-27 21:57 ` [PATCH v18 83/83] sg: bump version to 4.0.47 Douglas Gilbert
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210427215733.417746-2-dgilbert@interlog.com \
--to=dgilbert@interlog.com \
--cc=hare@suse.de \
--cc=jejb@linux.vnet.ibm.com \
--cc=linux-scsi@vger.kernel.org \
--cc=martin.petersen@oracle.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox