[PATCH v6 00/18] s390/dasd: ESE thin-provisioning performance improvements

[Stefan Haberland <sth@linux.ibm.com>]

Extent Space Efficient (ESE) volumes allocate a track on its first write.
Today the first write to an unallocated track fails with No Record Found
(or similar), the driver formats the track and retries the write.
That NRF loop dominates the write path of a freshly provisioned volume.

This series lets the driver write and allocate a track in a single
operation using WRITE_FULL_TRACK, so the common "write to a
not-yet-allocated track" case avoids the NRF penalty.
Because full-track writes are only a win while a volume is still sparse, an
adaptive heuristic (exposed through a single sysfs knob) probes
the workload and falls back to normal writes once the device is mostly
allocated. On top of that the series adds an on-disk format label so an
ESE volume can be recognised without querying the hardware, and re-enables
discard for ESE volumes so freed space can be returned to the pool.

The sysfs knob is 0..100. 0 pins normal writes, 100 pins full-track,
and the default of 50 enables the heuristic; the adaptive range
interpolates the heuristic parameters between the two ends.

Patch 17 releases whole extents only: a discard range is rounded inward to
extent boundaries and partially covered boundary tracks are dropped, so an
extent shared with a live allocation is never released. Sub-extent discards
that cover no whole extent are rejected rather than over-released and
corrupting data.

Patch 18 fixes a use-after-free: the CCW build path read the base address
and LSS directly from conf.ned, which the reload worker can free
concurrently with I/O. The values are now read from the copies kept in the
device uid / private structure, refreshed under the ccwdev lock when the
configuration is (re)read.

Fixes (pre-existing):
  01  Do not complete a failed ESE read as successful
  02  Propagate partial completion length across ERP recovery
  03  Guard sysfs discipline callbacks against unallocated private data

v5->v6:
  - New patch 01 (prepended pre-existing fix): dasd_int_handler() no longer
    completes a failed ESE read as DASD_CQR_SUCCESS. ese_read() can return
    an error before it has zeroed the destination buffer, which would hand
    the block layer stale memory; it now fails the request through the
    normal error path.
  - Patch 03 (sysfs guard): query_host_access() now also guards
    private->lcu, which is NULL in the same set_online window while
    host_access_count is world-readable; the added !private guard alone did
    not cover the private->lcu->pav dereference.
  - Patch 10 (build_cp builder): the newly added
    dasd_eckd_build_cp_tpm_writefulltrack() is marked __maybe_unused
    (removed again in patch 11, where it is first called) so the
    intermediate commits do not emit -Wunused-function / break a
    CONFIG_WERROR build.
  - Patch 11 (WRITE_FULL_TRACK): dasd_generic_requeue_all_requests() now
    skips an aborted (replaced) request instead of requeuing it - the
    request is retired by its full-track replacement, so requeuing it here
    would double-handle the block request. (The shared flush path already
    handled this via __dasd_cleanup_cqr().)
  - Patch 11 (WRITE_FULL_TRACK): the ERP retry request now inherits
    filldata (dasd_3990_erp_add_erp), so an ERP retry of a full-track write
    stays recognised as one and is failed rather than misrouted back into
    ese_format() - which would rebuild a fresh full-track write and could
    loop on a persistent INV_TRACK_FORMAT. ese_format() also fails fast on
    a permanent -EINVAL build error instead of retrying it.
  - Patch 13 (adaptive heuristic): the NRF-rate computation uses a u64
    intermediate so nrf * 1000 cannot overflow int, and the FT0_STABLE
    branches now re-assert fulltrack = 0 so a value left behind by a racing
    sysfs full_track_bias write self-corrects instead of leaving the device
    in full-track mode.
  - Patch 15 (detect ESE from label): the rewritten dasd_eckd_is_ese() now
    guards a NULL device->private, matching its sibling
    dasd_eckd_on_demand_format()
  - Patch 17 (re-enable discard): a discard that covers no whole extent is
    now completed as a benign no-op (BLK_STS_OK) instead of
    BLK_STS_NOTSUPP.
    Discard is advisory, and -EOPNOTSUPP only ever originates from the
    discard builder's "no whole extent to release" paths; returning NOTSUPP
    made filesystems treat the device as lacking discard and stop issuing
    it. The request is ended after the queue lock is dropped.
    dasd_eckd_disc_limits() also guards a zero extent size before the
    modulo, so malformed RDC data cannot divide by zero at bring-up.

v4->v5:
  - The sysfs private-data guard (patch 02) was moved out of the callers
    (the DASD_DEFINE_ATTR() macro and the individual sysfs handlers) into
    each discipline callback that actually dereferences device->private, so
    a caller need not know whether a callee uses private. This also covers
    the callbacks that were still unguarded: is_ese(), the extent-pool /
    space attributes (ext_size(), ext_pool_id(), space_configured(),
    space_allocated(), logical_capacity(), ext_pool_warn_thrshld(),
    ext_pool_cap_at_warnlevel(), ext_pool_oos()), hpf_enabled(),
    reset_path() and query_host_access().
  - Full-track channel-program builder (patch 09): crosses_page() and
    reserve_nocross() now use offset_in_page(); reserve_nocross() fails
    atomically without advancing the fill pointer; datasize simplified to
    trkcount * tlf; plus minor cleanups (declaration order, dead code,
    comments).
  - full_track_bias_store() (patch 12) received the same guard-in-callee
    treatment: the offline check was removed from the store, and
    dasd_ese_adaptive() now verifies device->discipline before invoking the
    is_ese callback. The heuristic interpolation endpoints d50/d100 were
    renamed v50/v100 for clarity.
  - On-disk format label (patch 13): the informational per-format string
    now records the running kernel version (uname -r) instead of a fixed
    driver name; the field was renamed kernel_version and enlarged to
    64 bytes.
  - Discard (patch 16): build_cp_discard() now uses roundup()/rounddown()
    with a named inclusive last track for the extent-alignment (no
    functional change); the count_exts() comment was trimmed to just
    describe the formula.

v3->v4:
  - Two pre-existing bugs surfaced during review are prepended as fixes:
      - ERP recovery now carries proc_bytes back to the original request,
        so a partially completed ESE read that is recovered through the ERP
	chain no longer completes the whole request and returns zeroed data
	for the unread remainder (patch 01).
      - the ese / on_demand_formatting sysfs attributes now check
        device->private before dereferencing it, closing an unprivileged
	NULL-pointer read during the set_online window (patch 02).
  - The dasd_alloc_device() GFP_ATOMIC->GFP_KERNEL conversion was split out
    of the ESE infrastructure patch into its own patch (04): it runs in
    process context and is an independent change.
  - New patch (05) names the Extended Address Volume track-address shifts
    used by set_ch_t()/set_chr_t() instead of open-coding the constants.
  - Full-track write path: the ESE format handler's error, partial
    completion and abort exits now cancel the device timer and schedule the
    bottom halves (a failed request previously waited for the timeout); the
    exits were consolidated behind shared out:/out_retry: labels. free_cp()
    now releases the bounce buffers of an aborted (format-replaced) write
    instead of leaking them. build_cp_tpm_writefulltrack() budgets
    page-boundary padding in the fill buffer, and its in-loop track-end
    length matches the physical track.
  - Adaptive mode selection: full_track_bias no longer enables full-track
    writes on non-ESE volumes by default (only an explicit ft_bias=100
    forces it); the sysfs store rejects an offline device, and the
    heuristic re-asserts the endpoint mode per I/O so a racing sysfs write
    cannot wedge it.
  - On-disk format label: a failed full format invalidates the cached label
    so is_ese() falls back to the hardware field, and the fulltrack
    heuristic is (re)applied after the format commits rather than before.
  - Discard: discard is no longer advertised for raw-track-access (USERAW)
    volumes, which would otherwise route a discard into the raw CCW
    builder.
  - conf.ned fix: the cached unit address is kept in a dedicated field, so
    the lockless CCW-build readers cannot observe the transient zero while
    create_uid() repopulates the uid.

Stefan Haberland (18):
  s390/dasd: Do not complete a failed ESE read as successful
  s390/dasd: Propagate partial completion length across ERP recovery
  s390/dasd: Guard sysfs discipline callbacks against unallocated
    private data
  s390/dasd: Optimize max blocks per request for track alignment
  s390/dasd: Use GFP_KERNEL in dasd_alloc_device()
  s390/dasd: Add defines for the Extended Address Volume track address
  s390/dasd: Add infrastructure for ESE full-track write
  s390/dasd: Add range-based format-track collision detection
  s390/dasd: Extend prepare_itcw() to support WRITE_FULL_TRACK
  s390/dasd: Add dasd_eckd_build_cp_tpm_writefulltrack()
  s390/dasd: Use WRITE_FULL_TRACK in ESE format handler
  s390/dasd: Add full_track_bias to control fulltrack write mode
  s390/dasd: Derive adaptive ESE fulltrack heuristic from ft_bias
  s390/dasd: Stamp a format label into newly formatted volumes
  s390/dasd: Detect ESE volumes from the on-disk format label
  s390/dasd: Report ESE capability and format mode at device online
  s390/dasd: Re-enable discard support for ESE volumes
  s390/dasd: Read cached unit address and LSS in the CCW build path

 drivers/s390/block/dasd.c          |  226 +++--
 drivers/s390/block/dasd_3990_erp.c |    1 +
 drivers/s390/block/dasd_devmap.c   |   87 +-
 drivers/s390/block/dasd_eckd.c     | 1258 +++++++++++++++++++++++++---
 drivers/s390/block/dasd_eckd.h     |   68 +-
 drivers/s390/block/dasd_erp.c      |   11 +-
 drivers/s390/block/dasd_int.h      |  149 +++-
 7 files changed, 1604 insertions(+), 196 deletions(-)

-- 
2.53.0