[PATCH v6 0/7] Improve the performance of RISC-V vector unit-stride/whole register ld/st instructions

[Max Chou <max.chou@sifive.com>]

Hi,

This version fixes several issues in v5
- The cross page bound checking issue
- The mismatch vl comparison in the early exit checking of vext_ldst_us
- The endian issue when host is big endian

Thank for Richard Henderson's suggestions that this version unrolled the
loop in helper functions of unmasked vector unit-stride load/store
instructions, etc.

And this version also extends the optimizations to the unmasked vector
fault-only-first load instruction.

Some performance result of this version 
1. Test case provided in 
    https://gitlab.com/qemu-project/qemu/-/issues/2137#note_1757501369
    - QEMU user mode (vlen=128):
        - Original:   ~11.8 sec
        - v5:          ~1.3 sec
        - v6:          ~1.2 sec
    - QEMU system mode (vlen=128):
        - Original:   ~29.4 sec
        - v5:          ~1.6 sec
        - v6:          ~1.6 sec
2. SPEC CPU2006 INT (test input)
    - QEMU user mode (vlen=128)
        - Original:  ~459.1 sec
        - v5:        ~300.0 sec
        - v6:        ~280.6 sec
3. SPEC CPU2017 intspeed (test input)
    - QEMU user mode (vlen=128)
        - Original: ~2475.9 sec
        - v5:       ~1702.6 sec
        - v6:       ~1663.4 sec


This version is based on the riscv-to-apply.next branch(commit 90d5d3c).

Changes from v5:
- patch 2
    - Replace the VSTART_CHECK_EARLY_EXIT function by checking the
      correct evl in vext_ldst_us.
- patch 3
    - Unroll the memory load/store loop
    - Fix the bound checking issue in cross page elements
    - Fix the endian issue in GEN_VEXT_LD_ELEM/GEN_VEXT_ST_ELEM
    - Pass in mmu_index for vext_page_ldst_us
    - Reduce the flag & host checking
- patch 4
    - Unroll the memory load/store loop
    - Fix the bound checking issue in cross page elements
- patch 5
    - Extend optimizations to unmasked vector fault-only-first load
      instruction
- patch 6
    - Switch to memcpy only when doing byte load/store
- patch 7
    - Inline the vext_page_ldst_us function

Previous versions:
- v1: https://lore.kernel.org/all/20240215192823.729209-1-max.chou@sifive.com/
- v2: https://lore.kernel.org/all/20240531174504.281461-1-max.chou@sifive.com/
- v3: https://lore.kernel.org/all/20240613141906.1276105-1-max.chou@sifive.com/
- v4: https://lore.kernel.org/all/20240613175122.1299212-1-max.chou@sifive.com/
- v5: https://lore.kernel.org/all/20240717133936.713642-1-max.chou@sifive.com/

Max Chou (7):
  target/riscv: Set vdata.vm field for vector load/store whole register
    instructions
  target/riscv: rvv: Replace VSTART_CHECK_EARLY_EXIT in vext_ldst_us
  target/riscv: rvv: Provide a fast path using direct access to host ram
    for unmasked unit-stride load/store
  target/riscv: rvv: Provide a fast path using direct access to host ram
    for unit-stride whole register load/store
  target/riscv: rvv: Provide a fast path using direct access to host ram
    for unit-stride load-only-first load instructions
  target/riscv: rvv: Provide group continuous ld/st flow for unit-stride
    ld/st instructions
  target/riscv: Inline unit-stride ld/st and corresponding functions for
    performance

 target/riscv/insn_trans/trans_rvv.c.inc |   3 +
 target/riscv/vector_helper.c            | 598 ++++++++++++++++--------
 2 files changed, 400 insertions(+), 201 deletions(-)

-- 
2.34.1