Re: [PATCH v5 7/8] x86/string: extend memcpy_flushcache() fixed-size fastpaths

["Li Zhe" <lizhe.67@bytedance.com>]

On Mon, 6 Jul 2026 10:19:02 -0700, bp@alien8.de wrote:

> On Mon, Jul 06, 2026 at 03:43:46PM +0800, Li Zhe wrote:
> > The extra IS_ALIGNED() check is meant as a conservative performance
> > choice, not because MOVNTI itself would be invalid on a misaligned
> > destination.
> 
> Dude, you can't just wait for me to pull every unobvious decision in the code
> out of you - you need to write it down in your commit messages! I'm not a mind
> reader, you know.
> 
> > As the Intel optimization reference manual notes, higher-throughput
> > data movement usually comes with alignment considerations, and can
> > be more efficient when the destination is naturally aligned.
> 
> This can't get any more vague.
> 
> > So for these larger fixed-size cases, I kept the inline path limited
> > to the aligned case. For misaligned destinations, falling back to
> > __memcpy_flushcache() is expected to be the better choice, since it
> > already handles the unaligned head first and only then enters its
> > MOVNTI loops.
> >
> > I will make that rationale explicit in the code comment and commit
> > message in v6.
> 
> No, you need to show whether the alignment brigns anything and then add it or
> remove it. We don't do things based on hearsay in the kernel - only on hard
> performance numbers.

Thanks. I measured the exact tradeoff on the physical x86_64 host with
an out-of-tree microbenchmark.

The question for this patch is whether misaligned 32/48/64/80/96-byte
copies should still stay in the fixed-size switch, or whether an extra
alignment check should redirect them to the generic
__memcpy_flushcache() path.

To isolate that choice, I benchmarked two memcpy_flushcache()-style
helpers which shared the same generic body. The only difference was
whether those fixed-size cases always stayed in the fixed-size MOVNTI
switch, or whether misaligned destinations were redirected to the
generic path:

variant A:
    switch (len) {
    case 32 ... 96:
        do fixed-size MOVNTI stores;
        return;
    default:
        generic __memcpy_flushcache()-style body;
    }

variant B:
    switch (len) {
    case 32 ... 96:
        if (!IS_ALIGNED(dst, 8))
            goto generic_path;
        do fixed-size MOVNTI stores;
        return;
    default:
    generic_path:
        generic __memcpy_flushcache()-style body;
    }

On the same CPU, with the same source buffer, destination layout,
stride, and iteration count, I measured the median time per memcpy.
The wmb() was used only to separate rounds and was not included in the
timed interval.

For offset 0, the two variants were essentially the same. For
misaligned destinations (offsets 1..7 averaged), keeping those sizes in
the fixed-size switch took about:

    32-byte memcpy: 0.410 us
    48-byte memcpy: 0.411 us
    64-byte memcpy: 0.426 us
    80-byte memcpy: 0.426 us
    96-byte memcpy: 0.428 us

Redirecting the same misaligned cases to the generic
__memcpy_flushcache() path took about:

    32-byte memcpy: 0.962 us
    48-byte memcpy: 0.956 us
    64-byte memcpy: 0.923 us
    80-byte memcpy: 0.998 us
    96-byte memcpy: 1.001 us

So based on these measurements, the extra alignment check is not buying
us anything here. It consistently makes the misaligned 32/48/64/80/96
cases slower by redirecting them to the generic path.

I will drop the separate alignment check and fold 32/48/64/80/96 into
the main fixed-size switch in the next version.

Thanks,
Zhe