Re: [PATCH RFC 0/7] block: Introduce CBD (CXL Block Device)

[Dongsheng Yang <dongsheng.yang@easystack.cn>]

在 2024/5/22 星期三 上午 2:41, Dan Williams 写道:
> Dongsheng Yang wrote:
>> 在 2024/5/9 星期四 下午 8:21, Jonathan Cameron 写道:
> [..]
>>>> If we check and find that the "No clean writeback" bit in both CSDS and
>>>> DVSEC is set, can we then assume that software cache-coherency is
>>>> feasible, as outlined below:
>>>>
>>>> (1) Both the writer and reader ensure cache flushes. Since there are no
>>>> clean writebacks, there will be no background data writes.
>>>>
>>>> (2) The writer writes data to shared memory and then executes a cache
>>>> flush. If we trust the "No clean writeback" bit, we can assume that the
>>>> data in shared memory is coherent.
>>>>
>>>> (3) Before reading the data, the reader performs cache invalidation.
>>>> Since there are no clean writebacks, this invalidation operation will
>>>> not destroy the data written by the writer. Therefore, the data read by
>>>> the reader should be the data written by the writer, and since the
>>>> writer's cache is clean, it will not write data to shared memory during
>>>> the reader's reading process. Additionally, data integrity can be ensured.
> 
> What guarantees this property? How does the reader know that its local
> cache invalidation is sufficient for reading data that has only reached
> global visibility on the remote peer? As far as I can see, there is
> nothing that guarantees that local global visibility translates to
> remote visibility. In fact, the GPF feature is counter-evidence of the
> fact that writes can be pending in buffers that are only flushed on a
> GPF event.

Sounds correct. From what I learned from GPF, ADR, and eADR, there would 
still be data in WPQ even though we perform a CPU cache line flush in 
the OS.

This means we don't have a explicit method to make data puncture all 
caches and land in the media after writing. also it seems there isn't a 
explicit method to invalidate all caches along the entire path.

> 
> I remain skeptical that a software managed inter-host cache-coherency
> scheme can be made reliable with current CXL defined mechanisms.


I got your point now, acorrding current CXL Spec, it seems software 
managed cache-coherency for inter-host shared memory is not working. 
Will the next version of CXL spec consider it?
>